Let's Encrypt goes TITSUP

Let’s Encrypt suffered from a major service disruption today leaving users unable to access various services. The cause of the problem seems to have been an update to Boulder (ACME CA) which has since been reversed.

Even though the inability to issue certificates was surely a nuisance, the biggest problem might have been a nonresponding OCSP responder. In short, if an OCSP responder is unable to return the certificate status for your request, you might experience the following error:

Sec error OSCP try server later

On the webserver side of things the error log quickly filled up with these entries:

[ssl:error] (70007)The timeout specified has expired: [client 151.29.xx.xx:] AH01985: error reading response from OCSP server
[ssl:error] AH01941: stapling_renew_response: responder error

Anyhow, thankfully their services seem to have been mostly restored according to their status page.