Implementing SSL/TLS for All

 

At Squarespace, it is believed we are responsible for providing a platform on which customers and their visitors can be assured a secure browsing experience. Today, we’re proudly announcing completely free Secure Sockets Layer/Transport Layer Security (SSL/TLS) for all custom domains on our platform.

TLS and its predecessor SSL are cryptographic protocols that provide communications security over a computer network. When configured correctly, TLS makes your website more secure by ensuring privacy, data integrity, and authentication:

  • Privacy: TLS encrypts the connection between the browser and web server and securely transmits information (e.g., login credentials) to prevent unauthorized parties from eavesdropping.
  • Data integrity: TLS prevents unauthorized parties from altering data during transmission.
  • Authentication: TLS protects against impersonation by requiring web-server proof of identity.

Digital Certificates

Squarespace has supported TLS on all Squarespace subdomains for many years through the use of our wildcard certificate. The anchor for any TLS-based communication is a public key certificate—it certifies ownership of a public key by the named subject of the certificate and can be used as a form of authentication.

The process of requesting the generation of a certificate used to be a hassle, to say the least. We believe that website owners should not have to pay extra or wrestle with complex technical issues in order to generate a certificate to offer basic security to their visitors. In order to support TLS for all custom domains, we needed to generate certificates for millions of domains on behalf of our customers but we had no way to do this in an automated way.

Let’s Encrypt & ACME

Automatic Certificate Management Environment (ACME) is a protocol, launched in the fall of 2015, that automates the issuance of domain-validated (DV) certificates. ACME was a game changer for Squarespace as it allowed us to generate DV certificates for every single one of our customers’ custom domains.

  • A customer changes the Domain Name Server (DNS) settings of their domain to point to Squarespace
  • Our servers can prove to the certificate authority (CA) that we have control of the domain
  • The CA issues a DV certificate for the domain, valid for 90 days

Let’s Encrypt is a free, automated, and open certificate authority (CA), run for the public’s benefit, that launched in April 2016 and provides free certificates for TLS via the ACME process. As a proud Silver-level sponsor, Squarespace is excited to partner with Let’s Encrypt and continue supporting technological advances that help make the web a safer and more transparent place. Let’s Encrypt is currently the world's second-largest certificate authority, covering more than 10 million unique domains. Squarespace will soon make up more than 30% of Let’s Encrypt’s total certificate volume.

Let’s Encrypt’s unexpired certificate growth

ACME Client

In order to get Let’s Encrypt certificates, we needed to choose an ACME client implementation. We initially looked into Certbot, Let’s Encrypt’s reference implementation and official client—it’s an all-in-one solution, but we just needed a client. We also looked for client implementations in Java, our language of choice, but given the importance of this client in our stack, we decided not to depend on a third party. Finally, out of all the ACME challenge types, we knew that tls-sni was the best option for us and we needed strong support in the client for it. While there is a higher technical barrier compared to the http or dns challenge types, it would be more beneficial in the long run. So, we began our own Squarespace ACME client implementation based on the RFC.

Our client implementation is asynchronous and based on Netty—the Apache HttpClient or even Java’s own URLConnection seem easier on the surface, but in our opinion they’re harder to integrate efficiently. The client supports Java 8+ only and it even supports Java 9’s reactive streams. We’ve also baked a solid concurrency model into it. If there is interest, we would be happy to contribute this to the tech community.

TLS in Action

After completing the implementation of our ACME client, we kicked off the efforts to build a system that would allow us to generate certificates for all of the custom domains on our platform.

The certificate subsystem, developed by our Core Services team, consists of four new microservices, as well as databases, caches, queueing components and an administration UI. The certificate proxy service is the entry point into the certificate subsystem. Its endpoints are minimal by design and are invoked through mutual TLS connections by external processes every time a domain is added or removed from our platform. The certificate proxy service communicates with the certificate subscription service - it handles the lifecycle of the certificates, including DNS validation, initial generation, renewals and revocation. The certificate subscription service may invoke the certificate gateway service to initiate communication with Let’s Encrypt through the Squarespace ACME client.

A reverse proxy, codenamed Echo, built by our Edge Infrastructure team is responsible for terminating all TLS traffic; it is written in Java and is powered by Netty. Echo supports 20+ different cipher suites and TLS versions 1.0, 1.1, and 1.2. SSL 2.0 and 3.0 are explicitly not supported. Echo also implements the SNI extension of the TLS protocol. We don’t bundle domains into SAN certificates; each domain has its own certificate. Support for OCSP stapling, which is challenging to implement on a Java-based stack, is also baked into Echo (check out our guest post in the Let’s Encrypt blog about this topic).

Echo invokes endpoints in the certificate server service through mutual TLS connections to 1) retrieve the self-signed certificates required to complete the tls-sni ACME challenge, 2) retrieve the certificate and private keys required to terminate production TLS traffic for custom domains. The certificate server service makes use of a large caching layer to avoid lookups to the database.

Several interactions between the certificate gateway service and Let's Encrypt are required in order for a certificate to be issued. To allow for delays and retries, the certificate gateway service implements an asynchronous flow using 3 message queues.

  • Authorizations queue: To control the rate at which requests are sent to Let’s Encrypt and avoid running into rate limit errors, any new requests that enter the service are put on this queue. When requests are taken off the queue, we issue an Authorize call and configure our reverse proxy to respond to a tls-sni challenge for the domain.

  • Pending Challenge queue: We then issue a Challenge call and schedule the challenge status check on this queue. Requests on this queue will be processed and re-queued until we either get a valid status or timeout.

  • Pending Retrieval queue: After receiving a successful challenge response, we submit an Issue call. Since the resulting certificate may not be available immediately, we schedule another status check on this queue and the same re-queueing/timeout logic applies. If and when we retrieve a valid certificate, the service notifies the certificate subscription service to store the information in our databases and caching layers.

A certificate administration UI was developed due to the sheer volume of domains, certificate generation attempts, and the multiple steps involved in the process. The UI allows us to more easily follow the lifecycle of the certificate generation request for any particular domain. It also allows us to rapidly root cause issues. The most common error during the process is incorrect DNS mappings, so we also built visualizations for the A record mappings of each domain.

Lifecycle of the certificate generation request

A record mappings for a domain

At the peak of our process of generating certificates for existing domains, the certificate gateway service was submitting approximately 100 requests per second to Let’s Encrypt. The following graph shows the metrics tracking each interaction:

ACME requests to Let’s Encrypt

The things to look out for when generating certificates at scale are:

  • Rate limits: Let’s Encrypt places limits on a) the number of certificates generated for a domain per week, b) the number of pending authorization requests.
  • Connection errors: Like in any other distributed system, requests to Let’s Encrypt may not complete and the system should be able to self-recover.
  • Pending authorizations: If you submit an authorization request, but never get around to completing the challenge due to connection errors, the rate limits for pending authorizations will kick in pretty quickly and those authorizations will stick around for 7 days. We recommend persisting the authorization URLs to allow the system to recover by either completing the challenge or discarding the authorization request.
  • Retries: DNS takes time to propagate and the system should acknowledge this and automatically retry certificate generation at regular intervals.

Another way Squarespace is making the move towards a more modern and secure Internet is through perfect forward secrecy (PFS). With PFS’s secure communication protocols, past conversations cannot be retrieved or decrypted. To that end, we were careful in selecting the supported cipher suites and our reverse proxy does not have session resumption enabled.

Our implementation and configuration receives an “A” rating from Qualys SSL Labs. We could achieve an “A+” rating if we enabled the HTTP Strict Transport Security (HSTS) headers, but that is not available as part of this launch.

The Product

Every existing custom domain on our platform has been automatically issued a certificate, whether it was purchased from Squarespace Domains or from an alternate registrar. Newly added domains will be automatically issued a certificate within seconds. In order for TLS to work, ensure the DNS records of your custom domain are set up properly. To verify that TLS is protecting a page, look for a URL beginning with https:// and a green, closed padlock icon. As an added benefit, domains hosted on Squarespace may enjoy a boost in search rankings.

Forcing TLS on Squarespace is currently opt-in. We're offering two settings to control where traffic is directed:

  • Insecure (default): Visitors are able to access the website over both HTTP and HTTPS. Search engines will index the insecure HTTP version of your site.
  • Secure: Visitors accessing your site will always be redirected to a secure HTTPS link. Search engines will index the secure HTTPS version of your site.

In the near future, we will be migrating all websites to the Secure setting. The end result is that millions more domains on the Internet will be secured via TLS, and we will have helped the Internet take a huge step forward in realizing the vision of security by default. With these end-to-end security advances, Squarespace continues to change the face of the Internet by providing the latest in cyber security technologies to all customers at no additional cost. If you’re interested in joining our team, we’re hiring!

Toshi Felton