Invalid SNI – What is it and how to fix it

Everyone was happy. The web application uptime and response rates have been well within the Service Level Agreements for months. Uptake of the published REST Services was on the rise. Both internally and externally, more and more systems were consuming the services and the three ORDS instances behind an NGINX load balancer took it all in their stride. No long running queries or inefficient PL/SQL code locking DB resources. Everything humming along like a finely tuned engine. Then it came time to upgrade…

The ORDS product team have been operating to the usual cadence of one major release every 3 months. Although not ancient, our system is a few releases behind and not only are there interesting new features that are worth exploring but there’s always the security fixes to recent critical 3rd party Common Vulnerabilities and Exposures ( CVE ) that justifies being at the most recent stable release.

Upgrade time comes around so the test environment is setup to run through the ORDS upgrade process before applying the same to production. Then this happens for all the requests: HTTP ERROR 400 Invalid SNI

What is my SNI and how did it get invalid?

The same web server could be hosting multiple sites. In the case of ORDS, the same ORDS instance could have multiple domain name entries that route to it. Server Name Indication ( SNI ) is an extension to the Transport Layer Security (TLS) networking protocol for a client ( such as your browser ) to indicate to the server ( ORDS running standalone mode for example ) which site it wishes to talk to. To put it simply, SNI is used at the start of the secure connection handshake between client and server. The client sends a Host header in the request to indicate what site it is requesting and the server goes looking for its certificate for that server name. See https://datatracker.ietf.org/doc/html/rfc4366#section-3.1 for more details on the role of Server Name Indication in keeping communication secure.

You’ll notice from the stack trace and the error page that the Invalid SNI response does not look like a standard ORDS error page. It is in fact generated by the Eclipse Jetty embedded in ORDS when running in standalone mode.

org.eclipse.jetty.http.BadMessageException: 400: Invalid SNI

It is a feature, not a bug – Jetty 10

When running ORDS in standalone mode an embedded Eclipse Jetty server is used. Before ORDS 22.3.0 that was Jetty 9 but since that release ORDS has been using Jetty 10 and for very good reason: more secure. As mentioned in the second paragraph at the start of this article there a fixes and optimisation that are worth making the upgrade for. Jetty 10 addressed some issues in the TLS handshake. Not least Better handling for wrong SNI #5379. Beforehand it didn’t really matter what the client sent in the Host header, Jetty would return any certificate it had.

What is in that certificate anyway?

The certificate, self-signed or otherwise, holds some important information about the server and is essential in establishing trust. To see the contents of the certificate, use the keytool utility in your Java distribution.

keytool -printcert -file self-signed.pem 

Owner: CN=localhost
Issuer: CN=localhost
Serial number: 5e90f747912dd350
Valid from: Thu Feb 08 17:55:19 GMT 2024 until: Fri Feb 07 17:55:19 GMT 2025
Certificate fingerprints:
	 SHA1: FB:F2:E7:30:B5:3F:D1:8B:AC:D0:8E:C3:49:15:3B:B2:75:F1:6E:AD
	 SHA256: 54:B1:4E:6E:92:DC:7F:88:E8:66:6B:69:91:C9:E1:01:CB:69:97:4A:B7:24:BA:F9:A0:52:AC:F3:C3:15:AB:39
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 3072-bit RSA key
Version: 3

The above output shows that the Owner of this certificate has a Common Name ( CN ) value localhost which is the default standalone.https.host configuration setting value.

What to do about it

Now that you know the root cause how do you go about resolving it? It is as simple as ensuring the Common Name ( CN ) or Subject Alternative Name ( SAN ) field in the certificate matches what the client is sending the in Host header of the request. For more information on having more than one hostname in a single certificate see a previous article: Beyond the Common Name: Secure multiple domains with a single self-signed certificate

ORDS will generate a self-signed certificate if necessary when it is started in standalone mode and configured for HTTPS. In other words, the standalone.https.port is specified in the global configuration settings or --secure option is used in the ords serve command. If no certificate exists at the configured standalone.https.cert location, ORDS will generate the self-signed certificate and key file.

The Common Name used for the self-signed certificate is taken from the standalone.https.host configuration setting. If not set, the value is localhost. Traditional certificate practices often rely solely on the Common Name (CN) field. That’s the approach taken by ORDS by default when generating the self-signed certificate.

This can sometimes catch people out when they start up ORDS in secure standalone mode the first time. The self-signed certificate is generated but they may not have specified the standalone.https.host configuration setting and when they do later, they still get HTTP 400 Invalid SNI responses. That’s because the self-signed certificate is already generated so no matter what the standalone.https.host configuration setting says, the certificate will not be generated again.

Simple steps

The simplest course of action is to rename the self-signed certificate and key files and restart ORDS. It will not find the files and therefore will attempt to generate new self-signed certificates.

In summary, make sure the standalone.https.host configuration setting matches the Host header value that clients will send for requests being routed to your ORDS standalone server.