Support `ClientCertificateContext` on `SslClientAuthenticationOptions` when connecting to TLS server

[rmaffitsancsoft]

Describe the feature request

I'm trying to connect to an EMQX MQTT server with mTLS certificate authentication from a .NET hosted worker running on Windows and Linux.

Both the server and client certificates are signed by an intermediate CA.

The intermediate CA is signed by a self-signed root CA.

The MQTT server is configured to authenticate using the root CA.

Both root CA and intermediate CAs are not in the trusted root store on either Windows or Linux.

EMQX listener config:

listeners.ssl.default {
  bind = "0.0.0.0:8883"
  ssl_options {
    cacertfile = "/tls/server/ca.crt" // Root CA certificate
    certfile = "/tls/server/tls.crt" // Server + intermediate chain certificate
    keyfile = "/tls/server/tls.key"
    gc_after_handshake = true
    handshake_timeout = 5s
    verify = verify_peer
    fail_if_no_peer_cert = true
  }
}

Files used:

• client.crt - Client + intermediate chain certificate (Read into a string variable clientCrt)
• client.key - Client private key (Read into a string variable clientKey)
• ca.crt - Root CA certificate (Read into a string variable rootCA)

Client options:

var mqttFactory = new MqttClientFactory();

var caChain = new X509Certificate2Collection();
caChain.ImportFromPem(rootCA);

var clientCert = X509Certificate2.CreateFromPem(clientCrt, clientKey);

var clientCerts = new X509Certificate2Collection();
clientCerts.Add(new X509Certificate2(clientCert.Export(X509ContentType.Pfx))); // Workaround to make ephemeral keys work on Windows
clientCerts.ImportFromPem(clientCrt); // This technically adds the client cert without private key to the collection, doesn't seem to break anything but should probably be cleaned up

using var mqttClient = mqttFactory.CreateMqttClient();

var mqttClientOptions = new MqttClientOptionsBuilder()
    .WithTcpServer("mqtt.domain.net", 8883)
    .WithTlsOptions(new MqttClientTlsOptionsBuilder()
        .WithIgnoreCertificateRevocationErrors()
        .WithClientCertificates(clientCerts)
        .WithTrustChain(caChain)
        .Build())
    .Build();

var connectionResult = await mqttClient.ConnectAsync(mqttClientOptions);

Testing with MQTTNet 5.0.1.1416 installed from Nuget, I get the following exception on Linux:

MQTTnet.Adapter.MqttConnectingFailedException: Error while authenticating. The decryption operation failed, see inner exception.
  ---> MQTTnet.Exceptions.MqttCommunicationException: The decryption operation failed, see inner exception.
  ---> System.IO.IOException: The decryption operation failed, see inner exception.
  ---> Interop+OpenSsl+SslException: Decrypt failed with OpenSSL error - SSL_ERROR_SSL.
  ---> Interop+Crypto+OpenSslCryptographicException: error:0A000418:SSL routines::tlsv1 alert unknown ca
    --- End of inner exception stack trace ---
    at Interop.OpenSsl.Decrypt(SafeSslHandle context, Span`1 buffer, SslErrorCode& errorCode)
    at System.Net.Security.SslStreamPal.DecryptMessage(SafeDeleteSslContext securityContext, Span`1 buffer, Int32& offset, Int32& count)
    --- End of inner exception stack trace ---
    at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)
    at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
    at MQTTnet.Implementations.MqttTcpChannel.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
    at MQTTnet.Adapter.MqttChannelAdapter.ReadFixedHeaderAsync(CancellationToken cancellationToken)
    at MQTTnet.Adapter.MqttChannelAdapter.ReceiveAsync(CancellationToken cancellationToken)
    at MQTTnet.Adapter.MqttChannelAdapter.ReceivePacketAsync(CancellationToken cancellationToken)
    --- End of inner exception stack trace ---
    at MQTTnet.Adapter.MqttChannelAdapter.WrapAndThrowException(Exception exception)
    at MQTTnet.Adapter.MqttChannelAdapter.ReceivePacketAsync(CancellationToken cancellationToken)
    at MQTTnet.MqttClient.Receive(CancellationToken cancellationToken)
    at MQTTnet.MqttClient.Authenticate(IMqttChannelAdapter channelAdapter, MqttClientOptions options, CancellationToken cancellationToken)
    --- End of inner exception stack trace ---
    at MQTTnet.MqttClient.Authenticate(IMqttChannelAdapter channelAdapter, MqttClientOptions options, CancellationToken cancellationToken)
    at MQTTnet.MqttClient.ConnectInternal(IMqttChannelAdapter channelAdapter, CancellationToken cancellationToken)
    at MQTTnet.MqttClient.ConnectAsync(MqttClientOptions options, CancellationToken cancellationToken)
    at MQTTnet.MqttClient.ConnectAsync(MqttClientOptions options, CancellationToken cancellationToken)
    at mqtttest.Worker.ExecuteAsync(CancellationToken stoppingToken) in /home/rmaffit/git/mqtttest/Worker.cs:line 169
    at Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(BackgroundService backgroundService)

Running the same code/configuration on Windows leads to handshake errors, I was able to get it to work by adding both the root CA and intermediate CA to the trusted/intermediate stores but that is not a desirable solution, and it still leaves Linux non-functional.

Testing the same certificate + key files using MQTT Explorer and MQTTX clients were able to authenticate to the server without issue.

After much testing and reviewing the docs, comments and other issues I came to the conclusion that the intermediate certificate that was part of the clientCerts collection being passed to the connection options was not being sent to the server when initiating the TLS connection.

Further digging revealed a somewhat recently added option to SslClientAuthenticationOptions that allows for explicit control over the certificate chain used to connect to the server.

Which project is your feature request related to?

• Client
• ManagedClient

Describe the solution you'd like

Update the IMqttClientCertificatesProvider interface and default implementation to include the following new method:

SslStreamCertificateContext GetClientCertificateContext();

Update the MqttTcpChannel implementation to call GetClientCertificateContext() and add to SslClientAuthenticationOptions.

It may also make sense to add some overload methods to WithClientCertificates in MqttClientTlsOptionsBuilder with some constructor argument changes to support passing the client cert + additional certificates separately.

I hacked together the following implementation that makes some assumptions about the ClientCertificates being passed to the connection TLS options builder (there is at least 1 certificate with a private key, the first certificate with the private key is the client certificate, any additional certificates that are a part of the certificates collection are added to the certificate context). I have confirmed this works on both Windows and Linux:

public SslStreamCertificateContext GetClientCertificateContext()
{
    if(_certificates != null)
    {
        X509Certificate2 clientCertificate = null;
        X509Certificate2Collection additionalCertificates = new X509Certificate2Collection();

        foreach(var certificate in _certificates)
        {
            if(certificate is X509Certificate2 certificate2)
            {
                if(clientCertificate == null && certificate2.HasPrivateKey)
                {
                    clientCertificate = certificate2;
                }
                else
                {
                    additionalCertificates.Add(certificate2);
                }
            }
        }

        if(clientCertificate != null)
        {
            return SslStreamCertificateContext.Create(clientCertificate, additionalCertificates);
        }
    }

    return null;
}

I have not tested but I suspect the websocket channel has similar issues and a similar solution could be used.

Describe alternatives you've considered

Add a new SslStreamCertificateContext property to MqttClientTlsOptions and add a new method WithClientCertificateContext to MqttClientTlsOptionsBuilder and pass the configuration through similar to the other TLS options in the MqttTcpChannel class. I originally tested with this approach and it worked but the API is confusing because it conflicts with WithClientCertificates which may lead to further confusion.

Additional context

dotnet/runtime#71194

I am happy to take up this issue and submit a PR with whichever direction makes the most sense.

[SilvioMessi]

Hello @rmaffitsancsoft , I'm experiencing the exact same issue with the "tlsv1 alert unknown ca" error.
Would it be possible for you to share a complete example of how to use the GetClientCertificateContext function you mentioned in your comment?