TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 Cipher with Server 2012 R2

I was recently working on an issue where a monitoring system that runs on top of Server 2012 R2 was not able to establish a TLS handshake with web servers to check their availability. After some investigation and a ticket with Microsoft, it was determined that SCHANNEL on Server 2012 R2 does not support modern ciphers (a few posts on Stack Overflow confirms the same thing).

I find it crazy that Microsoft don’t support modern ciphers on server operating systems that are still in support (albeit extended support in this case). Surely as web servers start to upgrade their cipher suites, this is going to break stuff and everyone will be forced to upgrade to newer Server OS’s whether they are ready to or not.

The troubleshooting process with Microsoft involved the following steps:

Make sure that the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 cipher exists in the following registry locations:

KeyValueType
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Cryptography\Configuration\Local\SSL\00010002FunctionsREG_MULTI_SZ
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002FunctionsREG_SZ

As well as this, the Microsoft support engineer advised to enable strong crypto in dot net framework by configuring the following key:

KeyValueType
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft.NETFramework\v4.0.30319SystemDefaultTlsVersionREG_DWORD

Lastly, to make sure that a TLS 1.2 connection was established, the following registry keys needed to be set to enforce TLS 1.2 and disable older SSL and TLS protocols.

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\Multi-Protocol Unified Hello]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\Multi-Protocol Unified Hello\Client]
"Enabled"=dword:00000000
"DisabledByDefault"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\Multi-Protocol Unified Hello\Server]
"Enabled"=dword:00000000
"DisabledByDefault"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\PCT 1.0]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\PCT 1.0\Client]
"Enabled"=dword:00000000
"DisabledByDefault"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\PCT 1.0\Server]
"Enabled"=dword:00000000
"DisabledByDefault"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0\Client]
"DisabledByDefault"=dword:00000001
"Enabled"=dword:00000000

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0\Server]
"Enabled"=dword:00000000
"DisabledByDefault"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0\Client]
"Enabled"=dword:00000000
"DisabledByDefault"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0\Server]
"Enabled"=dword:00000000
"DisabledByDefault"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Client]
"Enabled"=dword:00000000
"DisabledByDefault"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Server]
"Enabled"=dword:ffffffff
"DisabledByDefault"=dword:00000000

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.1]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.1\Client]
"Enabled"=dword:00000000
"DisabledByDefault"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.1\Server]
"Enabled"=dword:ffffffff
"DisabledByDefault"=dword:00000000

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client]
"Enabled"=dword:ffffffff
"DisabledByDefault"=dword:00000000

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server]
"Enabled"=dword:ffffffff
"DisabledByDefault"=dword:00000000

Once these registry keys were set and after a reboot, we could do some testing. The quickest way to determine what was happening was to run a packet capture and use Internet Explorer to browse to the site that was attempting to handshake using the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 cipher. Internet Explorer uses SCHANNEL for establishing TLS connections, so this is an easier test than mucking about with your app that also uses SCHANNEL. For the capture, I wrote a quick PowerShell script to use the built in netsh tool to capture packets and convert them to .pcapng for easy analysis using Wireshark.

# NetworkCapture.ps1

function startCapture {
    param ()
netsh trace start capture=yes report=disabled
}

function stopCapture {
    param ()
netsh trace stop
}

function convertCapture {
    param ()
# Setup function variables
$ETLToolZipPath = "$env:USERPROFILE\Downloads\etl2pcapng.zip"
$CapturePath = "$env:LOCALAPPDATA\Temp\NetTraces\NetTrace.etl"
$PCAPNGPath = "$env:LOCALAPPDATA\Temp\NetTraces\$(Get-Date -Format dd-MM-yyy-hhmm)_NetTrace.pcapng"
$ETLToolPath = "$env:USERPROFILE\Downloads\etl2pcapng\x64\"    

if ($ETLToolPath) {
    Write-Host "Tool already exists, skipping..."
}
else {
# Get Windows ETL to PCAPNG tool
Invoke-WebRequest -Uri https://github.com/microsoft/etl2pcapng/releases/download/v1.4.0/etl2pcapng.zip -OutFile $ETLToolZipPath -UseBasicParsing 

# Unzip the archive
Expand-Archive -Path $ETLToolZipPath -DestinationPath $env:USERPROFILE\Downloads\ -Force
}

# Run conversion
Set-Location -Path $ETLToolPath
.\etl2pcapng.exe $CapturePath $PCAPNGPath

Write-Output -InputObject "Capture has been converted and can be found here: $PCAPNGPath"
}

Here is the result from running the three functions listed in the above script. In between the startCapture and stopCapture functions, using Internet Explorer, I browsed to the site using the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 cipher.

PS C:\> startCapture
Trace configuration:
-------------------------------------------------------------------
Status:             Running
Trace File:         C:\Users\lab\AppData\Local\Temp\NetTraces\NetTrace.etl
Append:             Off
Circular:           On
Max Size:           250 MB
Report:             Disabled

PS C:\> stopCapture
Merging traces ... done
File location = C:\Users\lab\AppData\Local\Temp\NetTraces\NetTrace.etl
Tracing session was successfully stopped.

PS C:\> convertCapture
Tool already exists, skipping...
IF: medium=eth  ID=0    IfIndex=11
IF: medium=eth  ID=1    IfIndex=14
Converted 208 frames
Capture has been converted and can be found here: C:\Users\lab\AppData\Local\Temp\NetTraces\01-02-2021-0856_NetTrace.pcapng

The results of the trace confirmed that the TLS 1.2 hello was not successfully negotiated:

From here we see the client reaches out with the TLSv1.2 hello, but further down the server returns with “Handshake Failure”.

In the Windows system event log, we also see the following SCHANNEL error:

At this point I did some additional testing using newer operating systems. I already knew that the sites in question worked using Windows 10 but I wanted to confirm with Server 2016 and Sever 2019. As expected, SCHANNEL supports the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 cipher in newer versions of Windows and Internet Explorer is able to connect. The odd thing with Internet Explorer is that it throws the following error on Server 2012 R2, which is a bit misleading as RC4 is not in use or enabled:

This page can’t be displayed

Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to https://site.example.com again. If this error persists, it is possible that this site uses an unsupported protocol or cipher suite such as RC4 (link for the details), which is not considered secure. Please contact your site administrator.

Another thing that proves the issue is with SCHANNEL is that Chromium on Server 2012 R2 works and you can access TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 negotiated sites fine. This is because Chromium does not use SCHANNEL for establishing TLS.

Ultimately, after collecting some more logs and a bit of back and forward on email, Microsoft came back stating that TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 is not supported on Windows Server 2012 R2 and that newer ciphers can be considered a windows ‘feature’ that has been introduced on newer operating systems only. They also provided this article as a reference.

If you come across this issue and your app uses SCHANNEL to establish TLS connections, you may be in for an OS upgrade.

Thanks for reading. -Jesse

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s