SSL connection error during handshake on Windows Server 2008 R2

Posted by Thomas on Server Fault See other posts from Server Fault or by Thomas
Published on 2012-10-11T23:41:49Z Indexed on 2012/10/12 9:40 UTC
Read the original article Hit count: 276

Filed under:

I have a Windows 2008 R2 Server that runs a HTTPS Tunneling service. The software uses a certificate that is provided via the Windows certificate store. The certificate is located in the local computer private certificates. It supports server and client authentication with signing and keyencipherment.

Cert chain

The certificate chain looks fine. It's a Thawte SSL123 certificate.

Thawte Premium Server CA (SHA1) [?e0 ab 05 94 20 72 54 93 05 60 62 02 36 70 f7 cd 2e fc 66 66]
- thawte Primary Root CA [?1f a4 90 d1 d4 95 79 42 cd 23 54 5f 6e 82 3d 00 00 79 6e a2]
  - Thawte DV SSL CA [3c a9 58 f3 e7 d6 83 7e 1c 1a cf 8b 0f 6a 2e 6d 48 7d 67 62]
    - Server certificate

Issues

Most browsers accept the certificate without any warning. But IE 7 on Windows XP SP3 and Opera 12 on OSX just report an connection error. Opera complains:

Secure connection: fatal error (552)
https://www.example.com/
Opera was not able to connect to the server, because the server does not communicate via any secure protocol known to Opera.

A connection test using openssl s_client -connect www.example.com:443 -state says:

CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
52471:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:/SourceCache/OpenSSL098/OpenSSL098-35.1/src/ssl/s23_lib.c:182:

ssldump -aAHd host www.example.com during curl https://www.example.com/ reports:

New TCP connection #1: localhost(53302) <-> www.example.com(443)
1 1  0.0235 (0.0235)  C>SV3.1(117)  Handshake
      ClientHello
        Version 3.1 
        random[32]=
          50 77 56 29 e8 23 82 3b 7f e0 ae 2d c1 31 cb ac 
          38 01 31 85 4f 91 39 c1 04 32 a6 68 25 cd a0 c1 
        cipher suites
        Unknown value 0x39
        Unknown value 0x38
        Unknown value 0x35
        TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
        TLS_RSA_WITH_3DES_EDE_CBC_SHA
        Unknown value 0x33
        Unknown value 0x32
        Unknown value 0x2f
        Unknown value 0x9a
        Unknown value 0x99
        Unknown value 0x96
        TLS_RSA_WITH_RC4_128_SHA
        TLS_RSA_WITH_RC4_128_MD5
        TLS_DHE_RSA_WITH_DES_CBC_SHA
        TLS_DHE_DSS_WITH_DES_CBC_SHA
        TLS_RSA_WITH_DES_CBC_SHA
        TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA
        TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA
        TLS_RSA_EXPORT_WITH_DES40_CBC_SHA
        TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5
        TLS_RSA_EXPORT_WITH_RC4_40_MD5
        Unknown value 0xff
        compression methods
                unknown value
                  NULL
1    0.0479 (0.0243)  S>C  TCP FIN
1    0.0481 (0.0002)  C>S  TCP FIN

Thawte provides two Java based SSL Checkers. The Legacy Thawte SSL Certificate Installation Checker and the sslToolBox. Both validate the certificate under Windows XP but report connection errors under OSX and Windows 2008 R2.

Developer IT