ARR Configuration for backend sites with SNI binding on IIS
Published Aug 17 2021 03:21 PM 6,654 Views
Microsoft

Recently, I worked on a unique scenario with one of our premier customers wherein they were using Application Request Routing (ARR) module of IIS as a load balancer for a web application that is hosted on two backend IIS servers.

 

Below was Customer’s architecture:

MicrosoftTeams-image (3).png

As shown in the diagram, they were using SNI Https Binding for a custom hostname (https://arr.com) on backend servers.

 

Because of the above scenario, they had added the servers to the web farm on ARR like below:

 

 

 

appcmd.exe set config -section:webFarms /[name='webfarm'].[address='server1.contoso.com'].applicationRequestRouting.hostName:"arr.com"  /commit:apphost

 

 

 

 

 

 

appcmd.exe set config -section:webFarms /[name='webfarm'].[address='server2.contoso.com'].applicationRequestRouting.hostName:"arr.com"  /commit:apphost

 

 

 

Then set the preserveHostHeader to false for the webFarm, this applies only if we are not using the same hostname which we use for ARR website. If we are using the same hostname as ARR's, then we need to set preserveHostHeader to true.

 

 

 

appcmd.exe set config -section:webFarms /[name='webfarm'].applicationRequestRouting.protocol.preserveHostHeader:"False"  /commit:apphost

 

 

 

The overall configuration of their web farm looks like below:

 

MicrosoftTeams-image (1).png

 

 

MicrosoftTeams-image.png

 

Once we test it from the client machine by firing a request to https://arr.com from any browser we were getting “502.3 – A security error occurred”

 

Then we took a network trace to understand what is really happening between ARR server and backend IIS sever with respect to TLS negotiation as the error mostly talks about abnormal behavior in TLS negotiation.

 

After analyzing the network trace, below is what we found:

 

 

 

74 1:43:44 PM 8/17/2021 0.8680192 ARR Server1 TCP TCP:Flags=......S., SrcPort=50231, DstPort=HTTPS(443), PayloadLen=0, Seq=2882094152, Ack=0, Win=64800 ( Negotiating scale factor 0x8 ) = 64800 {TCP:12, IPv6:3}
78 1:43:44 PM 8/17/2021 0.9006741 Server1 ARR TCP TCP:Flags=...A..S., SrcPort=HTTPS(443), DstPort=50231, PayloadLen=0, Seq=2784624939, Ack=2882094153, Win=65535 ( Negotiated scale factor 0x8 ) = 16776960 {TCP:12, IPv6:3}
79 1:43:44 PM 8/17/2021 0.9007579 ARR Server1 TCP TCP:Flags=...A...., SrcPort=50231, DstPort=HTTPS(443), PayloadLen=0, Seq=2882094153, Ack=2784624940, Win=517 (scale factor 0x8) = 132352 {TCP:12, IPv6:3}
80 1:43:44 PM 8/17/2021 0.9010972 ARR Server1 TLS TLS:TLS Rec Layer-1 HandShake: Client Hello. {TLS:14, SSLVersionSelector:13, TCP:12, IPv6:3}
92 1:43:44 PM 8/17/2021 0.9332598 Server1 ARR TCP TCP:Flags=...A...., SrcPort=HTTPS(443), DstPort=50231, PayloadLen=0, Seq=2784624940, Ack=2882094670, Win=261 (scale factor 0x8) = 66816 {TCP:12, IPv6:3}
99 1:43:44 PM 8/17/2021 0.9922412 Server1 ARR TLS TLS:TLS Rec Layer-1 HandShake: Server Hello.; TLS Rec Layer-2 Cipher Change Spec; TLS Rec Layer-3 SSL Application Data {TLS:14, SSLVersionSelector:13, TCP:12, IPv6:3}
100 1:43:44 PM 8/17/2021 0.9922412 Server1 ARR TCP TCP:[Continuation to #99]Flags=...A...., SrcPort=HTTPS(443), DstPort=50231, PayloadLen=1220, Seq=2784626160 - 2784627380, Ack=2882094670, Win=261 (scale factor 0x8) = 66816 {TCP:12, IPv6:3}
101 1:43:44 PM 8/17/2021 0.9922412 Server1 ARR TCP TCP:[Continuation to #99]Flags=...A...., SrcPort=HTTPS(443), DstPort=50231, PayloadLen=1220, Seq=2784627380 - 2784628600, Ack=2882094670, Win=261 (scale factor 0x8) = 66816 {TCP:12, IPv6:3}
102 1:43:44 PM 8/17/2021 0.9922412 Server1 ARR TCP TCP:[Continuation to #99]Flags=...AP..., SrcPort=HTTPS(443), DstPort=50231, PayloadLen=628, Seq=2784628600 - 2784629228, Ack=2882094670, Win=261 (scale factor 0x8) = 66816 {TCP:12, IPv6:3}

 

 

 

 

Frame number 80 has the Client Hello and below is how it looks like:

 

 

 

Frame: Number = 80, Captured Frame Length = 641, MediaType = WiFi
+ WiFi: [Unencrypted Data] .T....., (I)
+ LLC: Unnumbered(U) Frame, Command Frame, SSAP = SNAP(Sub-Network Access Protocol), DSAP = SNAP(Sub-Network Access Protocol)
+ Snap: EtherType = IPv6, OrgCode = **************
+ Ipv6: Next Protocol = TCP, Payload Length = 537
+ Tcp: Flags=...AP..., SrcPort=50231, DstPort=HTTPS (443), PayloadLen=517, Seq=2882094153 - 2882094670, Ack=2784624940, Win=517 (scale factor 0x8) = 132352
  TLSSSLData: Transport Layer Security (TLS) Payload Data
- TLS: TLS Rec Layer-1 HandShake: Client Hello.
  - TlsRecordLayer: TLS Rec Layer-1 HandShake:
     ContentType: HandShake:
   + Version: TLS 1.0
     Length: 512 (0x200)
   - SSLHandshake: SSL HandShake ClientHello(0x01)
      HandShakeType: ClientHello(0x01)
      Length: 508 (0x1FC)
    - ClientHello: TLS 1.2
     + Version: TLS 1.2
     + RandomBytes:
       SessionIDLength: 32 (0x20)
       SessionID: Binary Large Object (32 Bytes)
       CipherSuitesLength: 32
     + TLSCipherSuites: Unknown Cipher
     + TLSCipherSuites: Unknown Cipher
     + TLSCipherSuites: Unknown Cipher
     + TLSCipherSuites: Unknown Cipher
     + TLSCipherSuites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 { 0xC0,0x2B }
     + TLSCipherSuites: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256   { 0xC0,0x2F }
     + TLSCipherSuites: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 { 0xC0,0x2C }
     + TLSCipherSuites: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384   { 0xC0,0x30 }
     + TLSCipherSuites: Unknown Cipher
     + TLSCipherSuites: Unknown Cipher
     + TLSCipherSuites: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA      { 0xC0,0x13 }
     + TLSCipherSuites: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA      { 0xC0,0x14 }
     + TLSCipherSuites: TLS_RSA_WITH_AES_128_GCM_SHA256         { 0x00, 0x9C }
     + TLSCipherSuites: TLS_RSA_WITH_AES_256_GCM_SHA384                      { 0x00, 0x9D }
     + TLSCipherSuites: TLS_RSA_WITH_AES_128_CBC_SHA            { 0x00, 0x2F }
     + TLSCipherSuites: TLS_RSA_WITH_AES_256_CBC_SHA            { 0x00, 0x35 }
       CompressionMethodsLength: 1 (0x1)
       CompressionMethods: 0 (0x0)
       ExtensionsLength: 403 (0x193)
     - ClientHelloExtension: Unknown Extension Type
        ExtensionType: Unknown Extension Type
        ExtensionLength: 0 (0x0)
     - ClientHelloExtension: Server Name(0x0000)
        ExtensionType: Server Name(0x0000)
        ExtensionLength: 19 (0x13)
        NameListLength: 17 (0x11)
        NameType: Host Name (0)
        NameLength: 14 (0xE)
        ServerName: server1.contoso.com
     + ClientHelloExtension: Unknown Extension Type
     + ClientHelloExtension: Renegotiation Info(0xFF01)
     + ClientHelloExtension: Elliptic Curves(0x000A)
     + ClientHelloExtension: EC Point Formats(0x000B)
     + ClientHelloExtension: SessionTicket TLS(0x0023)
     + ClientHelloExtension: Unknown Extension Type
     + ClientHelloExtension: Status Request(0x0005)
     + ClientHelloExtension: Signature Algorithms(0x000D)
     + ClientHelloExtension: Unknown Extension Type
     + ClientHelloExtension: Unknown Extension Type
     + ClientHelloExtension: Unknown Extension Type
     + ClientHelloExtension: Unknown Extension Type
     + ClientHelloExtension: Unknown Extension Type
     + ClientHelloExtension: Unknown Extension Type
     + ClientHelloExtension: Unknown Extension Type
     + ClientHelloExtension: Unknown Extension Type

 

 

 

Interestingly, the “ServerName” field of ClientHelloExtensions “Server Name” is set to “server1.contoso.com” although we have set an alternate “hostName” for that particular server (server1.contoso.com).

 

So, this is the field used for SNI to work on IIS. Because there was another binding for this hostname (server1.contoso.com) on the backend IIS server (server1.contoso.com) on a different site, we were getting a different certificate in the Server Hello because of which we were seeing a "502.3 – A security error has occurred". It is the same scenario for server2.contoso.com as well.

 

We were expecting the TLS serverName in Client Hello to be the same as the hostName field that is set for the backend servers added (arr.com).

 

We checked the ARR source code internally to understand why this behavior and understood that this behavior is by design.

 

As per the source code, we establish TLS connection with the backend server using the “address” property we provide to the backend server in web farm settings.

 

Now, logically speaking when we have multiple servers in the backend and when we want to use the same custom hostname, it is not going to be possible for ARR to route the traffic.

 

We need unique identifier when it comes to addressing the backend server and address value is unique.

 

It’s not like SNI is not totally supported, it is supported but not in a way current configuration are.

 

So what’s the solution for this?

 

We need to make use of different hostnames for different backend servers, add them to address property and configure SNI for those hostnames with valid certificate in the backend servers to get this to work.

 

(Otherwise, we will have to move away from SNI and go back to legacy bindings which will restrict the number of bindings on the servers)

 

For example, let's say we have two servers "server1.contoso.com" with SNI binding "https://arr1.com" and "server2.contoso.com" with SNI binding "https://arr2.com" behind the load balancer "loadbalancer.contoso.com".

 

The external interface of ARR load balancer will be "https://xyz.com ". This can be either a SNI binding or legacy binding on the site that we have configured to use.

 

ExampleImage.png

 

If this is a new setup, you would need to do the below:

 

1. Right-click on Server Farms and click on Create Server Farm

2. Enter the desired server farm names.

3. Post this add the desired server names in the Server address section. We will consider it as arr1.com and arr2.com ( as shown in the above figure )

4. Post this you will be prompted to check if the URL rewrites rule can be added for this server farm. Procced with yes

 

After this you will see something like below:

 

 

Kiran_Angadi_0-1632690118135.png

 

 

Once this is done, run the below appcmd command to ensure preserveHostHeader is set to False.  This will ensure that we don't pass xyz.com as the host header and instead pass the respective hostname that we have added as a host header.

 

 

 

appcmd.exe set config  -section:webFarms /[name='webfarm1'].applicationRequestRouting.protocol.preserveHostHeader:"False"  /commit:apphost

 

 

 

If you are performing this on an existing setup, you need to follow the below steps:

1. Open the configuration editor at the server level

2. Chose the webFarms which appears as the last option. Click on the ellipsis in front of (collection) and you would see something like below:

 

Kiran_Angadi_1-1632690157291.png

 

3. Expand the applicationRequestRouting and then protocol and set the preserverHostHeader to true.

 

Kiran_Angadi_2-1632690210265.png

 

4. Once that is done, click on the ellipsis in front of (Collection). You would see something like below:

Kiran_Angadi_3-1632690236392.png

 

5. Change the address of both the servers accordingly

6. Once this is done, close all the pop-up windows and hit on Apply on the right top corner of the screen

 

Hope this helps! Happy Learning :)

 

2 Comments
Co-Authors
Version history
Last update:
‎Sep 26 2021 02:37 PM
Updated by: