[cf-dev] Is anyone successfully using IPSec along with Windows Server 2016 (1709)?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Is anyone successfully using IPSec along with Windows Server 2016 (1709)?

aaron_huber
Administrator
We're testing out the new Windows version and everything appears to be working correctly with the exception of traffic from the routers to the containers via the NAT on the Windows cells.  The IPSec session is working between the router and the Windows host itself but there is just no response when connecting to a mapped port inside a container.  For example:

router (10.10.10.10) -> windows2016-cell (10.10.10.11) - works fine for any open port (rep, consul etc.) on the cell itself
router (10.10.10.10) -> windows2016-cell (10.10.10.11) -> container (172.30.0.10) - no response to the external port for either HTTP or SSH (for example, 40000 and 40001)

As soon as we turn off IPSec the traffic works just fine and we can access the app via the gorouter and cf ssh is connecting successfully.  The error message from the router looks like this:

curl http://10.10.10.11:40000/
curl: (7) Failed to connect to 10.10.10.11 port 40000: Connection refused

Please let me know if you were able to get this working correctly.

Aaron
_._,_._,_

Links:

You receive all messages sent to this group.

View/Reply Online (#8023) | [hidden email] | [hidden email] | Mute This Topic | New Topic

Change Your Subscription
Group Home
[hidden email]
Terms Of Service
Unsubscribe From This Group

_._,_._,_
Reply | Threaded
Open this post in threaded view
|

Re: [cf-dev] Is anyone successfully using IPSec along with Windows Server 2016 (1709)?

A William Martin
Pivotal has a commercial offering providing IPSec integration for the Windows stack. As a first stab, perhaps our public docs may contain some configuration clues for you?


On Sun, May 27, 2018 at 8:18 PM Aaron Huber <[hidden email]> wrote:
We're testing out the new Windows version and everything appears to be working correctly with the exception of traffic from the routers to the containers via the NAT on the Windows cells.  The IPSec session is working between the router and the Windows host itself but there is just no response when connecting to a mapped port inside a container.  For example:

router (10.10.10.10) -> windows2016-cell (10.10.10.11) - works fine for any open port (rep, consul etc.) on the cell itself
router (10.10.10.10) -> windows2016-cell (10.10.10.11) -> container (172.30.0.10) - no response to the external port for either HTTP or SSH (for example, 40000 and 40001)

As soon as we turn off IPSec the traffic works just fine and we can access the app via the gorouter and cf ssh is connecting successfully.  The error message from the router looks like this:

curl: (7) Failed to connect to 10.10.10.11 port 40000: Connection refused

Please let me know if you were able to get this working correctly.

Aaron

_._,_._,_

Links:

You receive all messages sent to this group.

View/Reply Online (#8025) | [hidden email] | [hidden email] | Mute This Topic | New Topic

Change Your Subscription
Group Home
[hidden email]
Terms Of Service
Unsubscribe From This Group

_._,_._,_
Reply | Threaded
Open this post in threaded view
|

Re: [cf-dev] Is anyone successfully using IPSec along with Windows Server 2016 (1709)?

aaron_huber
Administrator
After further testing with both the 1709 and 1803 versions of Windows Server 2016 it does appear that the WinNAT component being used by the Host Network Service in Windows does not play well with IPSec.  We've confirmed with both the open source Cloud Foundry and commercial PCF deployments that traffic to the containers stops working once IPSec is enabled.  We've also tested using out-of-the-box installations of both 1709 and 1803 with Docker for Windows and can replicate the same results - traffic to a container over the NAT connection to an exposed port stops working as soon as IPSec is enabled.

If anyone has successfully been able to get this working, please reply with any details.  In the mean time we'll be trying to reach out to Microsoft to confirm if this is a bug that can be fixed or if this is working as intended.

Aaron
_._,_._,_

Links:

You receive all messages sent to this group.

View/Reply Online (#8045) | [hidden email] | [hidden email] | Mute This Topic | New Topic

Your Subscription | Group Home | [hidden email] | Terms | Unsubscribe

_._,_._,_
Reply | Threaded
Open this post in threaded view
|

Re: [cf-dev] Is anyone successfully using IPSec along with Windows Server 2016 (1709)?

aaron_huber
Administrator
Just to close on this Microsoft has confirmed that this is expected for now, using IPSec along with WinNAT is not supported in 1709, 1803, or the upcoming Windows Server 2019.  They are considering it for inclusion in a future release but there is no timeline.  For now there is no way to encrypt the traffic between the gorouter and the containers on Windows which will prevent us (and others I'm sure) from moving off of Windows Server 2012 R2 for legacy .NET apps.  Hopefully Envoy will be working on Windows soon (https://github.com/envoyproxy/envoy/issues/129) so we can remove the IPSec dependency.

Aaron
_._,_._,_

Links:

You receive all messages sent to this group.

View/Reply Online (#8062) | [hidden email] | [hidden email] | Mute This Topic | New Topic

Your Subscription | [hidden email] | Unsubscribe [[hidden email]]

_._,_._,_
Reply | Threaded
Open this post in threaded view
|

Re: [cf-dev] Is anyone successfully using IPSec along with Windows Server 2016 (1709)?

A William Martin
Thanks, Aaron.

A couple of notes: The Garden Windows team has started working on contributing to the Envoy Windows support. We're betting on this as the most likely path forward for data-in-motion security, along with Istio support.

We're still planning how long the team (along with BOSH Windows) can maintain 2012 R2 support (as supporting a new Windows OS every 6 months is important but tedious). Our current thinking is to maintain it for about 12 months from now to give us time to achieve parity on the 2016 stack.

William


On Thu, Jun 14, 2018 at 12:04 PM Aaron Huber <[hidden email]> wrote:
Just to close on this Microsoft has confirmed that this is expected for now, using IPSec along with WinNAT is not supported in 1709, 1803, or the upcoming Windows Server 2019.  They are considering it for inclusion in a future release but there is no timeline.  For now there is no way to encrypt the traffic between the gorouter and the containers on Windows which will prevent us (and others I'm sure) from moving off of Windows Server 2012 R2 for legacy .NET apps.  Hopefully Envoy will be working on Windows soon (https://github.com/envoyproxy/envoy/issues/129) so we can remove the IPSec dependency.

Aaron

_._,_._,_

Links:

You receive all messages sent to this group.

View/Reply Online (#8063) | [hidden email] | [hidden email] | Mute This Topic | New Topic

Your Subscription | [hidden email] | Unsubscribe [[hidden email]]

_._,_._,_