Certopolis

CompTIA Security+ (SY0-801)

IPsec vs TLS VPN: tunnel mode, transport mode, split tunneling

IPsec protects traffic at the network layer, using ESP to encrypt and IKE to negotiate keys, and is used for both site-to-site and remote access VPNs. A TLS VPN, still often called an SSL VPN, carries traffic inside TLS, usually on port 443. Either way, the VPN protects traffic only between the client or branch and the VPN gateway.

IPsec parts and modes

IPsec has two main parts. Encapsulating Security Payload (ESP) carries the encrypted, integrity-protected traffic. Internet Key Exchange (IKE) authenticates the endpoints and negotiates the session keys. The older Authentication Header (AH) gives integrity without encryption, and NIST SP 800-77 Rev. 1 no longer recommends it.

In tunnel mode, ESP encrypts the whole original packet, including its IP header, and adds a new outer IP header that names the tunnel endpoints, such as the two gateways. An eavesdropper sees only those gateway addresses. Tunnel mode is used for gateway-to-gateway and remote access VPNs and works through NAT.

In transport mode, ESP keeps the original IP header and encrypts only the payload, so the real source and destination stay visible. It is mostly used host to host inside data centers and does not work through NAT.

TLS VPNs

TLS has a handshake protocol that authenticates the parties and agrees keys, and a record protocol that protects the traffic with those keys (RFC 8446). A TLS VPN is harder to block than IPsec because it can run on any port, usually 443. NIST notes that over TCP it can slow badly when packets are lost, and that it usually runs as an application, so it is slower than kernel-based VPNs such as IPsec.

What a VPN does not protect

NIST SP 800-46 Rev. 2 says a tunnel protects traffic between the client device and the VPN gateway but not between the gateway and internal servers. Encrypted traffic also cannot be read by firewalls or intrusion detection systems, so sensors that need to inspect it go behind the gateway, where it is decrypted.

Split tunneling sends only traffic for internal resources through the VPN and lets everything else go straight to the internet. It reduces load on the gateway, but the organization can no longer inspect or protect that other traffic. NIST advises considering disabling it for teleworkers on untrusted networks such as hotspots.

Where the exam catches people

  • Believing a site-to-site tunnel hides every address: in tunnel mode the outer header shows the two gateway addresses; only the inner addresses, ports and payload are encrypted.
  • Believing the VPN encrypts traffic all the way to the server: protection ends at the VPN gateway, so the internal leg needs its own encryption.
  • Believing split tunneling leaves internal traffic unencrypted: internal traffic still goes through the tunnel; the risk is the internet traffic the organization can no longer inspect.
  • Believing AH encrypts data: AH gives integrity only, and NIST no longer recommends it; ESP provides encryption.
  • Believing transport mode is the site-to-site choice: gateway-to-gateway VPNs use tunnel mode, which also works through NAT.

Written from CompTIA's documentation

Try 3 questions on this

Pick an answer to see why each option is right or wrong.

  1. Two branch offices connect their networks through a site-to-site IPSec VPN. An attacker captures packets on the internet link between the two sites. Which of the following can the attacker still read from the captured packets?

    Right. In the game that’s Renown for your Security Architecture building.

    Not that one: A is right. In the game your Security Architecture building would lose a little Renown, and this question would come back later.

    Show the full explanation and doc quotes

    Only the gateway addresses in that outer header stay readable.

    Written from nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-77r1.pdf

  2. A company enables split tunneling on the VPN client that its teleworkers use from home and hotel networks. Which of the following is the main security drawback of this change?

    Right. In the game that’s Renown for your Security Architecture building.

    Not that one: B is right. In the game your Security Architecture building would lose a little Renown, and this question would come back later.

    Show the full explanation and doc quotes

    That prevents the organization from examining much of the teleworkers' network traffic and from protecting that traffic's confidentiality and integrity.

    Written from nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-46r2.pdf

  3. Teleworkers open documents on a company's internal file server through a remote access VPN. A new policy requires the documents to be encrypted on every network link they cross. Which of the following must the company add?

    Right. In the game that’s Renown for your Security Architecture building.

    Not that one: C is right. In the game your Security Architecture building would lose a little Renown, and this question would come back later.

    Show the full explanation and doc quotes

    The link from the gateway to the file server is unencrypted until the company adds encryption there.

    Written from nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-46r2.pdf