A guide to
VPN protocols

Long before 2020, Virtual Private Networks (VPNs) were among one of the many tools used to secure access to company networks from remote or outside the network locations. However, they took on greater significance when many more employees moved from office buildings to dining room tables. VPNs not only help with employee mobility and business continuity but can also safeguard business data and transactions amid rising cyber security threats.

These multiple uses are strong drivers behind the global VPN market's growth, estimated to be growing at 15.3% annually with a projected market size of $77.1 billion by 2026. While not necessarily household names, familiar protocols like Secure Socket Layer (SSL) and Transport Layer Security (TLS) are employed whenever you see HTTPS in a browser address bar.

A short introduction to the VPN protocols that are behind this security option will help show why it is so important, how the protocols work and the different options which might work best for your organization, especially since public internet connections have become more prevalent and accessible for everybody.

VPN protocols are a set of standard specifications for establishing a secure tunnel using encryption, verification and security for data transmission between a device (or endpoint) and another device (or endpoint). For VPNs to be most effective, protocol standards are necessary to build secure ways to transmit data over the public internet which is otherwise open and not secure for sending data.

Think of VPN sessions as tunnels, or encrypted connections that connect to a company’s private, secured network. They are used to transmit sensitive data across an unsecured “underlay” network and protect that information from malicious actors. By using VPN tunnels it allows you to transmit data so that it is not directly viewable because it is sent through an encrypted virtual tunnel which makes the data unreadable.

Since the VPN standard calls for adding encryption and transmission protocols, they can impact the overall bandwidth available to transfer data. This additional encryption overhead increases the packet size and reduces the amount of data that can be transmitted over the equivalent network transport without a VPN. While there are newer technologies that limit the impact to the network, typically a VPN cuts the available bandwidth about 20%.

To put it differently, sometimes speed and capacity outweigh the need for security. It is worth remembering that not all data transmitted over a network is sensitive or in need of being secured over a VPN. 

Different protocols are available to meet a variety of user requirements. When deciding what are the best VPN protocols for your organization, it is worth considering:

• Speed vs. privacy. Some are designed for fast and easy transmission, while others prioritize security and data privacy.
• Platform-specific vs. multi-use. Some VPN protocols are designed to work on only one platform, whereas others can work across multiple platforms. This is an important detail to keep in mind if employees are mixing and matching platforms across their devices.
• Specific vs. multiple uses. Some protocols are designed to work stand alone for specific purposes, while others can be bundled together, supporting an array of encryption methods.

Examples of use cases for common protocols include:

A site-to-site VPN connects multiple networks, such as a corporate network with multiple branch offices.

Internet Protocol Security (IPSec) authenticates and then encrypts individual IP packets. It offers a suite of security protocols that add encryption services to other protocols. This protocol is commonly used for site-to-site VPN tunnels. Layer 2 Tunnel Protocol (L2TP) is often paired with IPSec because, while it creates the tunnel and handles authentication, it doesn't have any encryption.

Mobile VPN apps can help users to protect their phones and tablets, whether they are being used on their mobile data network or on home or public Wi-Fi.

One of the most used VPN protocols in this scenario is Internet Key Exchange version 2 (IKEv2). This is because it can facilitate virtually seamless switching between mobile data and Wi-Fi because it automatically reconnects back to a VPN server whenever the connection is interrupted.

A newer protocol, WireGuard, has been described as a "game-changer" for its ability to combine fast speeds with strong encryption security. However, it will only work on devices where you can install apps. Further, it doesn't counter deep-packet inspection so may not be able to work with certain network firewalls.

Secure Socket Tunneling Protocol (SSTP) is a closed-source VPN protocol that's easy to use on Windows platforms because it's a Microsoft proprietary protocol. SSTP may be a good choice for companies that exclusively use Microsoft Windows.

OpenVPN is an open source protocol and supported by many VPN services. It bundles different protocols into a library for various encryption methods, which are beneficial for stable connections and connecting to remote servers. However, it doesn't run natively on Windows, so your organization will require third-party software if you use Windows.

Understanding what operating system or computer platform is being planned for use in the near future is critical in this decision.

Some VPN service providers use existing open-source protocols as the basis for their own proprietary tunneling protocols. For example NordVPN's NordLynx is based on WireGuard. However, they don’t offer the same level of transparency because they have added proprietary features, so users are unable to tell what has been changed from the original open-source protocol. This can make validating their level of security difficult, if not impossible.

Your needs will determine the best VPN protocols for your organization; or even if a private network is a better option. Securing data transmissions is the primary reason for using a VPN; the devices you use and the type of data you exchange will play a large role in the VPN protocol you choose.