The Ubiquiti Diaries: A Site-to-Site VPN Story
by Ganesh T S on December 21, 2022 8:00 AM EST- Posted in
- Networking
- Ubiquiti Networks
- UniFi
- VPN
The CGNAT Spanner in the Works
Carrier-Grade NAT (CGNAT) is used by ISPs to circumvent the lack of IPv4 addresses available for allocation to their customers. As more and more devices come online and connect to the Internet directly, it is not surprising that there is a dearth of IPv4 addresses to hand out. CGNAT is mostly encountered with WAN connections enabled by mobile service providers (think of smartphones connecting to a 4G or 5G network). However, many wired ISPs (such as Airtel in India) have also started rolling out CGNAT. The easiest way to identify if one is behind a CGNAT is to search for one's IP address on Google. Google helpfully displays your public IP address as the first result. If this address is different from the WAN IP reported by your gateway (and that is usually between 100.64.0.0/16 and 100.127.0.0/16), you are likely behind a CGNAT. This is different from a local IP gathered by a gateway placed behind ISP-supplied equipment that acts as a DHCP server itself.
CGNAT is rarely a problem for regular web browsing and content consumption. Problems start cropping up when dealing with services that require opening up ports on the gateway for bidirectional communication - such as VoIP services, multi-player gaming, and VPN setups. Even simple things taken for granted like systems in the network being able to communicate with a remote NTP server (for time synchronization) are not possible. Since the NAT happens at the ISP end, it is not possible for port forwarding to be configured either. Some services manage to find their way around CGNAT using STUN / TURN, but power users end up finding one roadblock after another. On the whole, CGNAT is a pain in the neck for users accustomed to obtaining public-facing WAN IPs.
NTP Synchronization Attempt behind CGNAT, and the 'Solution' (Using a 'Local' NTP Server)
Ideally speaking, configuring the Site-to-Site Manual IPSec VPN on the USG Pro 4 (having a public WAN IP) with a remote server address of 0.0.0.0, and providing the USG Pro 4's WAN IP as the remote server address on the UDM side (behind the CGNAT) should have worked by allowing the CGNAT side to initiate the site-to-site connection with the server. I did try out this suggestion made by an Ubiquiti employee many years ago - however, the initial key exchange / handshake process appeared to be break down with the response from the USG Pro 4 apparently getting dropped by the UDM's ISP.
An online search for enabling site-to-site VPNs with one site behind a CGNAT brought up two sets of results - one suggesting a connection via an intermediate VPS (cloud server hosted by a 3rd party), and the other trying to make the public WAN IP-possessing side to act as an OpenVPN server and the CGNAT side to act as a client and initiate the connection. I was not interested in bringing a VPS into the picture, and my trials with OpenVPN did not yield any promising results.
At this point, I realized that my Android phone was able to open up a Teleport connection with the UDM despite the gateway going behind CGNAT. So, my first attempt was to get a system to connect to the Teleport VPN and route traffic from specific devices through the system's Teleport VPN connection.
Facebook
Twitter
RSS
35 Comments
View All Comments
bradh352 - Wednesday, December 21, 2022 - link
First thing that comes to mind is why you didn't attempt to use ipv6 addresses to create the ipsec vpn? I know comcast/xfinity supports ipv6, and I'd have to imagine anyone deploying CGNAT for ipv4 is providing a public ipv6 address, thus negating any of the issues described.Typically these ipsec vpn sessions are in tunnel mode which means they can transport both ipv4 and ipv6 packets, even if the public ips being used are only ipv4.
Maybe ubiquiti doesn't support this in their UI for some reason? The underlying system should be capable of this though (strongswan afaik).
edzieba - Wednesday, December 21, 2022 - link
"I'd have to imagine anyone deploying CGNAT for ipv4 is providing a public ipv6 address"Sadly, such a sane and customer-oriented approach remains firmly in your imagination for the vast majority of ISP CGNAT deployments. Most commonly, IPv6 will just not be available at all.
ganeshts - Wednesday, December 21, 2022 - link
Airtel does provide an IPv6 address with their CGNAT configuration. Sadly, there is no IPv6 support on the Comcast front over here in the US, and Ubiquiti doesn't support IPv6 in their VPN configuration either (at least from the web UI perspective).ViRGE - Wednesday, December 21, 2022 - link
"Sadly, there is no IPv6 support on the Comcast front over here in the US"Huh? Comcast was one of the very first major US ISPs to implement IPv6. They've been running a full dual stack implementation for nearly a decade now.
https://web.archive.org/web/20160329232139/http://...
cgull.at - Wednesday, December 21, 2022 - link
I was amused to see that the IPv6 post I was contemplating was ninja-ed before birth by the very first post.I suspect the reason Ganesh isn't seeing IPv6 is his "ancient cable modem". Very likely it's not DOCSIS 3.0 or later and doesn't do IPv6. The DOCSIS 3.0 standard was released in 2006, 3.1 in 2013. Upgrade, already!
cgull.at - Thursday, December 22, 2022 - link
Also: Comcast was one of the major leaders and instigators of "World IPv6 Day". That was back in January 2012. As I recall, somewhere around 50% of their customers were IPv6-enabled then.Why? They were running out of IPv4 addresses to give to customers, which also explains why Ganesh is seeing CGNAT now.
My ISP had (and probably still has) IPv4 addresses in reserve. They haven't enabled IPv6 for consumer internet service yet.
dersteffeneilers - Saturday, December 24, 2022 - link
with my ISP over in Germany, you can use both IPv4 with CGNAT and IPv6, but you only get an IPv6 address if you already have an IPv4 one. Ridiculous, I can't imagine a technical reason for that.Leeea - Thursday, December 22, 2022 - link
Nobody in their right mind uses ipv6 unless they absolutely have to.They really should come up with another standard that is less ideologically pure and way more practical.
ballsystemlord - Thursday, December 22, 2022 - link
Could you expand on that a bit more Leeea?It's unclear to me why an IP addressing scheme would be impractical as a result of ideological purity.
jack21159 - Thursday, December 22, 2022 - link
IPv6 was made for ultra-nerd and it's difficult to understand.I mean, IPv4 still is a learning curve, but at least it's easier to understand. Most people don't know how to segment a network (/23 , /24) or do custom routes, but that's fine you can use it even if you don't understand all the concept.
IPv6 by contrast, no. a course is needed to understand that.
they could have just added a few Bytes to the standard 4 Bytes scheme (ie 255.255.255.255.255.255 for exemple) But nooo let use hexadecimal, something than only computers and ultra nerds understand !