Verizon 3G PPP disconnects due to non-NATted packets

When connecting a Linux system to Verizon's 3G network using PPP, I noticed that there were frequent disconnects caused by Verizon terminating the connection.  It took a lot of investigating to figure out the problem, and even though I figured this out last July I wanted to post it online for posterity in case it helps someone else.

I found that the Linux kernel netfilter was not NATting (masquerading) packets that were in an invalid state.  There is some finger pointing between the kernel people and the netfilter people in discussing whose fault this problem is, but the fact is that the connection tracker may allow some packets through as RELATED, ESTABLISHED even though they are in an invalid state.  These invalid packets are not NATted, and after Verizon gets 10 invalid packets on a PPP connection, they terminate the connection.

The problem was that the Linux kernel TCP CLOSE_WAIT connection tracker timeout was too long, which causes the TCP state machine to lose/forget the state even though the connection tracker is maintaining the RELATED,ESTABLISHED connection.  Packets associated with a TCP connection in an unknown or invalid state don't get NATted even though they still may get forwarded.  The non-NATted packets can then leak out of the Linux system, and run afoul of Verizon's network security.

The bottom line is that in the iptables rules, you have to DROP packets that are in an INVALID state.  That way the packets will get dropped and won't show up on the Verizon PPP network interface un-NATted.  The iptables rule might look something like this:

iptables -t filter -I FORWARD -p tcp -m state --state INVALID -j DROP

Verizon's tech support was not helpful in resolving this issue other than to confirm that they were dropping the connection due to packets that had not been NATted.  The following writeup is helpful to explain the problem in more detail:

http://www.smythies.com/~doug/network/iptables_notes/index.html

An Important Mentor

With regards to the discipline of software engineering, and the profession of engineering in general, I'd like to say that my friend and former colleague Don Murray, P.Eng., was a key mentor and influence in my life particularly with regards to professional development.  Don is a senior software engineer in Calgary, Alberta, and his web site is http://dmurray.ca/ .  This is just a shout-out and thank you to Don.

Welcome

I'm Jim Ockers, and this is my personal/professional web site.  I'm a computer systems and software/network engineer living in San Diego, California.  Prior to being here, I lived and worked in Grand Junction, Colorado and Calgary, Alberta.  My background is in upstream oil & gas information technologies and within that sector I've been involved in:

  • IT, systems administration, and systems programming;
  • network and communications engineering and design, including satellite network systems;
  • computer systems and instrumentation product and systems design, including industrial & embedded systems;
  • teaching, mentoring and leadership of individuals and groups

I'm a registered/licensed professional engineer in both Alberta and Colorado.  I also was the President & Chairman of the Board of a large condominium corporation in Calgary, which was a lot like being the mayor of a small city.  More information about my professional credentials and background, including curriculum vitae, will be found on the Professional/CV pages.

Welcome, and thanks for visiting.

Jim