Why your PS5's TCP couldn't come home

GhostPort has a feature called route_via=isp. You tag a device — usually a game console or smart TV — and instead of being tunneled through our infrastructure, that one device exits via your residential IP. Streaming services that geo-fence to your home address keep working. Game matchmaking gets your real ASN. Everything else on the network stays behind the tunnel.

It shipped. UDP worked. PSN sign-in didn't.

This is the story of a netfilter bug that lived in our killswitch logic for longer than we'd like to admit, and an nftables packet-flow detail we had quietly wrong.

The symptom

A customer's PS5 was on a trusted-device exit. They could load the dashboard. They could ping the gateway. STUN-style UDP punched through to PSN voice fine. But every TCP connection sat in SYN_RECV forever. Game matchmaking timed out. Store wouldn't load. Sign-in spun.

From conntrack -L on the router:

# 31 stuck flows, all SYN_RECV, all to PSN endpoints
tcp  6 60 SYN_RECV src=192.168.1.42 dst=23.45.x.x sport=51234 dport=443 \
  src=23.45.x.x dst=<ISP-WAN-IP> sport=443 dport=51234 [UNREPLIED] mark=0

The conntrack table knew the connections existed. The kernel was tracking them. But they never advanced past SYN_RECV. Which meant: the SYN was leaving the PS5, getting to PSN, PSN was returning a SYN-ACK… and that SYN-ACK was vanishing somewhere between arriving on our WAN interface and leaving toward the PS5.

The debug chain

The diagnostic walk took a few hours and looked like this:

• tcpdump on eth0 (WAN ingress): SYN-ACKs from PSN are arriving. Good. ISP is doing its job.
• conntrack state: SYN_RECV with [UNREPLIED] on the reverse tuple. So conntrack sees the SYN go out but never sees the SYN-ACK match the existing flow. Suspicious — that's exactly what you'd see if the SYN-ACK never made it back through the netfilter chains.
• Ping from router to PS5: works perfectly. So the wlan0 TX path is fine. ARP is resolved. The Pi can absolutely send L2 frames to that device.
• tcpdump on wlan0 (LAN egress): zero SYN-ACKs heading toward the PS5. So the packets are dying between WAN ingress and LAN egress. Inside the box.
• Rule counters: the per-device "let return packets through" rule we'd written had counter=0 packets — despite roughly five thousand outbound packets having gone the other way and the killswitch's default-drop policy chewing through whatever fell off the end.
• Realization: the rule isn't matching. Not "matching and dropping" — never firing at all. Why?

The rule that didn't match

Here's the rule (paraphrased — full version in the commit linked below):

insert rule inet filter killswitch \
  oifname "wlan0" ether daddr AA:BB:CC:DD:EE:FF \
  ct state established,related counter accept

The intent: "on the LAN egress interface, if the L2 destination is the PS5's MAC and conntrack says this is a return packet, let it through." Reads fine. Looks fine. Counters say it never matches a single packet.

The mistake is the assumption that ether daddr in the FORWARD hook refers to the L2 destination the frame will be transmitted with. It does not. Not for routed traffic. Not on a router doing IP forwarding.

What's actually in the skb at FORWARD time

For an IPv4 packet being forwarded between two L3 interfaces, the kernel's path is roughly:

• ip_rcv → PREROUTING → routing decision → FORWARD
• FORWARD finishes → POSTROUTING → ip_output
• ip_finish_output2 → neighbor lookup (ARP / IPv6 ND) → neigh_output
• Egress L2 destination MAC gets written into the skb here → dev_queue_xmit → on the wire.

The next-hop MAC — the PS5's MAC, in our case — gets stamped onto the frame after the FORWARD chain has already run. At FORWARD time, what you read from ether daddr is whatever was in the L2 header when the frame arrived: the router's own ingress MAC.

So our rule said "if the destination MAC is the PS5's MAC" and the kernel said "the destination MAC is the router's WAN-side MAC, which I'll rewrite later." The rule could not have matched a return packet if it tried.

And our killswitch chain had a default drop policy. So every SYN-ACK fell off the end and into the bit bucket. UDP looked like it worked because the outbound flow primed conntrack and a related-state rule earlier in the chain caught the response before the broken rule got a chance to not-match. TCP's three-way handshake hits the killswitch differently and depended entirely on the bad rule.

The fix: conntrack mark

The right primitive for "I want the return half of this flow to be identifiable" is conntrack mark. ct mark is stored on the connection-tracking entry, not the packet, and it's preserved across both directions of a flow.

The new shape, distilled:

# 1. On outbound, mark the flow at the conntrack layer.
chain mangle_out {
  iifname "wlan0" ether saddr AA:BB:CC:DD:EE:FF \
    meta mark set 0x100 ct mark set 0x100
}

# 2. On return, match ct mark, not ether daddr.
chain killswitch {
  oifname "wlan0" ct mark 0x100 \
    ct state established,related counter accept
}

The outbound rule still uses ether saddr — and that one is fine, because for inbound packets, the L2 source is the device we care about; it's the egress side where MAC resolution hasn't happened yet that bites you. The mark goes onto the conntrack tuple, survives the return trip, and the return rule matches on something that actually exists at the right moment.

Live evidence post-fix

Within seconds of regenerating and applying the rules on the same PS5:

tcp  6 432000 ESTABLISHED src=192.168.1.42 dst=23.45.x.x sport=51234 dport=443 \
  src=23.45.x.x dst=<ISP-WAN-IP> sport=443 dport=51234 \
  [ASSURED] mark=256

mark=256 is 0x100 in decimal — conntrack confirming the flow carried the mark across both directions. ASSURED means traffic has flowed in both directions. The 31 stuck SYN_RECV entries cleared. PSN signed in. Game matchmaking connected. Done.

The part of the story we owe you

We didn't go straight from "TCP broken" to "ct mark fix" cleanly. Before we found the real cause, we shipped an extra watchdog — a process that would notice when a route_via=isp device's flows looked stuck and try to "heal" them by flushing conntrack state for that source IP.

It made the symptom worse. Now the device couldn't even get a UDP handshake to stay alive long enough to be useful, because the watchdog kept blowing away half-formed entries thinking it was helping. We reverted that watchdog before we shipped the real fix. The dead-end is in the git history.

We mention it because every clean debug story has at least one of those, and pretending otherwise is dishonest. The reason we shipped the watchdog first is that we hadn't yet figured out the root cause — so we reached for "automate a recovery" instead of "understand why." That's almost always backwards.

Read the actual fix

This is one of the reasons we made GhostPort Phantom OS source-available under Elastic License v2 — so when we tell a debug story, you can verify it. The commit is real, the diff is small, and you can trace it back to the symptom we described above.

What we'd tell our past self

• Rule counter = 0 is louder than it looks. When a rule you know should match isn't matching, the answer is almost never "the rule is right but something upstream is dropping the packet first." It's usually "the rule is wrong about what's in the skb at the moment it runs."
• Don't reach for a watchdog before you understand the bug. Recovery automation built on top of a misunderstanding can mask the symptom in some scenarios and amplify it in others. Find the cause first.
• For anything that needs to be matched across both directions of a flow, use ct mark. Not source MAC, not destination MAC, not the rare and brittle tricks involving meta iif on the return path. Conntrack mark is what it's for.