1774297200 english networking

Fixing the IP shortage using application level routing protocols

IPv4 addresses has been run out (or more like wasted), and the solution is IPv6. But it's 2026 and none of the ISP's I had so far offered IPv6. I didn't forced the issue either. But I think this issue was solved long ago. Just no one seems to notice it.

Let me explain. At the bottom there is the link layer. That's where computers are hooked up to the same physical medium (or bus), one computer transmits, the others receive and if the network card finds the packet relevant it shows it to the operating system. This is called a network segment. In this layer the communication uses MAC addresses, and has nothing to do with IP. At first all computers were hooked up to the same physical medium, but later they were plugged into a switch which is a smarter device that knows which computer to send the packet to after it learns the MAC address of them, otherwise it broadcasts the packet on all ports like in the old days. But still the number of computers in a single segment is limited. Even with switches, the maximum number is around 1000 computers per segment. If you want more devices, then you need more segments.

To make communication possible between different segments, you need a computer that have 2 network cards such that one is connected to one segment, and the other is connected to the other. This computer is the router. In order to route packets, knowing the MAC address of the destination computer is not sufficient, you also need to know the identifier of the network segment where the host is and you also need to know how to get there. That's what routing tables do: they tell which computer you need to send the packet to in the current segment that knows how to send the packet towards the destination segment. If the destination is outside the current segment, the routing table defines the computer to send the the packet to for forwarding. If the destination is inside the current segment, the routing table says that the host is on link and you can send the packet to them directly. Routing tables also often have a default entry that defines the computer to send all traffic to when we don't know a better suited computer for the routing.

In case of IPv4 all of these details are neatly hidden, and it took me more than a decade to realize what is under the hood. When you configure an IP you have the network mask which essentially designates certain number of bits from upper end of the address as the network identifier, then the lower bits become the device identifier. The correspondence between these device identifiers and MAC addresses are created by the DHCP and ARP protocols. Or in case IPv6 DHCPv6 and NDP, or in case of SLAAC it's done using a mathematical formula. DHCP assigns the device identifier to your MAC address and lets your computer also learn the network identifier too. The device identifier combined with the network identifier forms the IP address. ARP is the reverse. One computer broadcasts into the segment that "who has this x.x.x.x IP" and then the machine that has it, answers. In case of SLAAC the whole DHCP and ARP can be avoided because the device part is calculated from the MAC address, while the network identifier is given by a router advertisement.

Note that the packet has 4 addresses. On the link layer there are 2 MAC addresses that are valid in the current segment. On the network layer there are 2 IP addresses that encode the network identifier and device identifier for both the source and destination. Each time when the packet passes through a router the MAC addresses change, but the IP addresses remain constant.

Basically the routing and addressing protocol described above solves the problem of having only a limited number of computers in a network segment, by having multiple segments and let computers that are connected to multiple segment forward traffic through them. The IPv4 address exhaustion poses a similar problem: due to address space limitations, the number of computers on the internet is limited. So the idea is: can we make multiple internets then use a high-network protocol to send traffic through them just like the way it happens between the network and link layers? A private network is a separate internet that uses a private IP space that is not routable on the internet. So a high-network layer protocol can be used to traverse packets through those. The contraints for this are the following:

This leaves us with an application layer UDP based protocol to implement this. In this protocol the 6 byte IP+port pair becomes the low layer address. The high layer address consists of an internet identifier plus the ip+port. The source and destination high layer addresses are part of the UDP packet's payload while the IP and port in the network layer constantly changes as the packet traverses through the networks.

Let's see an example. Alice and Bob wants to communicate. Alice's computer is on a private network and has the IP address 192.168.42.3. Alice's router has the address 192.168.42.1 on the LAN side. The router's address on the WAN side is 88.77.66.55 The application that wishes to communicate is bound on the port 4444. This gives the application the low layer address 192.168.42.3:4444. Alice's internet identifier is 21. The internet identifier of the public internet is 0. So the complete address becomes 21|192.168.42.3:4444. The application layer routing application is bound on 192.168.42.1:1234 on the LAN side. So the complete address is 21|192.168.42.1:1234. And is also bound on 88.77.66.55:2345 on the WAN side, so the complete address of that is 0|88.77.66.55:2345 The application layer routing table on the computer on Alice's network defines the internet identifier 21 as "in network", so any traffic destined to that network can be sent directly to the contained IP+port pair. While the routing table also defines a default gateway to be 21|192.168.42.1:1234.

Bob is on a private network. Bob's computer has an IP address 10.11.12.13. Bob's router has the IP address 10.11.12.1 on the LAN side. His router has the IP address of 44.33.22.11 on the WAN side. The application binds the port 3333 so the low level address on Bob's side becomes 10.11.12.13:3333. Bob's network is given the internet identifier of 22. So the complete address of the application becomes 22|10.11.12.13:3333. Bob's router have also bound port 4567 on the LAN side of the router which gives the endpoint 10.11.12.1:4567. Complete with internet identifier it becomes 22|10.11.12.1:4567. On the WAN side it binds 5678 so the low level address becomes 44.33.22.11:5678. Complete with internet identifier it becomes 0|44.33.22.11:5678. The application layer routing table in Bob's private network defines internet identifier 22 as "in network", so any traffic destined to that network can be sent directly to the contained IP+port pair. While the routing table defines a default gateway to be 22|10.11.12.1:4567.

Alice's router also have a routing table and multiple interfaces. It defines internet identifier 21 to be "in network" on the LAN side (use the 192.168.42.1:1234 endpoint), and defines internet identifier 0 to be "in network" on the WAN side (use the 88.77.66.55:2345 endpoint). For all other internet identifiers it defines the next hop. For example for internet identifier 22 it defines 0|44.33.22.11:5678 as the next hop.

Bob's router also have a routing table and multiple interfaces. It defines internet identifier 22 to be "in network" on the LAN side (use the 10.11.12.1:4567 endpoint), and defines internet identifier 0 to be "in network" on the WAN side (use the 44.33.22.11:5678 endpoint). For all other internet identifiers it defines the next hop. For example for internet identifier 21 it defines 0|88.77.66.55:2345 as the next hop.

Now send that packet. The source address is: 21|192.168.42.3:4444. The destination address is: 22|10.11.12.13:3333. The application writes this as the UDP packet's payload. Then the application looks up the routing table and uses the default route of 21|192.168.42.1:1234 to send the packet. So the source IP, port in the network layer is 192.168.42.3:4444 the destination is 192.168.42.1:1234.

The router receives the packet and see it's destined to internet 22. It looks up the routing table to find the next hop is 0|44.33.22.11:5678. 0 is on the WAN interface so it sends the packet from 88.77.66.55:2345 and it's destined to 44.33.22.11:5678. The payload is unchanged.

The packet travels through the internet and arrives at Bob's router 44.33.22.11:5678, it sees the destination of 22|10.11.12.13:3333. internet id 22 is "in network" on the LAN side so it uses the 10.11.12.1:4567 endpoint to send the packet directly to 10.11.12.13:3333 and Bob's application receives the packet.

Please note: there is no network address translation here. Each hop emits a brand new packet, there is no state to remember, the thing is stateless.

I've assumed that the router does all the logic, which makes you think that you need to change the router's firmware, but this is not strictly necessary at all. You can use a spare laptop and use port forwarding. For example Alice can designate 21|192.168.42.2:1234 as the LAN side, then designate 0|192.168.42.2:1235 as the WAN side, then have the unmodified router port forward 2345 to 192.168.42.2:1235. Then this machine would emit packets from 192.168.42.2:1235 that's destined to the internet, the router then would send this out from 88.77.66.55 using a random port. But incoming packets to 88.77.66.55:2345 will be forwarded to 192.168.42.2:1235. So bidirectional communication can work, but the source address and port can be pretty much anything. No logic should depend on or interpret the ip port of incoming packets.

Another issue is invalid addresses. If Bob's subnet is 10.11.12.0/24, then 22|8.9.10.11 is not a valid address in that network. Routers should check source addresses and if they are invalid they should rewrite it to observed source network layer IP and transport layer port. This can mainly happen when a client is behind NAT and that local network has no router configured. For example a client has an IP 192.168.11.12 and no application layer router configured on the network. The client binds to 4444. And uses network id 0, because they assume they are on the internet. Therefore the local triplet is going to be 0|192.168.11.12:4444 Let the destination triplet be 22|10.11.12.13:3333 (Bob again). The local endpoint is 192.168.11.12:4444 the remote endpoint is based on the routing table becomes 44.33.22.11:5678 as before. We send the packet but the NAT rewrites the source address and port to let's say to 99.100.101.102:34567. Bob's router then sees the packet with source address 0|192.168.11.12:4444 and destination of 22|10.11.12.13:3333. Bob's router finds that 192.168.11.12 is not a valid internet address (it's a private address), therefore it fixes it by rewriting it to 0|99.100.101.102:34567 based on the transport layer port and network and IP and that's what the application at Bob sees as source address. The application responds, the source address is 22|10.11.12.13:3333, and the destination is 0|99.100.101.102:34567. That's what the client application gets back in return and it's a way to find that something is messing with source address and the client is behind NAT.

So that invalid source addresses can be fixed, but invalid destination addresses are a different thing, it's an error to send to an non-existing host. These packets should be ignored.

To obtain the configuration, nodes should make a TCP connection to a configuration host on the network, download the network id, the allowed addresses, and the routing table from there. The IP port of this configuration host may be obtained by using an SRV record from a domain. More work is needed to find out the right way.

Feedback

Posts

See the latest posts below, click the "..." to see them all. Click the tags to filter by tag. You can also subscribe to RSS in those lists.

If AI and robots take our job how will we earn money? - 1775072820 english economy

Fixing the IP shortage using application level routing protocols - 1774297200 english networking

My Age of Empires II build order - 1774040820 english aoe2

Double entry bookkeeping explained - 1771536720 english finance

Chat2026 part 10: continuing application design - 1770850620 english chat2026-devblog

...