Compression

Encapsulation header (urgency=low)

length	description
1	packet type
4	retransmit mask
2	encapsulation seq
2	encapsulation ack
2	command stream seq
2	command stream ack
2	command stream len

command stream length field is completely redundant with length field in the udp header.

encapsulation seq/ack and command stream seq/ack can be 8 bit fields since it takes 30 seconds for them to wrap around even then. In case of 10 seconds of no communication between 2 peers, a special value of 0xff in them can signal that a 16bit seq/ack field follows.

retransmit mask can be replaced by 1 bit in most cases, since it is zero basically all the time. bit=0 signals mask==0, bit=1 signals mask follows.

packet type can be replaced by 2 bits, since after the initial game setup stage, only command 4 and 5 are seen on the wire. 0==command4, 1==command5, 2==8bit command follows, 3==reserved

encapsulation header savings: 6 + 31 + 4*8 + 16 bits == 85 bits of 120 bits (70%) assuming 8 * (20 + 8) bits for IP4+UDP, total savings: 85 bits of 344 (25%)

Command stream (urgency=low)

the command stream (for when encapsulation.type == 4) consist of a sequence of command packets with this structure:

length	description
1	command code
2	length
N	contents

Known command codes

code	length	argument	description
0	4	simtick increment	sender target simtick += argument
0x32	5	playerid,simtick	sender acknowledges simtick for playerid
0x33	4	simtick	? (simtick that was executed?)
0x34	4	simtick	? (simtick that is is closed and ready to be executed?)
0xc	var	?	? (sim command?)
0x16	var	?	? (lua command? e.g.: give resources, chat)
0x3	20	hash,simtick	desync detection hash

many more command codes that are still unknown.

The deflate compression appears to achieve a ca. 50% compression ratio, albeit for a case where not many commands were given to the simulation.

example packet dump (3 player game): https://github.com/faf5678/stuff/blob/master/packet_dump

it can be seen in the example dump that the majority of the communication consists of commands: 0,32,33,34.

Proposed new command packet structure:

code(4bit) + variable length content

code	description
0	ADV
1	C33
2	C34
3	ACK8-24
4	ACK0
5	ACK1
6	ACK2
7	ACK3
8	ACK4
9	ACK5
10	ACK6
11	ACK7
12	reserved
13	normal command packet follows
14	select sources/destinations
15	reserved

ADV +1bit [+3bit]
	1:  ADV 1 + ACK(our_id, 1)
	0:  +3bit ADV 1-8
C33 +1bit [+3bit]
	1:  previous C33 + 1
	0:  +3bit previous C33 +2-9
C34 +1bit [+3bit]
	1:  previous C34 + 1, C33 = C34
	0:  +3bit previous C33 +1-8
ACK0-ACK8 +1bit [+3bit]
	1:	previous ACKn + 1
	0:	+3bit previous ACKn +2-9
ACK8-24 +4bit +1bit [+3bit]
	4bit: playerid 8-24
	variable length
		1: previous ACKn + 1
		0: +3bit previous ACKn +2-9
normal command packet:
        1 byte type, 2 byte length, var content as before
        deflate compression as before
(TBD) select destinations: variable length
	1: all
	0: +N bits bitfield of destinations
(TBD) select sources: variable length
	+N bits bitfield of sources

justification: since any encapsulation packet can rely on the fact that any previous packets are received before it is being evaluated (with the help of command seq/ack), one can without complications reference any previous command package. Since simtick counters are basically always incremented by one, a relative encoding is much better.

the expected gains are for ADV, C33 and C34: 5 bits instead of 56 bits and for C32: 5 or 9 bits instead of 64

for the majority of command packets the compression is expected to be closer to 10-20% instead of 50% for deflate.

With proxy (urgency=normal)

Since most restricting link is assumed to be the uplink of a few peers and the fact that FA.exe sends identical data to all peers, the most urgent improvement would be in the form of a proxy that can read a command packet once and send out many copies to all peers. the back route would still go to the originating peer directly since it can be expected that the downlink is not so heavily limited as the uplink. In this case the originating peer has to initiate lost packet retransmissions, but it can instruct the proxy to do the retransmission (meaning that the proxy will have to keep the packets around for a little while). Note that the originating peer still has to send all seq/ack numbers for all peers to the proxy, since the proxy has no knowledge of returned packets. Alternatively the proxy could also participate in the back route, the penalty there being added latency and added traffic for the proxy, the benefit there is that it could manage all sequence numbers for itself. Another significant advantage would be that incoming packets from several peers could be coalesced, since they can be expected to also send the same command packets in close temporal proximity (see next paragraph).

"Centralized p2p" (urgency=low)

all traffic goes through proxies. The amount of traffic for each peer is expected to be constant over the amount of players in the game, as opposed to growing linearly with the number of players. The total amount of traffic is expected to grow linearly with number of players as opposed to quadratically.

Penalty: added latency, added proxy traffic

Benefits: less total traffic, basically constant traffic per peer

Since the real issue seems to be uplink bandwidth, there is little necessity for this at the moment. It is equivalent to the "With proxy" paragraph above where the proxy is part of the back route.

Coalesced P2P

One peer with limited bandwidth could use another peer as proxy for all traffic. This setup doesnt cause any central proxy traffic, and since the peer that acts as proxy already exchanges packets with the remaining peers, large gains in compression can be achieved. Proxied command packets can piggyback on a IP+UDP header that is sent anyway by the peer acting as proxy to the remaining peers and the data portion can be compressed more when multiple peer's messages are coalesced. That is the proxy peer will have way less than 100% overhead due to being a proxy for one additional peer. This scheme can be used in an uplink-only or uplink+backroute style.