Packet storage

A long capture can hold millions of packets. NetOtter keeps their decoded headers in RAM, and moves their payloads, compressed, to a temporary file.

The packet list

netotter_t.pkt_l is a doubly linked list of packet_list_t nodes (src/helpers/packet_list.c). Each node holds one fpacket_t and its id, list_id.

• Appending: plist_add_packet() appends after netotter_t.pkt_tail without walking the list, and gives the next id (1, 2, 3...). Ids are uint32_t, because a busy capture passes 65,535 packets in seconds.
• Lookup: plist_find() finds a packet by id, for i.
• Out of memory: both return an error, and the caller drops the packet.

The RAM budget

packets_add() (src/helpers/packet_utils.c) is the only way into the list, used both by raw_to_f() and by the .nto loader.

1. Charging: it adds the packet's fpacket_footprint() to netotter_t.pkt_bytes. The footprint is every allocation of the packet at glibc's real chunk sizes: about 320 bytes for a UDP/IPv4 packet. Payloads are in the spool, so they don't count.
2. Refusing: past PACKETS_RAM_MAX (2 GiB), it refuses the packet and sets capture_full.
3. Stopping: capture_full_check() runs after each tick, in both modes. It stops every interface (each tick, so an interface started by -i can't slip through) and announces it once with print_alert(). The CLI mode then stops as on Ctrl-C. The digit keys refuse to start an interface.

print_alert() writes in bold red when term_colors() says the terminal shows ANSI colours, and in capitals otherwise. limit_message() builds the alert texts from the real PACKETS_RAM_MAX, so a test build with -DPACKETS_RAM_MAX=... quotes its own limit.

The payload store

includes/payload_store.h, src/helpers/payload_store.c.

flowchart LR
  p["parsed payload<br/>(fpacket_t.payload)"] --> c{"lz_compress()<br/>smaller?"}
  c -- yes --> z["compressed bytes<br/>PAYLOAD_COMPRESSED"]
  c -- no --> r["raw bytes"]
  z --> wb["64 KiB write buffer"]
  r --> wb
  wb -->|"after each tick / load"| sp[("spool<br/>temporary file")]
  sp -->|"i: store_read_payload()"| d["lz_decompress()"]
  sp -->|"save: store_stored_bytes()"| s[".nto record, as is"]

Function	When
store_init()	At startup (not with -l): opens the spool and announces where it is.
store_payload()	End of process_packet(): compresses, appends, frees payload.
store_add()	Loading a .nto: appends the saved bytes as they are.
store_flush()	After each tick and each load: writes the buffer out.
store_read_payload()	i only: reads back and decompresses one payload.
store_stored_bytes()	Saving: reads back the stored bytes, untouched.

A packet records where its bytes are: stored_off (offset in the spool), stored_len, and stored_flags (PAYLOAD_COMPRESSED or not). payload_len stays the real length.

So the payloads are decompressed only for i. Saving never recompresses and loading never decompresses. That is also why a loaded capture can't learn hostnames from its payloads: the save file carries the names instead (see Save format).

The spool

The backend provides it behind spool_open / spool_write_at / spool_read_at / spool_close:

• Linux: an O_TMPFILE file (or mkstemp + unlink) in the cache folder. It has no name, so the kernel deletes it on close, even after a crash. pread / pwrite do the I/O.
• Windows: a FILE_FLAG_DELETE_ON_CLOSE file in the cache folder, with offsets passed through an OVERLAPPED structure.

It lives in the cache folder rather than /tmp, which is often a RAM disk. When it can't be created, the stored bytes stay in RAM (fpacket_t.stored). A failed write is reported once.

The compressor

libs/utils/lz_compress.c, lz_decompress.c, lz_range.c. Each payload is compressed on its own, so i can decompress one packet without the others, with no stored tables (fit for small buffers).

• Matching: LZ77, with matches found through hash chains (48 candidates deep) and lazy matching (a match is delayed by one byte when the next one is longer).
• Coding: an LZMA-style adaptive binary range coder with 11-bit probabilities.
  • Literals are coded in the context of the previous byte's top 3 bits.
  • Lengths go from 2 to 273; a length of 2 is only for repeating the last distance.
  • Distances are coded as a slot, then extra bits.
  • A "repeat last distance" token handles structured data.
• Integrity: every stream ends with a 16-bit check of the original bytes. A damaged stream is reported as unreadable instead of being shown as garbage.
• Giving up early: after 128 bytes that don't shrink, the encoder stops and the payload is stored raw. TLS and QUIC payloads are encrypted, so they never shrink.
• Untrusted input: the decoder may read someone else's save file. Every length and distance is checked against what has been written, and nothing is read or written out of bounds.

Measured on sample captures, plaintext payloads shrink to about 55% (zlib at level 9: 59%). Encrypted traffic doesn't shrink at all, so most of the RAM saving comes from the spool.