This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Packet parsing

raw_to_f() (src/parsers/raw_to_f.c) turns the raw frames of the queue into packets (fpacket_t), one at a time, through process_packet(). The Learning section explains each protocol's layout.

The packet: fpacket_t

Defined in includes/netotter_types.h. A packet owns one structure per decoded header, and fpacket_delete() frees them all.

Fields	Layer
if_id, ifname, type_src	where it was captured
link, eth	link layer: LINK_ETH with its Ethernet header, or LINK_RAW_IP
net_proto, net_type, ipv4 / ipv6 / arp	network layer: the EtherType, and the header matching net_type
ptype, tcph / udph, port_s, port_d	transport layer (T_TCP, T_UDP or T_UNK)
mcast	IGMP / MLD (ptype stays T_UNK: they have no ports)
payload, payload_len, l4_off	everything after the network header, while parsing
ts_us	capture time, microseconds since the Unix epoch (from the kernel ring on Linux, Npcap on Windows)
l7	DNS, ICMP and HTTP facts (l7_info_t), NULL for anything else
stored_len, stored_flags, stored_off, stored	the payload once stored (see Packet storage)

The layers

flowchart TD
  pl["parse_link()"] -->|EtherType| fnp["find_net_parser()<br/>net_parsers[]"]
  fnp --> ip4["parse_ipv4()"]
  fnp --> ip6["parse_ipv6()<br/>+ skip_ipv6_extensions()"]
  fnp --> arp["parse_arp()"]
  fnp -->|"none / failed"| unk["NET_UNK: payload = after the link header"]
  ip4 -->|"l4_proto"| pt["parse_transport()"]
  ip6 -->|"l4_proto"| pt
  pt --> tcp["parse_tcph()"]
  pt --> udp["parse_udph()"]
  pt --> igmp["parse_igmp() (IPv4, proto 2)"]
  pt --> mld["parse_mld() (ICMPv6 130/131/132/143)"]
  pt --> l7["parse_l7(): DNS, ICMP, HTTP"]

Application facts: parse_l7()

src/parsers/parse_l7.c runs right after the transport layer, while the payload is still plain. It fills fpacket->l7, which is saved with the packet, so the anomaly detection works on a loaded capture without decompressing anything:

• DNS (UDP or TCP port 53): id, response flag, rcode, answer count, the first question's type, and for its name: length, longest label, and FNV-1a hashes of the name and of its last two labels. Those hashes count distinct names and domains (random names, PTR bursts) without keeping the names.
• ICMP / ICMPv6: type and code, and for an error message, the packet it's about: protocol, addresses, ports. A port unreachable then refuses the UDP connection it names.
• HTTP (any TCP port): a request method (GET , POST ...) or a status line (HTTP/1.x NNN) at the start of a segment.

The l line shows them as tags: [DNS A?], [DNS NXDOMAIN], [DNS A 1 answer], [port unreachable 192.0.2.10:514/udp], [echo request], [GET], [HTTP 404]. i adds a DNS, ICMP or HTTP line; for DNS, it reads the question name again from the decompressed payload.

Link: parse_link()

• Ethernet (LINK_ETH): parse_eth() reads the 14-byte header into eth and sets net_proto to its EtherType.
• Raw IP (LINK_RAW_IP, VPN tunnels): there is no link header, so the EtherType is guessed from the IP version nibble (4 → 0x0800, 6 → 0x86dd).

Values of 1500 and below aren't EtherTypes but 802.3 lengths, shown as LLC.

Network: net_parsers[]

src/static_data/static_nw.c maps EtherType → parser:

static_net_parsers const net_parsers[] = {
    {ETYPE_IPV4, parse_ipv4},
    {ETYPE_IPV6, parse_ipv6},
    {ETYPE_ARP, parse_arp},
};

A parser has the net_parser_t signature:

int parse_xxx(fpacket_t *fpacket, const uint8_t *data, int len, net_result_t *res);

It decodes its header into the packet, sets net_type, and fills res:

• hdr_len: the header's length;
• payload_len: the bytes after it;
• l4_proto: the next protocol's number, or -1 when there's none.

It returns 0 on success, or 1 when the header can't be decoded, and then it changes nothing. With no parser, or a failed one, the frame is still kept, as NET_UNK.

Details per parser:

• parse_ipv4
  • Honours the header length field (IHL), so options are skipped, though not decoded.
  • Cuts the payload at the IP total length, because short frames get padded to 60 bytes.
  • Only the first fragment carries the transport header: l4_proto is -1 for the others.
• parse_ipv6
  • Reads the fixed 40-byte header.
  • skip_ipv6_extensions() then walks up to 8 Hop-by-Hop, Routing and Destination Options headers. It sets ipv6->upper to the protocol after them and ipv6->ext_len to their length, and stops at a Fragment header or anything cut short.
  • The payload still starts right after the fixed header; fpacket->l4_off = ext_len is where the transport header begins inside it.
• parse_arp: IPv4-over-Ethernet ARP only: operation, sender and target MAC and IP.

Transport: parse_transport()

It dispatches on the protocol number, reading at payload + l4_off:

• parse_tcph()
  • Reads the fixed header fields.
  • Honours the data offset, so options are counted in the header length (TCP header 40 bytes for a SYN), though not decoded.
  • Sets port_s / port_d.
• parse_udph(): reads the 8-byte header and sets the ports.
• parse_igmp() / parse_mld()
  • Fill one mcast_header_t: version, type, max response time, group.
  • Keep the first 16 records of v3 / v2 reports, with the announced record count kept exact.

Hostnames: hosts_learn()

The last reader of the plain payload: DNS answers, TLS ClientHello SNI and QUIC Initial packets. See Hostname learning. After it, store_payload() stores the payload and frees it.

Reading bytes

Multi-byte fields are big-endian. Use read_u16() / read_u32() (declared in network.h), never a cast like *(uint16_t *)p: a cast reads the bytes in the machine's order (wrong on x86) and may be misaligned.

Every parser checks lengths before reading: the frame comes from the network, and anyone can send anything.

Adding a network protocol (VLAN, RARP...)

1. Type: add a NET_* value, a header struct, and its pointer in fpacket_t (netotter_types.h). Free it in fpacket_delete().
2. Parser: write parse_xxx() with the net_parser_t signature, and add a row to net_parsers[].
3. Display: show it in the l line (print_ip, src/display/display_list.c) and in the i details (print_ip_info, display_info.c).
4. Save format: add a SAVE_HAS_* bit and handle it in put_headers() / get_headers() (src/save/save_record.c). This changes the .nto layout, so bump SAVE_FORMAT_VERSION; see Save format.
5. Filters, if useful: etype already matches any EtherType.
6. Build: add any new file to SRCS, then run make re (a struct changed) and make windows.

Decoded so far

Layer	Protocols
Link	Ethernet II, 802.3 (shown as LLC), raw IP
Network	IPv4 (options skipped), IPv6 (with extension headers), ARP (IPv4 over Ethernet)
Transport	TCP (options skipped), UDP, IGMP v1–v3, MLD v1–v2
Application, for names	DNS answers, TLS ClientHello, QUIC v1 / v2 Initial packets
Application, for the anomaly detection	DNS header and question, ICMP / ICMPv6 type and quoted packet, HTTP request method and response status