== Introduction == [[http://netsniff-ng.org/|netsniff-ng]] is a free, performant Linux networking toolkit. The gain of performance is reached by zero-copy mechanisms, so that on packet reception and transmission the kernel does not need to copy packets from kernel space to user space and vice versa. The toolkit can be used for network development and analysis, debugging, auditing or network reconnaissance. {{attachment:suite.png}} == Toolkit == '''*''' '''netsniff-ng''' is a high-performance network analyzer based on packet mmap(2) mechanisms. It can record pcap files to disc, replay them and also do an offline and online analysis. '''*''' '''trafgen''' is a high-performance network traffic generator based on packet mmap(2) mechanisms. It has its own flexible, macro-based low-level packet configuration language. '''*''' '''mausezahn''' is a performant high-level packet generator that can run on a hardware-software appliance and comes with a Cisco-like CLI. It can craft nearly every possible or impossible packet. '''*''' '''bpfc''' is a Berkeley Packet Filter (BPF) compiler that understands the original BPF language developed by McCanne and Jacobson. It accepts BPF mnemonics and converts them into kernel/netsniff-ng readable BPF ``opcodes''. '''*''' '''ifpps''' is a tool which periodically provides top-like networking and system statistics from the Linux kernel. It gathers statistical data directly from procfs files and does not apply any user space traffic monitoring that would falsify statistics on high packet rates. For wireless, data about link connectivity is provided as well. '''*''' '''flowtop''' is a top-like connection tracking tool that can run on an end host or router. It is able to present TCP or UDP flows that have been collected by the kernel's netfilter framework. GeoIP and TCP state machine information is displayed. '''*''' '''curvetun''' is a lightweight, high-speed ECDH multiuser VPN for Linux. curvetun uses the Linux TUN/TAP interface and supports {IPv4,IPv6} over {IPv4,IPv6} with UDP or TCP as carrier protocols. '''*''' '''astraceroute''' is an autonomous system (AS) trace route utility. Unlike traceroute or tcptraceroute, it not only display hops, but also their AS information they belong to as well as GeoIP information and other interesting things. For more information see the [[http://netsniff-ng.org/|netsniff-ng homepage]], [[https://github.com/borkmann/netsniff-ng/blob/master/README|README]], [[https://github.com/borkmann/netsniff-ng/blob/master/INSTALL|INSTALL]], [[http://netsniff-ng.org/faq.html|FAQ]], and the [[https://github.com/borkmann/netsniff-ng/tree/master/Documentation|Documentation directory]] == Installation == Tested on default installs of: <
> <
> Ubuntu Desktop 12.04 i386 <
> Ubuntu Desktop 12.04 x86_64 <
> Ubuntu Server 12.04 i386 <
> Ubuntu Server 12.04 x86_64 <
> <
> {{{ sudo apt-get install git build-essential flex bison ccache libnl-3-dev libnl-genl-3-dev libgeoip-dev libnetfilter-conntrack-dev libncurses5-dev liburcu-dev libnet1-dev libpcap-dev }}} Compile and install netsniff-ng. {{{ git clone https://github.com/borkmann/netsniff-ng cd netsniff-ng/src make sudo make install }}} For more options: {{{ make help }}} You may encounter an error like the following when building curvetun: {{{ ./nacl_path.sh: 20: ./nacl_path.sh: source: not found Done! source ~/.bashrc /bin/sh: 1: source: not found make: *** [nacl] Error 127 }}} If that's the case, then source your ~/.bashrc to pick up the two new NACL variables: {{{ source ~/.bashrc }}} Then build again: {{{ make && sudo make install }}} Documentation is installed in /usr/share/doc/netsniff-ng. <
> Sample configurations are installed in /etc/netsniff-ng == Examples == === Netsniff-ng === Listen to the first available interface and print a single packet. {{{ netsniff-ng --num 1 }}} {{attachment:netsniff-ng.png}} Write traffic coming in on eth0 to dump.pcap and don't print any output. {{{ netsniff-ng --in eth0 --out dump.pcap --silent --bind-cpu 0 }}} Netsniff-ng is great for full content packet capture. The following example will write a new pcap to the /mypcaps directory each day. {{{ netsniff-ng --in eth0 --out /mypcaps --interval 24hrs }}} Drop privileges to uid 1000 and write a new capture file to the current directory after every 10GB of traffic with the naming convention, 10gig.$timestamp.pcap. Then, capture single KiB intervals with the convention timestamp.pcap. {{{ netsniff-ng --in eth0 --out . --prefix 10gig. --interval 10GiB --user 1000 --group 1000 $ ls 10gig.1349501115.pcap 10gig.1349501161.pcap 10gig.1349501169.pcap netsniff-ng --in any --out pcaps/ --prefix "" --interval 1KiB $ ls 1360880930.pcap 1360880952.pcap 1360880975.pcap 1360880977.pcap 1360880978.pcap }}} Using mmap(), send quickly, packets from eth0 to eth1. {{{ netsniff-ng --in eth0 --out eth1 --mmap --silent --prio-high }}} Replay a network trace to an IDS listening on eth0 or attached to a hub. {{{ netsniff-ng --in dump.pcap --mmap --out eth0 -k1000 --silent --bind-cpu 1 }}} Apply a BPF filter, print matched packets in ASCII, accept jumbo frames, and increase verbosity: {{{ netsniff-ng --in any --filter http.bpf --jumbo-support --ascii -V }}} Write new file every 10 seconds to the current directory and print packet statistics for every interval by specifying verbose mode. Statistics are written as .(+gain/-loss) where gain is the number of packets processed and loss is the number of packets lost due to an overflowing buffer. {{{ netsniff-ng --in any -s --out . --interval 10sec -V RX: 64.00 MiB, 32768 Frames, each 2048 Byte allocated BPF: L0: ret #0xffffffff MD: RX sg lf64 none: prio 4 Running! Hang up with ^C! .(+139/-0).(+230/-0).(+78/-0) }}} Write a low-level BPF filter with bpfc and then pass to netsniff-ng. {{{ $ cat sample_bpf.txt ; tcpdump equivalent ; 'ether src aa:bb:cc:dd:ee:ff' ld [8] ; load 4 bytes from src MAC jneq #0xccddeeff,drop ; compare 4 bytes, move to next instruction if equal ldh [6] ; load 2 bytes from src MAC jneq #0xaabb,drop ; compare 2 bytes, move to next instruction if equal ret #1514 ; return 1514 bytes of packet drop: ret #0 ; return 0 (no packet) $ bpfc -i sample_bpf.txt > ethernet.bpfc { 0x20, 0, 0, 0x00000008 }, { 0x15, 2, 3, 0xccddeeff }, { 0x28, 0, 0, 0x00000006 }, { 0x15, 0, 1, 0x0000aabb }, { 0x6, 0, 0, 0x000005ea }, { 0x6, 0, 0, 0x00000000 }, $ netsniff-ng --in eth0 --out ethernet.pcap --filter ethernet.bpfc }}} Use tcpdump to dump BPF filter opcodes to file and pass to netsniff-ng. {{{ tcpdump -dd 'ip src 192.168.1.1 and tcp and port (53 or 80 or 443)' > myfilter.bpf netsniff-ng --in eth0 --filter myfilter.bpf --ascii }}} Use libpcap filters directly. {{{ netsniff-ng --in eth0 -f "ip and tcp and port (80 or 443)" }}} Print supported PCAP types and then write using netsniff-ng's custom PCAP format by issuing its magic number. This type supports nanosecond time stamps, packet type, etc. After, read from that capture file. {{{ $ netsniff-ng -D ... netsniff-ng pcap: magic: 0xa1e2cb12 features: timeval in ns packet length packet cap-length packet ifindex packet protocol hardware type packet type netsniff-ng -T 0xa1e2cb12 --in eth0 --out custom.pcap netsniff-ng --in custom.pcap }}} === Ifpps === Print system statistics every 1 second (1000ms): {{{ ifpps --dev eth0 --promisc }}} {{attachment:ifpps.png}} Write statistics every 5 seconds to a file in GNUPlot format and then print the PPS and drop count fields. {{{ ifpps --dev eth0 --interval 5 --promisc --csv -l > stats.csv awk '{ print $4,$5 }' stats.csv }}} === Flowtop === Install the updated GeoIP databases and place them where flowtop expects them to be. {{{ apt-get install zlib1g-dev make geoip }}} By default, flowtop tracks IPv4 & IPv6 flows carrying TCP. {{{ flowtop }}} {{attachment:flowtop.png}} Tell flowtop where to find the IPv4 GeoIP databases and watch for UDP traffic {{{ flowtop --ipv4 --city-db4 /usr/share/GeoIP/GeoLiteCity.dat --country-db4 /usr/share/GeoIP/GeoIP.dat --udp }}} {{attachment:flowtop_traceroute.png}} Examine IPv4 traffic carrying ICMP. {{{ flowtop --ipv4 --icmp }}} {{attachment:flowtop_icmp.png}} === Bpfc === Compile a low-level filter to BPF opcodes {{{ $ cat sample.bpfc ldh [12] ; Load Ethernet type field jeq #0x800, Cont, Drop ; Check value against IPv4 value Cont: ldb [23] ; Load IPv4 protocol field jeq #0x6, Keep, Drop ; Check value against TCP value Keep: ret #96 ; Return 96 bytes of packet Drop: ret #0 ; Discard packet bpfc -i sample.bpfc > sample.bpfo $ cat sample.bpfo { 0x28, 0, 0, 0x0000000c }, { 0x15, 0, 3, 0x00000800 }, { 0x30, 0, 0, 0x00000017 }, { 0x15, 0, 1, 0x00000006 }, { 0x6, 0, 0, 0x00000060 }, { 0x6, 0, 0, 0x00000000 }, }}} The following filter uses an undocumented linux kernel extension that filters based on CPU. It matches packet received using CPU-0. {{{ $ cat cpu.bpfc ld #cpu ; Load into accumulator jeq #0,L1,L2 ; Jump to L1 if true (equals 0), L2 if false L1: ret #-0 ; Return packet L2: ret #0 ; Discard packet }}} Compile filter and increase verbosity. {{{ bpfc -Vi cpu.bpfc Generated program: L0: ld #cpu L1: jeq #0x0, L2, L3 L2: ret #0xffffffff L3: ret #0x0 Validating: is runnable! Result: { 0x20, 0, 0, 0xfffff024 }, { 0x15, 0, 1, 0x00000000 }, { 0x6, 0, 0, 0xffffffff }, { 0x6, 0, 0, 0x00000000 }, }}} === Trafgen === Create a trafgen configuration file from a pcap and generate it out eth1 in random order. {{{ netsniff-ng --in ns-ng.pcap --out ns-ng.cfg -s trafgen --in ns-ng.cfg --out eth1 --rand }}} Download two trafgen configuration files and generate the traffic. In the first trafgen example, drop privileges to UID & GUID 1001. In the second example send 1000 packets then stop, after packets are sent, send an ICMP echo-request mortality test to the receiving host. {{{ wget http://pub.netsniff-ng.org/examples/trafgen/frag_packet03_small_frag.cfg wget http://pub.netsniff-ng.org/examples/trafgen/tcp_syn_flood.cfg trafgen --in frag_packer03_small_frag.cfg --out eth1 --user 1001 --group 1001 trafgen --in tcp_syn_flood.cfg --out eth0 --num 1000 --smoke-test 192.168.1.1 }}} Generate sample packet configuration. {{{ trafgen -e > sample.cfg trafgen --in one.cfg --out eth0 --cpp }}} Copy the example above but pass through stdin. {{{ trafgen -e | trafgen -i - -o eth0 --cpp --num 1000 }}} Trafgen language example with C PreProcessor use: {{{ /* Note: dynamic elements make trafgen slower! */ #define ETH_P_IP 0x0800 #define SYN (1 << 1) #define ECN (1 << 6) { /* MAC Destination */ fill(0xff, 6), /* MAC Source */ 0x00, 0x02, 0xb3, drnd(3), /* IPv4 Protocol */ c16(ETH_P_IP), /* IPv4 Version, IHL, TOS */ 0b01000101, 0, /* IPv4 Total Len */ c16(59), /* IPv4 Ident */ drnd(2), /* IPv4 Flags, Frag Off */ 0b01000000, 0, /* IPv4 TTL */ 64, /* Proto TCP */ 0x06, /* IPv4 Checksum (IP header from, to) */ csumip(14, 33), /* Source IP */ drnd(4), /* Dest IP */ drnd(4), /* TCP Source Port */ drnd(2), /* TCP Dest Port */ c16(80), /* TCP Sequence Number */ drnd(4), /* TCP Ackn. Number */ c32(0), /* TCP Header length + TCP SYN/ECN Flag */ c16((0x8 << 12) | SYN | ECN) /* Window Size */ c16(16), /* TCP Checksum (offset IP, offset TCP) */ csumtcp(14, 34), /* TCP Options */ 0x00, 0x00, 0x01, 0x01, 0x08, 0x0a, 0x06, 0x91, 0x68, 0x7d, 0x06, 0x91, 0x68, 0x6f, /* Data blob */ "gotcha!", } }}} Packet created from the above configuration: {{{ $ tcpdump -eXnnr trafgen_-e.pcap 17:15:34.412793 00:02:b3:ba:34:93 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 73: 106.17.67.46.38560 > 160.31.148.86.80: Flags [SE], seq 2933292545:2933292552, win 16, options [nop,nop,TS val 110192765 ecr 110192751], length 7 0x0000: 4500 003b 97ea 4000 4006 c11d 6a11 432e E..;..@.@...j.C. 0x0010: a01f 9456 96a0 0050 aed6 7e01 0000 0000 ...V...P..~..... 0x0020: 8042 0010 8db3 0000 0101 080a 0691 687d .B............h} 0x0030: 0691 686f 676f 7463 6861 21 ..hogotcha! }}} Create two packets with dynamic elements. The first packet is filled with the same hex byte for 64 bytes and the second packet is filled with random data for 64 bytes. The curly braces begin and end each packet. {{{ $ cat test.cfg { fill(0xff, 64) } { drnd(64) } # trafgen --dev eth0 --conf test.cfg --num 2 trafgen 0.5.8-rc0 2 packets to schedule 128 bytes in total Running! Hang up with ^C! 2 packets outgoing 128 bytes outgoing 0 sec, 4 usec on CPU0 (2 packets) }}} Results with netsniff-ng: {{{ # netsniff-ng --in eth0 --num 2 netsniff-ng 0.5.8-rc0 Running! Hang up with ^C! > eth0 64 1360380462s.868592131ns [ Eth MAC (ff:ff:ff:ff:ff:ff => ff:ff:ff:ff:ff:ff), Proto (0xffff, Reserved) ] [ Vendor (Unknown => Unknown) ] [ chr .................................................. ] [ hex ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff f f ff ff ] > eth0 64 1360380462s.869103408ns [ Eth MAC (e0:c8:c7:79:52:35 => 43:74:6c:95:76:d2), Proto (0x3c2a) ] [ Vendor (Unknown => Unknown) ] [ chr .0S......H....+^Sn...H...Y]c...h..tJ...M enable Password: mz mz-0.38# show ? }}} For more details, examples, and options please see the [[http://pub.netsniff-ng.org/netsniff-ng/tools/Mausezahn|documentation]] === Curvetun === To setup and configure curvetun see its [[https://github.com/borkmann/netsniff-ng/blob/master/Documentation/Curvetun|documentation]] ---- CategoryThirdPartySoftware