Getting Started

NFStream is a Python framework providing fast, flexible, and expressive data structures designed to make working with online or offline network data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world network data analysis in Python. Additionally, it has the broader goal of becoming a common network data analytics framework for researchers providing data reproducibility across experiments.

Table of contents

  1. Main Features
  2. Installation Guide
    1. Python packages manager
    2. Building NFStream from sources
      1. Linux Prerequisites
      2. MacOS Prerequisites
    3. Build Dependencies
    4. Build NFStream

Main Features

  • Performance: NFStream is designed to be fast: AF_PACKETV3/FANOUT on Linux, parallel processing, native C (using CFFI) for critical computation and PyPy support.
  • Encrypted layer-7 visibility: NFStream deep packet inspection is based on nDPI. It allows NFStream to perform reliable encrypted applications identification and metadata fingerprinting (e.g. TLS, SSH, DHCP, HTTP).
  • Statistical features extraction: NFStream provides state of the art of flow-based statistical feature extraction. It includes both post-mortem statistical features (e.g. min, mean, stddev and max of packet size and inter arrival time) and early flow features (e.g. sequence of first n packets sizes, inter arrival times and directions).
  • Flexibility: NFStream is easily extensible using NFPlugins. It allows to create a new feature within a few lines of Python.
  • Machine Learning oriented: NFStream aims to make Machine Learning Approaches for network traffic management reproducible and deployable. By using NFStream as a common framework, researchers ensure that models are trained using the same feature computation logic and thus, a fair comparison is possible. Moreover, trained models can be deployed and evaluated on live network using NFPlugins.

Installation Guide

Python packages manager

Binary installers for the latest released version are available on Pypi.

pip install nfstream

Building NFStream from sources

Linux Prerequisites

sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config flex bison gettext
sudo apt-get install libusb-1.0-0-dev libdbus-glib-1-dev libbluetooth-dev libnl-genl-3-dev

MacOS Prerequisites

brew install autoconf automake libtool pkg-config gettext

Build Dependencies

git clone --branch libgpg-error-1.39 https://github.com/gpg/libgpg-error
cd libgpg-error
./autogen.sh
./configure -enable-maintainer-mode --enable-static --enable-shared --with-pic --disable-doc --disable-nls
make
sudo make install
cd ..
rm -rf libgpg-error
git clone --branch libgcrypt-1.8.6 https://github.com/gpg/libgcrypt
cd libgcrypt
./autogen.sh
./configure -enable-maintainer-mode --enable-static --enable-shared --with-pic --disable-doc
make
sudo make install
cd ..
rm -rf libgcrypt
git clone --branch fanout https://github.com/tsnoam/libpcap
cd libpcap
./configure --enable-ipv6 --disable-universal --enable-dbus=no --without-libnl
make
sudo make install
cd ..
rm -rf libpcap
git clone --branch dev https://github.com/ntop/nDPI.git
cd nDPI
./autogen.sh
./configure
make
sudo mkdir /usr/local/include/ndpi
sudo cp -a src/include/. /usr/local/include/ndpi/
sudo cp example/ndpiReader /usr/local/bin/ndpiReader
sudo cp src/lib/libndpi.a /usr/local/lib/libndpi.a
cd ..
rm -rf nDPI

Build NFStream

git clone https://github.com/nfstream/nfstream.git
cd nfstream
python3 -m pip install -r requirements.txt
python3 setup.py bdist_wheel