SuStorID

• Autonomous Learning. Given a sample of web requests towards your web server, it can learn autonomously the profile of legitimate traffic, exploiting a number of machine learning algorithms.
• Anomaly-based Approach. SuStorID detect known as well as unknown (never-before-seen) web attacks by highlighting anomalous traffic.
• Multi-model Architecture. A multi-model architecture is employed to characterize different categories of anomalies and provide a human-readable representation of anomalous events.
• Real-time Counteractions. Depending on the set of raised anomalies and their reliability, a different counteraction can be taken.
• Easy integration with modsecurity. SuStorID can be coupled with modsecurity, the well known web application firewall, to gather training data and provide for real-time counteractions. So, SuStorID is a host-based Intrusion Detection System, and by means of modsecurity can access internal web server’s data (i.e. request and response message fields) exactly as Apache does.
• Inspection of Encrypted traffic. Since SuStorID can directly access Apache’s internal data, it can monitor both HTTP and HTTPS traffic.
• User-friendly Interface. The IDS interface is very important to investigate security issues, so SuStorID itself has been implemented as a web application with a user-friendly interface. SuStorID is written in Python exploiting the amazing Django framework, and some machine learning algorithms have been implemented in C language.

Last, but not least, SuStorID is free software, released under the GNU General Public License version 3!

1. Make sure that Python v.2.6 or higher (and python numpy) and Django v. 1.3 are installed on your system. On Ubuntu:

   $ apt-get install python django python-numpy

2. Download and extract SuStorID’s source code from the official repository.

3. Open a Python Interpreter and run:

   from random import choice
   ''.join([choice('abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)') for i in range(50)])
   

   paste the result for your SECRET_KEY in settings.py (initial value is empty).

4. Inside the main directory of the project digit

   $ python manage.py syncdb

   This will create all tables needed by SuStorID. Specify a username and a password for administration: these credentials will be necessary to manage SuStorID and visualize detection results. By default, SuStorID uses the SQLite database, however you may use any database supported by Django to enhance performances, or employ a client-server db infrastructure (e.g. through Postgres).

   $ python manage.py runserver

   This will run the default Django server on the loopback interface (IP 127.0.0.1 and TCP port 8000). As any Django application, settings can be modified on settings.py. You may also change the web server to anyone supported by Django (e.g. Apache), to increase reliability and performances.

4. Point your browser (make sure that javascript is ON) to http://127.0.0.1:8000, and insert your credentials. The SuStorID interface will guide you to its setup.

• Install Python Dev tools

  $ sudo apt-get install python-dev

• Inside SuStorID/base/raw_models/hmm/ digit:

  $ python setup.py config
  $ python setup.py build

  If necessary, install missing fortran/c compilers and headers (e.g. in Ubuntu this can be done by means of Synaptic).

• A new folder will be created, containing the dynamic libraries:

  cd build/lib.linux<platform_details>/logilab/hmm/

• Copy all dynamic libraries inside SuStorID/base/raw_models/hmm/

That’s it!

• gathering training data (i.e., web request samples)
• providing real-time counteractions against suspicious web traffic

NOTE: This guide assumes Ubuntu OS 10.04 (some steps may be valid, some other may change depending on the OS).

1. Download modsecurity for SuStorID source code from the official repository.

2. Install the required packages to build this code:

   $ sudo apt-get install libc6-dev libc6 apache2 apache2-threaded-dev libxml2 libxml2-dev gcc libpcre3-dev

3. Within the main directory of the project digit

   $ ./configure

   This command will highlight any other missing dependency: correct it by installing the related headers/libraries.

4. Get into the Apache2 folder, and digit

   $ make
   $ sudo make install

   If you don’t encounter errors, the dynamic loadable library /usr/lib/apache2/modules/mod_security2.so should be installed. It is necessary to load the mod_unique_id module of apache (thus we create a symbolic link inside the mods-enabled folder)

   $ sudo ln -s /etc/apache2/mods-available/unique_id.load /etc/apache2/mods-enabled/

   Inside the sustorid_config directory of this project, digit

   $ sudo cp security2.load /etc/apache2/mods-available/
   $ sudo ln -s /etc/apache2/mods-available/security2.load /etc/apache2/mods-enabled/
   $ sudo mkdir /etc/apache2/modsecurity_conf
   $ sudo cp modsecurity_conf/mod_security.conf /etc/apache2/modsecurity_conf/

   Make sure that the apache2 user (e.g. www-data) can read such files (check access permissions for such files).

5. Add the following line to /etc/apache2/apache2.conf

   Include /etc/apache2/modsecurity_conf/

   This tells Apache to load the SuStorID configuration /etc/apache2/modsecurity_conf/mod_security.conf. Read the documentation in the Configuration of modsecurity for SuStorID section to correctly configure modsecurity for SuStorID. The default configuration enables the logging module only, that is, without active Intrusion Detection/Protection: the IDS has not been trained yet and needs some training data.

6. Restart Apache2

   $ sudo /etc/init.d/apache2 restart

   As soon as Apache restarts modsecurity for SuStorID will be activated.

    

• IDServer: you should specify (space-separated) IP address and TCP port where SuStorID is listening to.
• IntrusionDetection: set to On to enable Intrusion Detection by SuStorID (in conjunction with modsecurity rules). Please note that this only works if SecRuleEngine On, because this tells modsecurity to actively counteract against detected threats.
• PostRequests: set to On to enable request forwarding, that is, to automatically send requests and responses to SuStorID (it is necessary to get sample requests for learning).

That’s it. Enjoy!