FLoP - Fast Logging Project for snort, version 1.5.1 ---------------------------------------------------- The aim of the Fast Logging Project is to decouple the output to a database from the real snort process. snort has to process the network packets to find alert signatures. If an alert is found the result has to be stored somewhere to process or simply store it. The problem with the normal database output plugin is that snort is blocked until all data is written to the database. If there are any problems with the database then snort is hanging and probably some network packets get lost. The FLoP project decouples this output and processing from snort. Further all alerts are spooled to a central server where they are feed in a database via unix sockets which is much faster than TCP/IP via any physical network. (Starting with version 1.5.1 a TCP connection is supported but should only be used on a loopback device.) Since snort will write to an unix domain socket there is no blocking possible. (Of course you can loose some alerts if no process reads these alerts. But this is another story...) Addition: There is still blocking possible if the reading process blocks. But this will never happen with sockserv since one thread is still waiting for input. The program sockserv generates such an unix socket and reads all incoming alerts. The program consists of two threads, one reads the alert and buffers them in memory. The second take the alerts and spools them to the central server. Here the program servsock reads these alerts. This program forks off a child process for each sockserv connection. These child processes consists of two threads. One thread receives the alerts and the second one feeds them to the database. Again a buffering takes place to overcome some slow downs due to too many inserts in the database. Finally to avoid a buffer overrun in form of a SIGSEGV there is a 'drop' feature implemented. This works with a high and low water mark, if more then high water alerts are stored in memory they are dropped to an unix socket until the low water mark is reached. The drop program is able to receive these dropped alerts and will send an email consisting of a short message which alerts were dropped. (The only real thing missing is the payload.) Additionally there is an alert feature. This is intended for sending email if alerts with a high priority arrive. This should inform an admin earlier then the periodic check of the database. And last but not least there is a false positive generator to build network packets which are able to raise alerts within snort. (Of course there exist some similar programs but I needed one which is able to generate alerts with a high rate to check if all programs are able to deal with it or will break. With a slight extension of the database it is possible to store the full network packet in the database. A program called getpacket is able to rebuild a pcap file from this data. This pcap file can then be analyzed with tools like ethereal Further helpful programs are in the contrib directory: + classification.pl: Adds the classification.config to the database + some cgi scripts to analyze alerts via a web server + stats.pl: A script to gather statistics from snort + rules.pl: A perl script to add the signatures and references of the rule files to the database README files: README: ok, you read this now so you should know about it README.debug: This describes how to activate the debugging functions of sockerv, servsock, alert, drop README.endian: Some comments on the endianess of the sensors and the central server README.payload: This describes how the full payload (more than only the alert level payload, e.g. TCP or UDP) can be stored in the database README.snort: This explains the changes made to snort introduced by the patch. README.ctrl: How to use the control thread to change settings of a running system. README.barnyard: This files shows some differences between barnyard (or mudpit) and FLoP contrib/README: This is a survey of the useful files in the contrib directory contrib/README.rules.pl: How to add all signatures with references to the database. This should speed up the insert process since most rules are now part of the database and need not to be inserted together with the alert If you find this software useful then send a picture postcard to: Dirk Geschke Plankensteinweg 61 85435 Erding GERMANY (Maybe I will get at least one picture postcard?) For any other feedback send me an email: