New features developed for the new Snort File Preprocessor

ExtraData fields in events and S3 storage

In redborder we strongly believe that innovation is only possible through the sharing and cross-pollination of ideas. This is the reason why all our solutions are developed under Open Source licenses.

Part of our daily work involves working with the community researching, developing and sharing new enhancements that are useful both to our team, as well as any developer who wants to incorporate Open Source technology into their projects.

Our latest contribution in this field is related to to the incorporation of two new features in the Snort File preprocessor and some changes in Barnyard2 in order to include the processing of ExtraData fields that can be included in Snort events.

The following is a summary of the improvements we have integrated, and the location of these code modifications in GitHub.

Snort File preprocessor: features included

Recently, Snort has built-in a File preprocessor, which is able to detect files inside the analyzed traffic. The three features that the new preprocessor incorporates are:

• File type identification.

• SHA256 digest calculation file.

• File capturing to disk, to memory and to other host.

redBorder's contribution

• In Snort v2.9.7.3:
• S3 delivery of captured file.
• Inclusion of additional information (SHA256, file size, downloading and source file name) in the event generated by Snort to detect a file.

• In Barnyard2:
• Since Barnyard2 v2.1.13 doesn't take into account the ExtraData fields, we've also changed it drastically, altering the way the spooler process analyzes events and records.

The following diagram shows how the overall operation would be, which involves network traffic analysis by Snort, detecting the existence of files and proceeding to its S3 delivery.

Additionally, Snort would generate an event related to the detection of those files. Also, it has worked to include in the events generated by Snort the following information relative to the detected file: SHA256 sum, file size and the HTTP address from which it comes.

 

 

Next are examples shown for the configuration of new features:

• Configuration example for S3 storage:

include file_magic.conf
preprocessor file_inspect:\
    type_id, \
    capture_queue_size 5000, \
    signature, \
    capture_disk /var/log/snort/files/ 5000, \
    s3_bucket bucket, \
    s3_cluster S3 server, \
    s3_access_key access key, \
 s3_secret_key secret key

• Configuration example for inclusion of ExtraData fields in events:

include file_magic.conf
preprocessor file_inspect:\
   type_id, \
   capture_queue_size 5000, \
   signature, \
   capture_disk /var/log/snort/files/ 5000, \
   track_extradata
include snort_files.rules

Where could i find these features?

These features have been developed over Snort v2.9.7.3 and Barnyard2 v2.1.13 and are available in our github server. Please follow the links below to find them:

• Snort features:

https://github.com/redBorder/snort/tree/feature/file_extradata
https://github.com/redBorder/snort/tree/feature/file_s3

• Barnyard2 changes:

https://github.com/redBorder/barnyard2/tree/Feature/Managing_ExtraData_fields
(just needed if you're interested on Snort ExtraData feature)

Please take into account that this is a very early version that could contain some bugs, so we
would be happy to receive any feedback and suggestions.

This publication follows the general redborder principles of divulging new features and enhancements in Snort in appreciation for the enormous collective effort of this community.

For that, we hope this can be useful to you.

You can find more information in the link below:
http://sourceforge.net/p/snort/mail