Security Onion 20111028 now available!

Security Onion 20111028 is now available!  This resolves Issue 135 by updating the NSM scripts to start Snort with the AFPACKET DAQ for higher performance.  For more information about the AFPACKET DAQ, please see:

http://manual.snort.org/node7.html
http://vrt-blog.snort.org/2010/08/snort-29-essentials-daq.html

In-place Upgrade

Existing Security Onion users can perform an in-place upgrade using the following command (if you're behind a proxy, remember to set your proxy variables as described in the FAQ):

sudo -i "curl -L http://sourceforge.net/projects/security-onion/files/security-onion-upgrade.sh > ~/security-onion-upgrade.sh && bash ~/security-onion-upgrade.sh"

Screenshots

Upgrade Process

Security Onion 20111025 now available!

Security Onion 20111025 is now available!  This resolves Issue 84 by updating Snort to version 2.9.1.2 and its DAQ to version 0.6.2.  For more information about Snort 2.9.1.2, please see:

http://blog.snort.org/2011/10/snort-2912-has-been-posted.html

Please note that if you are using the Registered (30-day delay) VRT ruleset you will need to wait until the rules are released for Snort 2.9.1.2.  For more information, please see:

http://blog.snort.org/2011/10/vrt-rule-release-for-10202011-snort.html

Please also note that the new snort.conf will overwrite your existing snort.conf.  Your existing snort.conf will be backed up to /nsm/backup/20111025/NAME_OF_SENSOR/.  Please copy any customizations (HOME_NET, etc.) from the backup copy to the production copy /etc/nsm/NAME_OF_SENSOR/snort.conf.

In-place Upgrade

Existing Security Onion users can perform an in-place upgrade using the following command (if you're behind a proxy, remember to set your proxy variables as described in the FAQ):

sudo -i "curl -L http://sourceforge.net/projects/security-onion/files/security-onion-upgrade.sh > ~/security-onion-upgrade.sh && bash ~/security-onion-upgrade.sh"

Screenshots

Installing new packages

Backing up config files and copying new files into place

Running PulledPork to download new ruleset

Stopping the old Snort and starting the new Snort

snort -V

Security Onion 20111020 now available!

Security Onion 20111020 is now available!  This resolves Issue 133 by updating the NSM scripts to spawn daemonlogger (instead of snort) for full packet capture.  Since daemonlogger is simpler than snort and specifically designed for packet capture, it is more efficient and possibly more secure.

In addition, daemonlogger defaults to a snaplen of 65535, so this is a PARTIAL solution to the problem described here.  I emphasize that this only a partial solution because it only solves the full packet capture problem and not the packet reassembly problem.  NIC offloading should still be disabled to allow Snort to do proper target-based reassembly and thus minimize the likelihood of insertion/evasion attacks.  For more information, please see the Snort manual.

In-place Upgrade

Existing Security Onion users can perform an in-place upgrade using the following command (if you're behind a proxy, remember to set your proxy variables as described in the FAQ):

sudo -i "curl -L http://sourceforge.net/projects/security-onion/files/security-onion-upgrade.sh > ~/security-onion-upgrade.sh && bash ~/security-onion-upgrade.sh"

Screenshots

Upgrade Process

When is full packet capture NOT full packet capture?

I was looking at some packets recently and noticed the Wireshark message "Packet size limited during capture".  This was strange since the packets came from a Sguil sensor performing full packet capture using Snort's default snaplen on a standard Ethernet connection (no Jumbo frames and no VLAN tags).  Drilling down into the packet capture, some of the packets were 2900 bytes and Snort was only capturing the first 1500 bytes.  The full packet capture was not "full" packet capture after all.

So where did the 2900-byte packets come from?

The OS had enabled by default the following NIC offload features:
tcp-segmentation-offload (tso)
generic-segmentation-offload (gso)
generic-receive-offload (gro)

For more information about these features and their side effects, please see:
http://www.unleashnetworks.com/blog/?p=307
http://wiki.wireshark.org/CaptureSetup/Offloading
http://manual.snort.org/node7.html
http://www.inliniac.net/blog/2007/04/20/snort_inline-and-tcp-segmentation-offloading.html

I won't repeat all the information in those links, but I'll summarize by saying that the NIC was reassembling packets before being passed up the stack to Snort.  I disabled the offload features and then verified that this resulted in no more packets larger than 1500 bytes.  The packet capture truly was "full" packet capture.

[ UPDATE: A reader asked why we couldn't simply change Snort's default snaplen to a larger value to capture the 2900-byte packets.  While it's true that would solve the "full" packet capture problem, another problem would remain.  Since the packets are being reassembled on the NIC, Snort is not able to properly perform target-based reassembly (see the Snort manual link above).  This opens the door for potential IDS evasion/insertion attacks.  NIC offload settings need to be disabled so that Snort sees the same packets the destination host does. ]

Some NIC/driver combinations may disable these offload settings by default, while others enable it by default.  You should check your sensors now before you get into a situation where you really need that full packet capture and find out that you don't actually have it.  To check, run ethtool with the "-k" (lower-case k) option on the interface you'd like to check.  For example, to check eth0:

ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off

You should repeat this for every interface in your system, as you may have NICs from different manufacturers with different defaults.

You can set these options using the "-K" (upper-case K) option to ethtool and specify which option you'd like to set.  For example, to disable tcp-segmentation-offload for eth0:

ethtool -K eth0 tso off

You can set multiple options in one "ethtool" command, but this can be problematic if your card doesn't support all of the settings.  To avoid this, you could invoke ethtool for each option like this:

ethtool -K eth0 rx off
ethtool -K eth0 tx off
ethtool -K eth0 sg off
ethtool -K eth0 tso off
ethtool -K eth0 ufo off
ethtool -K eth0 gso off
ethtool -K eth0 gro off
ethtool -K eth0 lro off 

Or we could simply wrap the ethtool command in a for-loop like this:

for i in rx tx sg tso ufo gso gro lro; do ethtool -K eth0 $i off; done

These settings will remain in effect only while the OS is booted, so this needs to be applied at every boot.  This can be done by adding the above for-loop as a "post-up" command for each of the interfaces in /etc/network/interfaces.  If you're still using the graphical Network Manager to configure your interfaces, I've put together some documentation on disabling it and configuring interfaces and their offloading features via /etc/network/interfaces:

https://securityonion.net/docs/network-configuration.html

I'd really like some feedback on this:

• What were your default settings? (ethtool -k eth0)
• Did you have any problems disabling the offload features?
• Did you notice any difference in performance after disabling the offload features?
• Is there a better way of disabling offload features globally?  I tried putting the commands in /etc/rc.local and /etc/init/securityonion.conf, but the only way I could get it to work consistently was via /etc/network/interfaces as documented above.
• I'm considering disabling offload features by default in the Security Onion Setup script.  Can anyone think of any reason why this might be a bad idea?

Security Onion: Network Security Monitoring in Minutes at BSides Atlanta

I'll be presenting "Security Onion: Network Security Monitoring in Minutes" at #BSidesATL on Friday, November 4.  For more information, please see: http://goo.gl/w2yZg.  


Hope to see you there!

In Search Of Evil User Agents

I've got a guest blog post over at PaulDotCom describing how to find evil User Agents on your network using the new httpry functionality in Security Onion:
In Search Of Evil User Agents

Security Onion 20111013 now available!

Security Onion 20111013 is now available!  This simple update resolves Issue 131.

In-place Upgrade

Existing Security Onion users can perform an in-place upgrade using the following command (if you're behind a proxy, remember to set your proxy variables as described in the FAQ):

sudo -i "curl -L http://sourceforge.net/projects/security-onion/files/security-onion-upgrade.sh > ~/security-onion-upgrade.sh && bash ~/security-onion-upgrade.sh"

Screenshots

Upgrade Process

Security Onion 20111001 now available!

Security Onion 20111001 is now available!  This simple update resolves two issues in /usr/local/bin/pulledpork_update.sh:

• Issue 127 requests that /usr/local/bin/pulledpork_update.sh determine whether it is running interactively or via crontab and perform accordingly.
• A comment on Issue 87 requests that the rule backups /etc/nsm/rules/backup/ be purged after a specified number of days.
• The default number of days is 30.
• This default can be overridden by setting the $DAYSTOKEEP_RULE_BACKUPS variable in /etc/nsm/securityonion.conf.

In-place Upgrade

Existing Security Onion users can perform an in-place upgrade using the following command (if you're behind a proxy, remember to set your proxy variables as described in the FAQ):

sudo -i "curl -L http://sourceforge.net/projects/security-onion/files/security-onion-upgrade.sh > ~/security-onion-upgrade.sh && bash ~/security-onion-upgrade.sh"