New NSM and Setup packages available

I've updated our NSM and Setup packages to resolve a few issues:

Issue 429: nsm_server_clear needs latest Squert database updates
https://code.google.com/p/security-onion/issues/detail?id=429

Issue 451: nsm_sensor_clean should purge old files in /nsm/bro/extracted
https://code.google.com/p/security-onion/issues/detail?id=451

Issue 454: Disabling PADS agent blocks PRADS and results in no SANCP
records flowing
https://code.google.com/p/security-onion/issues/detail?id=454
(thanks to Kevin Branch for the patch)

Issue 435: Setup should allow you to set PF_RING min_num_slots
https://code.google.com/p/security-onion/issues/detail?id=435

Issue 446: Setup should delete /var/lib/sphinxsearch/data/binlog*
https://code.google.com/p/security-onion/issues/detail?id=446

Issue 452: Setup phase 2 should populate sniffing interfaces from
/etc/network/interfaces
https://code.google.com/p/security-onion/issues/detail?id=452

Issue 439: /etc/cron.d/sensor-newday updates
https://code.google.com/p/security-onion/issues/detail?id=439

Issue 440: BPF JIT addition to /etc/sysctl.d/10-securityonion.conf
https://code.google.com/p/security-onion/issues/detail?id=440

The new packages are as follows:
securityonion-nsmnow-admin-scripts - 20120724-0ubuntu0securityonion66
securityonion-setup - 20120912-0ubuntu0securityonion92

They have been tested by the following (thanks!):
David Zawdie

Setup now selects interfaces based on /etc/network/interfaces

If you allow Setup to configure /etc/network/interfaces, then it will use that information later to automatically select the proper interface(s) for monitoring:

PF_RING min_num_slots
Setup now creates /etc/modprobe.d/pf_ring.conf to set parameters for the PF_RING kernel module.  If you run Quick Setup, it will just use the default value of 4096 for min_num_slots.  However, if you choose Advanced Setup, you will have the opportunity to change that default value.

If you've already run Setup and want to modify min_num_slots, you can manually create /etc/modprobe.d/pf_ring.conf.  For example, to increase min_num_slots to 65534, do the following:

echo "options pf_ring transparent_mode=0 min_num_slots=65534" | sudo tee /etc/modprobe.d/pf_ring.conf

After creating /etc/modprobe.d/pf_ring.conf, you'll need to reload the PF_RING module as follows (or just reboot):

sudo nsm_sensor_ps-stop
sudo rmmod pf_ring
sudo nsm_sensor_ps-start

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!

New securityonion-sostat package available

I've packaged a new version of sostat that resolves a few issues:

Issue 437: sostat: more detailed interface stats via ip(8)
https://code.google.com/p/security-onion/issues/detail?id=437

Issue 457: sostat: add /proc/net/pf_ring/info
https://code.google.com/p/security-onion/issues/detail?id=457

Issue 458: sostat: include pf_ring Slots
https://code.google.com/p/security-onion/issues/detail?id=458

Issue 459: sostat: netsniff-ng loss output incorrect when running BPF
https://code.google.com/p/security-onion/issues/detail?id=459

The version number of the new package is securityonion-sostat - 20120722-0ubuntu0securityonion12 and it has been tested by the following (thanks!):
David Zawdie

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!

New securityonion-setup package available

I've packaged a new version of Setup that resolves a couple of issues:

Issue 436: sosetup-network: replace ifconfig with iproute2's ip tool
https://code.google.com/p/security-onion/issues/detail?id=436
This patch from Jon Schipp updates sosetup-network so that it uses "ip" instead of "ifconfig".

Issue 441: sosetup-network shouldn't stop network-manager
https://code.google.com/p/security-onion/issues/detail?id=441
The last release of sosetup added a new question to sosetup-network to allow you to modify /etc/network/interfaces and then have the choice to not immediately reboot. If you did this over "ssh -X" it would stop Network Manager which would drop your ssh connection before it could ask if you want to reboot. It should no longer try to stop Network Manager so you should see the final reboot question when running over ssh.

The version number of the new package is securityonion-setup - 20120912-0ubuntu0securityonion90 and it has been tested by the following (thanks!):
Scott Runnels
Matt Gregory
JP Bourget

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!

New securityonion-elsa-extras package available

Scott Runnels has fixed a couple of bugs in the recent securityonion-elsa-extras package.  The updated package version is:
securityonion-elsa-extras - 20131117-1ubuntu0securityonion25

Issues Resolved

Issue 438: /etc/cron.d/elsa updates
https://code.google.com/p/security-onion/issues/detail?id=438

Issue 442: securityonion-elsa-extras: fix BRO_NOTICE parsers
https://code.google.com/p/security-onion/issues/detail?id=442

Issue 444: securityonion-elsa-extras: wrong mysql directory in /etc/elsa_node.conf
https://code.google.com/p/security-onion/issues/detail?id=444

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Update process

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!

Bro 2.2 and ELSA 1.5 packages now available

We have some new packages available for Bro 2.2 and ELSA 1.5!

ELSA 1.5 with support for Bro 2.2 and more log types

Release Notes

IMPORTANT! If you are upgrading a distributed deployment, it is vitally important that you upgrade the master before upgrading the sensors!  After upgrading the master and all sensors, if the ELSA web interface doesn't show all of your nodes properly, you may need to do the following:

• restart autossh on each sensor:
  sudo pkill -USR1 autossh
• stop/start (NOT restart) starman on each sensor:
  sudo service starman stop
  sudo service starman start
• restart Apache on your master server:
  sudo service apache2 restart

If you have email configured on your sensor and you start getting lots of email from the ELSA cron job, you can fix it by changing the last line of /etc/cron.d/elsa as follows (moving 2>&1 to the end of the line):

* * * * * root perl /opt/elsa/web/cron.pl -c /etc/elsa_web.conf > /dev/null 2>&1

If you had previously installed the APT1 scripts per http://blog.securityonion.net/2013/02/seth-halls-bro-module-for-apt1-detection.html, the update will detect this and automatically enable the new version of the APT1 scripts.  If you would like to manually enable the APT1 scripts, do the following:

sudo sed -i 's|#@load apt1|@load apt1|g' /opt/bro/share/bro/site/local.bro
sudo broctl check && sudo broctl install && sudo broctl restart

The new version of Setup asks if you want to configure Bro to extract files (EXEs by default).  If you've already run Setup and want to enable file extraction, do the following:

sudo sed -i 's|#@load file-extraction|@load file-extraction|g' /opt/bro/share/bro/site/local.bro
sudo broctl check && sudo broctl install && sudo broctl restart

The new version of Setup configures Snorby to allow you to pivot from an IP address in Snorby to an ELSA query for that IP address.  If you've already run Setup and want to add this capability to Snorby, click Administration and then click Lookup Sources and add the following (also see screenshot in the Screenshots section):

https://elsa.ip.addr.ess:3154/?query_string="${ip}"%20groupby:program

Issues Resolved

Issue 362: sguil-db-purge - add DAYSTOREPAIR option
https://code.google.com/p/security-onion/issues/detail?id=362

Issue 395: Bro 2.2
https://code.google.com/p/security-onion/issues/detail?id=395

Issue 426: Update http_agent for Bro 2.2
https://code.google.com/p/security-onion/issues/detail?id=426

Issue 420: Setup should no longer disable Bro PF_RING since it should
work in 2.2
https://code.google.com/p/security-onion/issues/detail?id=420

Issue 424: Setup should write out changes to /etc/network/interfaces
and then prompt for reboot
https://code.google.com/p/security-onion/issues/detail?id=424

Issue 415: Setup should ask user about DAYSTOKEEP and DAYSTOREPAIR
https://code.google.com/p/security-onion/issues/detail?id=415

Issue 396: Setup should give the option of enabling file extraction in Bro
https://code.google.com/p/security-onion/issues/detail?id=396

Issue 433: Setup should configure Snorby to pivot from an IP address to ELSA
https://code.google.com/p/security-onion/issues/detail?id=433

Issue 431: Update APT1 scripts for Bro 2.2
https://code.google.com/p/security-onion/issues/detail?id=431

Issue 350: Modify Sguil client to allow pivoting directly to ELSA query
https://code.google.com/p/security-onion/issues/detail?id=350

Issue 346: New ELSA packages
https://code.google.com/p/security-onion/issues/detail?id=346

Issue 343: Add more Bro logs to ELSA
https://code.google.com/p/security-onion/issues/detail?id=343

Issue 434: nsm_sensor_ps-start shouldn't call sensor_cleandisk anymore
https://code.google.com/p/security-onion/issues/detail?id=434

New/Updated packages

securityonion-bro - 2.2-0ubuntu0securityonion9
securityonion-bro-scripts - 20121004-0ubuntu0securityonion17
securityonion-elsa - 1090-1ubuntu0securityonion11
securityonion-elsa-extras - 20131117-1ubuntu0securityonion19
securityonion-elsa-node-perl - 20130819-0ubuntu0securityonion2
securityonion-elsa-web-perl - 20131029-0ubuntu0securityonion0ubuntu1
securityonion-http-agent - 0.3.1-0ubuntu0securityonion3
securityonion-libapache-logformat-compiler-perl - 0.13-0ubuntu0securityonion1
securityonion-libcapture-tiny-perl - 0.22-0ubuntu0securityonion0
securityonion-libclass-method-modifiers-perl - 2.04-1ubuntu0securityonion1
securityonion-libcookie-baker-perl - 0.01-1ubuntu0securityonion1
securityonion-libdevel-stacktrace-perl - 1.30-1ubuntu0securityonion0
securityonion-libexception-class-perl - 1.37-1ubuntu0securityonion1
securityonion-libextutils-config-perl - 0.007-1ubuntu0securityonion0
securityonion-libextutils-helpers-perl - 0.021-1ubuntu0securityonion0
securityonion-libextutils-installpaths-perl - 0.009-1ubuntu0securityonion0
securityonion-liblog-log4perl-appender-socket-unix-perl - 1.04-1ubuntu0securityonion0
securityonion-liblog-syslog-constants-perl - 1.02-1ubuntu0securityonion0
securityonion-liblog-syslog-fast-perl - 0.61-1ubuntu0securityonion1
securityonion-libmoo-perl - 1.003-1ubuntu0securityonion1
securityonion-libmoox-types-mooselike-numeric-perl - 1.01-1ubuntu0securityonion1
securityonion-libmoox-types-mooselike-perl - 0.25-1ubuntu0securityonion0
securityonion-libplack-middleware-xforwardedfor-perl - 0.1030-1ubuntu0securityonion0
securityonion-librole-tiny-perl - 1.003-1ubuntu0securityonion1
securityonion-libtest-name-fromline-perl - 0.11-1ubuntu0securityonion1
securityonion-libtest-time-perl - 0.04-1ubuntu0securityonion1
securityonion-nsmnow-admin-scripts - 20120724-0ubuntu0securityonion64
securityonion-setup - 20120912-0ubuntu0securityonion89
securityonion-sguil-client - 0.8.0-0ubuntu0securityonion15
securityonion-sguil-db-purge - 20120722-0ubuntu0securityonion7

The new packages have been tested by the following (thanks!):

Heine Lysemose
JP Bourget
Matt Gregory
David Zawdie

Screenshots

Bro update

ELSA update

ELSA update with support for more Bro logs

http_agent update

New Sguil client supports pivoting from IP address to ELSA query

Pivoting from Sguil/Snorby to ELSA

Manually adding ELSA as a Lookup Source after running Setup

Pivoting from Snorby to ELSA

New Setup screen for DAYSTOKEEP

New Setup screen for DAYSTOREPAIR

New Setup screen for enabling Bro file extraction

ELSA query for BRO_SOFTWARE

ELSA query for BRO_FILES

ELSA query for BRO_NOTICE

ELSA query for BRO_WEIRD

Updating
The new packages are now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!