New ELSA and Sphinx packages now available

Scott Runnels has been hard at work updating our ELSA packages and building our own custom Sphinx package!  These new packages should resolve the following issues:

Issue 290: Update ELSA to r713
Issue 289: ELSA - include YUI library
Issue 300: /etc/elsa_web.conf needs 127.0.0.1 to have ports defined as well
Issue 298: Build new sphinx package with --enable-id64 compile-time option
Issue 324: sphinx should check for proper permissions before starting
Issue 299: sphinx.conf - swap "3307" with "9312"
Issue 327: Remove sphinx default cronjob as it is unnecessary and can cause issues

The new packages have been tested by the following (thanks!):
Brad Shoop
David Zawdie
Matt Gregory

UPDATE 5/3 21:18 - We have reports of issues with the sphinxsearch upgrade.  Please do not upgrade until we've determined the root cause.

UPDATE 5/4 00:05 - We've determined the root cause and are trying to determine the best fix.

UPDATE 5/4 13:00 - We're currently building a new package.  Will update later today after it has finished building and has been tested.

UPDATE 5/5 08:24 - The new sphinxsearch package has had some initial testing which appears to be successful. If you can test it in a non-production environment, we'd appreciate any feedback on our mailing list.

UPDATE 5/7 07:21 - Added the "Cleaning Up Perl Processes" and "Rebuilding Indexes" sections below.

UPDATE 5/7 09:45 - Added the "Known Issues" section below.

Updating
The new packages are now available in our stable repo. You can initiate the upgrade process using the graphical Update Manager or using the following one-liner:

sudo apt-get update && sudo apt-get dist-upgrade

Warning
Ubuntu recently released some MySQL updates, so you may also be prompted to update MySQL at the same time.  If so, please cancel the update and use our recommended procedure for updating MySQL:
http://code.google.com/p/security-onion/wiki/MySQLUpdates

Cleaning Up Perl Processes
One of the issues fixed in this release was the extra perl processes occurring as a result of the ELSA LiveTail feature.  LiveTail has been disabled in these new packages, but you may still have some extra perl processes hanging around from before the upgrade.  You can resolve this by rebooting (Ubuntu recently released some kernel updates so you may need to do this anyway) or by doing the following:

sudo service syslog-ng stop && sudo pkill -9 perl && sudo service syslog-ng start

Rebuilding Indexes
Another issue fixed in this release is that sphinxsearch is now compiled with id64. Previously, we were using the stock Ubuntu package of sphinxsearch which used the CRC32 algorithm and could result in keyword collisions, meaning that you get results that don't actually match what you were searching for. To ensure that all of your indexes are using the new id64 support, you should reindex as follows (note this may take anywhere from minutes to hours):

sudo indexer --rotate --all

Known Issues
If you access ELSA from a browser whose local timezone is not UTC *and* you haven't enabled the use_utc setting in your ELSA Preferences, then each search rolls the From time back the same number of hours as the UTC offset.  For example, suppose your local workstation is set to Eastern time and you login to ELSA and notice that the From defaults to:
2013-05-05 18:01:50

When you then perform a search, the From changes to:
2013-05-05 14:01:50

The workaround is to enable the use_utc setting in your ELSA Preferences (which is probably a good idea anyway to ensure that your timestamps in ELSA match your timestamps in Sguil/Squert/Snorby):
https://code.google.com/p/enterprise-log-search-and-archive/wiki/Documentation#Preferences

Screenshots

Upgrade Process

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!