Thursday, June 15, 2017

Quick Intro to Security Onion Elastic Stack Technology Preview 2

Here's a video showing the installation of Security Onion Elastic Stack Technology Preview 2 and a brief introduction to the interface and workflow:
https://www.youtube.com/playlist?list=PLljFlTO9rB15SMpdBpLi084FiTBJsBjXZ

We'd love your feedback!  Please send it to our mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists

Thanks!

Tuesday, June 6, 2017

Friday, June 2, 2017

Towards Elastic on Security Onion: Technology Preview 2 (TP2)

We recently announced our move towards the Elastic stack:
http://blog.securityonion.net/2017/03/towards-elk-on-security-onion.html

In the last few weeks, we've made tremendous progress, so it's time for our second technology preview (TP2)!

Changes from the last Technology Preview
  • upgraded from Elastic 2.4.4 to 5.4.0
  • Elasticsearch, Logstash, and Kibana each run in their own Docker containers
  • lots more dashboards
  • new Logstash parsers to support more log types
  • IPv6 support
  • experimental script to migrate data from ELSA to Elastic
  • Squert now leverages the same single sign on as Kibana and CapMe

Warnings and Disclaimers
  • This technology PREVIEW is PRE-ALPHA, BLEEDING EDGE, and TOTALLY UNSUPPORTED!
  • If this breaks your system, you get to keep both pieces!
  • This script is a work in progress and is in constant flux.
  • This script is intended to build a quick prototype proof of concept so you can see what our ultimate Elastic configuration might look like.  This configuration will change drastically over time leading up to the final release.
  • Do NOT run this on a system that you care about!
  • Do NOT run this on a system that has data that you care about!
  • This script should only be run on a TEST box with TEST data!
  • This script is only designed for standalone boxes and does NOT support distributed deployments.
  • Use of this script may result in nausea, vomiting, or a burning sensation.

Enough disclaimers?  Let's do this!

Start with a disposable TEST VM with the following minimum requirements:

  • 2 CPU cores
  • 8GB RAM
  • 20GB virtual hard drive
  • (1) management interface with full Internet access
  • (1) sniffing interface (separate from management interface)
  • Security Onion 14.04.5.2 ISO image installed
  • Setup ran in Evaluation Mode
Download the script:
wget https://raw.githubusercontent.com/Security-Onion-Solutions/elastic-test/master/securityonion_elsa2elastic.sh
Run the script with sudo privileges:
sudo bash securityonion_elsa2elastic.sh
Please read through all the WARNINGS and DISCLAIMERS and ONLY proceed if you agree.

The script will take at least 10 minutes depending on the speed of your hardware and Internet connection.  At the end of the script, it will prompt you to access Kibana via the following URL:
https://localhost/app/kibana

You should then see our new Security Onion login window.  Enter the same credentials that you use to login to Sguil/Squert.  This login window will provide single sign on for Kibana, Squert, and CapMe to allow seamless pivoting to full packet capture!

Once logged into Kibana, you will automatically start on our Overview dashboard and you will see links to other dashboards as well.  As you search through the data in Kibana, you should see Bro logs,  syslog, and Snort alerts.  Logstash should have parsed out most fields in most Bro logs and Snort alerts.  Notice that the search panels at the bottom of the dashboards display the source_ip and destination_ip fields with hyperlinks.  These hyperlinks will take you to a dashboard that will help you analyze the traffic relating to that particular IP address.  UID fields are also hyperlinked.  Clicking on a UID hyperlink will start a new Kibana search for that particular UID.  In the case of Bro UIDs this will show you all Bro logs related to that particular connection.  Each log entry also has an _id field that is hyperlinked.  This hyperlink will take you to CapMe, allowing you to request full packet capture for any arbitrary log type!  This assumes that the log is for tcp or udp traffic that was seen by Bro and Bro recorded it correctly in its conn.log.

Previously, in Squert and Sguil, you could pivot from an IP address to ELSA.  Those pivots have been removed and replaced with a pivot to Kibana.

For screenshots, please see the Screenshot Tour at the bottom of this post.

TODO
For the current TODO list, please see:
https://github.com/Security-Onion-Solutions/security-onion/issues/1095

Feedback
We're releasing this now because we want to get your feedback as early as possible in this project.  Please try it out and send your feedback to our mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists

What do you think?

What works well?

What needs to be improved?

Any questions or other comments?

Thanks in advance for any and all feedback!

Want to show your support for Security Onion?
Security Onion t-shirts are available in our CafePress store!
http://www.cafepress.com/securityonion/11820053

Need Security Onion Training?
We offer both onsite and online training (although please note that Elastic will not be added to training classes until we reach a stable release):
https://securityonionsolutions.com/onsitetraining
https://securityonionsolutions.com/onlinetraining

Conference
Our annual Security Onion Conference will be Friday September 15, 2017:
https://securityonion.net/conference

Hope to see you there!

Screenshot Tour

Overview Dashboard


Bro Notices Dashboard


HIDS Alerts Dashboard


NIDS Alerts Dashboard 

NIDS alerts now contain the rule that generated the alert

Bro Connections Dashboard



Bro Connections Traffic Map


IPv6 Support


Bro DCE/RPC Dashboard


Bro DHCP Dashboard


Bro DNP3 Dashboard


Bro DNS Dashboard 


Bro Files Dashboard




Bro FTP Dashboard


Bro HTTP Dashboard


Bro Intel Dashboard


Bro IRC Dashboard


Bro Kerberos Dashboard 


Bro Modbus Dashboard



Bro MySQL Dashboard


Bro NTLM Dashboard


Bro PE Dashboard


Bro RADIUS Dashboard


Bro RDP Dashboard


Bro RFB Dashboard


Bro SIP Dashboard


Bro SMB Dashboard


Bro SMTP Dashboard


Bro SNMP Dashboard


Bro Software Dashboard


Bro SSH Dashboard


Bro SSL Dashboard


Bro Tunnels Dashboard


Bro Weird Dashboard


Bro X.509 Dashboard


Host Logs Dashboard

Stats Dashboard

Each Dashboard has a search panel with important fields hyperlinked

Clicking the Source IP hyperlink takes you to the Indicator Dashboard searching for the Source IP

Click the Destination IP hyperlink takes you to the Indicator Dashboard searching for the Destination IP

Clicking the UID hyperlink takes you to the Indicator Dashboard searching for that UID


Clicking the _ID hyperlink takes you to CapMe for full packet capture


Clicking the Squert link in Kibana takes you directly to Squert thanks to Single Sign On


SSO allows you to pivot seamlessly from Squert to CapMe for full packet capture


New scripts to manage Elastic stack


Tuesday, May 30, 2017

Less than 2 weeks left to register for 4-day Security Onion Training in Alexandria VA!

Registration for our 4-day training in Alexandria VA closes on June 12, so there are less than two weeks left to sign up!

For more details and to register, please see:
https://securityonionsolutions.com/onsitetraining

Monday, May 22, 2017

Only 3 weeks left to register for 4-day Security Onion Training in Alexandria VA!

Registration for our 4-day training in Alexandria VA closes on June 12, so there are only three weeks left to sign up!

For more details and to register, please see:
https://securityonionsolutions.com/onsitetraining

Friday, May 12, 2017

Only 1 month left to register for 4-day Security Onion Training in Alexandria VA!

Registration for our 4-day training in Alexandria VA closes on June 12, so there is only one month left to sign up!

For more details and to register, please see:
https://securityonionsolutions.com/onsitetraining

Monday, May 8, 2017

Only 1 week left for Early Bird discount for 4-day training in Augusta GA!

The earlybird discount for the 4-day training class in Augusta GA is still valid for one more week (expires on May 15).  When registering, please enter the following promotional code to receive your 15% discount!
earlybird

For more details and to register, please see:
https://securityonionsolutions.com/onsitetraining

Tuesday, April 18, 2017

Wes Lambert has joined Security Onion Solutions LLC as Senior Engineer

If you've been a part of the Security Onion community for any amount of time, chances are you've seen Wes Lambert answer questions, test new packages, and submit pull requests.  I'm excited to announce that Wes is now an official employee of Security Onion Solutions LLC.

Congratulations, Wes, and welcome aboard!

Friday, April 14, 2017

Security Onion Conference 2017 CFP

This year's Security Onion Conference will be held in Augusta, GA on Friday, September 15, 2017 (please mark your calendar!). Registration will open in June.

CFP

Want to speak at Security Onion Conference? We want to hear from you!

How are you...
...using Security Onion to fight evil?
...handling lots of traffic using Security Onion?
...integrating Security Onion with other technologies?
...automating common tasks with your own scripts?

Each talk should be 30 minutes with an additional 10 minutes for questions.

UPDATE 2017-06-09: CFP is now closed!

Schedule

April 14 - CFP open
June 5 - CFP closes
June 29 - Speakers selected and notified
June 29 - Registration opens
September 11-14 - Security Onion 4-day training in Augusta
September 15 - Security Onion Conference
September 16 - BSides Augusta

Thursday, March 16, 2017

Towards ELK on Security Onion: A Technology Preview

UPDATE: We've released a newer preview! Please see:
http://blog.securityonion.net/2017/06/towards-elastic-on-security-onion.html

Over the last few years, we've had lots of folks ask for ELK (Elasticsearch, Logstash, and Kibana) on Security Onion.  The time has come to begin working towards ELK on Security Onion!

In the grand tradition of "release early, release often", we're releasing a very early Technology Preview of what ELK on Security Onion might look like.  This Technology Preview consists of a script that will take a Security Onion VM in Evaluation Mode and convert it from ELSA to ELK.  We're releasing this now because we want to get your feedback as early as possible in this project.

Thanks
Special thanks to Justin Henderson for his Logstash configs and installation guide!
https://github.com/SMAPPER/Logstash-Configs

Special thanks to Phil Hagen for all his work on SOF-ELK!
https://github.com/philhagen/sof-elk

Warnings and Disclaimers

  • This technology PREVIEW is PRE-ALPHA, BLEEDING EDGE, and TOTALLY UNSUPPORTED!
  • If this breaks your system, you get to keep both pieces!
  • This script is a work in progress and is in constant flux.
  • This script is intended to build a quick prototype proof of concept so you can see what our ultimate ELK configuration might look like.  This configuration will change drastically over time leading up to the final release.
  • Do NOT run this on a system that you care about!
  • Do NOT run this on a system that has data that you care about!
  • This script should only be run on a TEST box with TEST data!
  • This script is only designed for standalone boxes and does NOT support distributed deployments.
  • Use of this script may result in nausea, vomiting, or a burning sensation.

Bring on the ELK
Enough disclaimers?  Let's do this!

Start with a disposable TEST VM with the following minimum requirements:

  • 2 CPU cores
  • 4GB RAM
  • 20GB virtual hard drive
  • (1) management interface with full Internet access
  • (1) sniffing interface (separate from management interface)
  • Security Onion 14.04.5.2 ISO image installed
  • Setup ran in Evaluation Mode

Download the script:
wget https://raw.githubusercontent.com/Security-Onion-Solutions/elastic-test/master/securityonion_elsa2elastic.sh
Run the script with sudo privileges:
sudo bash securityonion_elsa2elastic.sh
Please read through all the WARNINGS and DISCLAIMERS and ONLY proceed if you agree.

The script will take at least 10 minutes depending on the speed of your hardware and Internet connection.

After a minute or two, you should be able to access Kibana via the following URL:
https://localhost/app/kibana

You should see our new Security Onion login window.  Enter the same credentials that you use to login to Sguil and Squert.  This login window will provide single sign on for both Kibana and CapMe to allow seamless pivoting to full packet capture!

Once logged into Kibana, you will automatically start on our Overview dashboard and you will see links to other dashboards as well.  These dashboards are designed to work at 1024x768 screen resolution in order to maximize compatibility.

As you search through the data in Kibana, you should see Bro logs, syslog, and Snort alerts.  Logstash should have parsed out most fields in most Bro logs and Snort alerts.

Notice that the search panels at the bottom of the dashboards display the source_ip and destination_ip fields with hyperlinks.  These hyperlinks will take you to a dashboard that will help you analyze the traffic relating to that particular IP address.

UID fields are also hyperlinked.  This hyperlink will start a new Kibana search for that particular UID.  In the case of Bro UIDs this will show you all Bro logs related to that particular connection.

Each log entry also has an _id field that is hyperlinked.  This hyperlink will take you to CapMe, allowing you to request full packet capture for any arbitrary log type!  This assumes that the log is for tcp or udp traffic that was seen by Bro and Bro recorded it correctly in its conn.log.  CapMe should try to do the following:

  • retrieve the _id from Elasticsearch
  • parse out timestamp
  • if Bro log, parse out the CID, otherwise parse out src IP, src port, dst IP, and dst port
  • query Elasticsearch for those terms and try to find the corresponding bro_conn log
  • parse out sensor name (hostname-interface)
  • send a request to sguild to request pcap from that sensor name

Previously, in Squert, you could pivot from an IP address to ELSA.  That pivot has been removed and replaced with a pivot to ELK.

Screenshots
Using wget to download the script


Running the script as root with "sudo bash securityonion_elsa2elk.sh"

TODO and HARDWARE REQUIREMENTS

Thanks to Justin Henderson and Phil Hagen!

WARNINGS and DISCLAIMERS

Instructions at end of script

New Security Onion login window (use your existing Sguil/Squert credentials) provides single sign on for both Kibana and CapMe

Overview Dashboard contains graphs and links to other dashboards

All of our dashboards include a search panel at the bottom so you can quickly drill into details

Indicator Dashboard is great for seeing the most interesting data types for a particular IP address

Notices Dashboard shows Bro Notices

NIDS Dashboards shows NIDS alerts from Snort or Suricata 
Bro_conn Dashboard allows you to slice and dice Bro's conn.log



Bro_dns Dashboard allows you to slice and dice Bro's dns.log

Bro_http Dashboard allows you to slice and dice Bro's http.log

Bro_ssl Dashboard allows you to slice and dice Bro's ssl.log

Scrolling down the Bro_http Dashboard, we see raw logs with hyperlinks to pivot to further information

Clicking the source IP address in the previous screenshot takes us to the Indicator Dashboard for the source IP

Clicking the destination IP address takes us to the Indicator Dashboard for the destination IP

Clicking the uid field takes us to the Indicator Dashboard for the Bro connection ID

Clicking the _id hyperlink takes us to CapMe to retrieve full packet capture for that stream

Feedback
We're releasing this now because we want to get your feedback as early as possible in this project.  Please try it out and send your feedback to our mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists

What do you think?

What works well?

What needs to be improved?

Any questions or other comments?

Thanks in advance for any and all feedback!

UPDATE 2017-03-16 Fixed link to Justin Henderson's github repo

UPDATE 2017-06-01 Renamed github repo from elk-test to elastic-test

UPDATE 2017-06-03 Added link to Technology Preview 2