I've got a guest blog post over at PaulDotCom describing how to find evil User Agents on your network using the new httpry functionality in Security Onion: In Search Of Evil User Agents
httpry is a great tool, but there is a minor gotcha in that it doesn't do any defragmentation or TCP segment reassembly etc. It only considers client->server segments that begin with one of the specified HTTP methods, and server->client segments that begin with "HTTP".
This means that you may not always get complete information in the httpry output, most notably in the field that denotes the Host: header. This can happen when the Host: header isn't in the first TCP segment that the client sends to the server, either because that's just the way the browser sent it, or because of malicious intent (evasion).
tshark can do a similar job, complete with reassembly code. Fiddle with the -e fields as required:
Thanks for your comment. Ultimately, we may end up replacing httpry with Suricata's HTTP logging which should have no problems with defragmentation or reassembly.
Hi Doug,
ReplyDeletehttpry is a great tool, but there is a minor gotcha in that it doesn't do any defragmentation or TCP segment reassembly etc. It only considers client->server segments that begin with one of the specified HTTP methods, and server->client segments that begin with "HTTP".
This means that you may not always get complete information in the httpry output, most notably in the field that denotes the Host: header. This can happen when the Host: header isn't in the first TCP segment that the client sends to the server, either because that's just the way the browser sent it, or because of malicious intent (evasion).
tshark can do a similar job, complete with reassembly code. Fiddle with the -e fields as required:
sudo tshark -i eth1 -R http.request -T fields -e http.host -e http.request.uri
It's a somewhat heavier option, though :)
alec
Hi Alec,
ReplyDeleteThanks for your comment. Ultimately, we may end up replacing httpry with Suricata's HTTP logging which should have no problems with defragmentation or reassembly.
Thanks,
Doug