Passive DNS and PassiveDNS/PRADS

For those of you not familiar with the concept of Passive DNS, there are lots of stuff on it on the intertubes…

Just some of the links:
Some use cases: http://conferences.npl.co.uk/satin/presentations/satin2011slides-Rasmussen.pdf
A public passive dns db: http://www.bfk.de/bfk_dnslogger.html?query=sans.org#result
Or just click here: http://lmgtfy.com/?q=passivedns

I have not found any good tools yet that lets you build your own passive DNS DB, so I have started to walk down that path…
First off, I have coded a DNS sniffer (passivedns) I have ported the same functionality over into PRADS. All code is in beta at the moment.

I announce this release, so if anyone is interested, I will take input on the output format 🙂
My first tests shows that the passive DNS data collected on a small network is too much… My plan is to implement a in memory “state” so that it don’t prints out the same record more than X times over a time interval (say, if a record is the same, just print it once a day, but if it changes, print it immediate). When that is done, Ill write a parser to feed it into a DB and a query tool to fetch passive DNS records on request.

Feedback is always welcome!

Advertisement

Packetcapture with Snort using the “tag” option

I did this several years ago, but when I switched to full packetcapture I did not have the need for catching pcap of traffic firing a rule.

You can do this with the tag option in Snort. If you want to know more, please read README.tag.

I will present you with a signature that will log the first 1000 bytes or 100 seconds (What ever comes first!) after the packet that triggered the event. Im looking for a SYN flag in a TCP session and I start my logging from there (0,packets means that there are no limits on amount of packets).

alert tcp 85.19.221.54 any <> $HOME_NET any (msg:”GL Log Packet Evil-IP 85.19.221.54 (gamelinux.org)”; flags:S; tag:session,1000,bytes,100,seconds,0,packets; classtype:trojan-activity; sid:201102011; rev:1;)

I use unified2 as output plugin for Snort (something that also Sourcefire 3D does IIRC), so I need to fetch the pcap from the unified log. Snort 2.9.0 and newer ships with a new tool that will help you here, u2boat. This will carve out the pcaps from the unified log:

# u2boat /var/log/snort/<unified.log.timestamp> /tmp/snort.pcap


From there, you can read the /tmp/snort.pcap with tcpdump or wireshark etc. or just fetch the evil-IP packets:

# tcpdump -r /tmp/snort.pcap -w /tmp/Evil-85.19.221.54-traffic.pcap 'host 85.19.221.54'


If you love it in console, you can read the pcap with tcpflow etc:

# tcpflow -c -r /tmp/Evil-85.19.221.54-traffic.pcap


I did could not seem to verify that the “0,packets” actually do work. I added the following line also to my snort.conf:

config_tagget_packet_limit: 0

But again, not sure if it works.

I wanted to do some more testing before releasing this blog, but it has been sitting around for a while, so If I play more with it and have something new, Ill post a new post 🙂

BTW, turning you Sourcefire 3D into a packetcapture device is easy 🙂 adding the rule as above, you can just click the “Download Packet(s)” Button in the Event Information/Packet Information view 🙂 Use such a rule with care though…

10 years of gamelinux.org….

January 2011 gamelinux.org has its 10th birthday…

Did you know that gamelinux.org started out as the website for GamelinuX, a linux distribution for gaming?
I never got a working release that I wanted to present to the public, and after 2 years of working on the GamelinuX distro, the project came to an halt, as my Master degree and personal life took too much time from hacking on the distro. The GamelinuX project got official dead in September 2001 And thinking of it now… do I have copies of the Alpha CDs somewhere??? I should have, but I dont know where…

My first security related post was in July 2003, when Free-X released an exploit for Xbox, that would let you install linux on it…

In March 2007, the blog entered its current form, leaving phpnuke/drupal (and clones) for wordpress.

Gamelinux.org has always been about Open Source and hacking (‘as in finding a way to make things work’). As I started to play with Linux in 1998, Linux has been my OS of choice since. My reasons for continuing to blog security related topics on this domain, was that “Game Linux” was for me also associated with “gaming linux”, meaning “hunting linux” – find ways to break it/exploit it.

I went online for the first time with my Linux machine in 1998, and went to IRC/EFnet and the channel #Oslo. I asked anyone if they where into hacking/cracking, and asked for pointers on where/how to best start reading and learning more about it. Not long after, some guy told me to look in my /root/ directory and there was a dir that had a dozen of exploits… I realized that I had been hacked, and decided then not to get back online before I knew more about how to protect my self. The sploit used, IIRC, was a buffer overflow in wu-ftpd that shipped with the Red Hat release then, and wu-ftpd was default enabled 🙂

I stayed offline for about 2 months with my Linux machine, using the university machines to read more about hardening linux, firewalling, IDS, HIDS and such… As long as I can remember, I have been interested in hacking/cracking and defending from it. So linux+security has been an active interest for ~13 years now, and with my first related job experience ~10 years ago working for a Managed Security Service Provider (MSSP).

Thinking back the last 15 years, it has been some good years. I love what I’m doing and I have no plans on quitting!

Yet another rule manager for VRT/ET/ETPRO or Suricata/Snort rules…

As I installed a new home router/firewall some months back, I installed it with an IDS (Sguil) just to have something to play with at home. I never got comfy with oinkmaster or pulledpork as I had to dig into config files too much…

Based on my idea for sidrule on how to manage rules, and also baring in mind cerdo, I quickly made a sidrule like tool in perl. I talked to some people about it and they liked my approacher on how to do rule management. I got some very positive feedback, so I decided to rewrite it and publish the code (Get polman 0.3.1 here).

To cope with not having a configuration file, you have to start polman with the –configure option to add a RuleDB (or more) and a Sensor (or more). A RuleDB is a “database” that holds rules. The idea is that you can load Sourcefire VRT rules for Snort version X.X into one RuleDB, and you can also load Emerging Threats rules for Snort X.X into the same RuleDB and have nice set of rules to play with. As I’m currently testing Suricata, I can with the same tool, and without any extra configfile or too much hassle, make a second RuleDB, but this time one with the VRT rules and with the ET suricata rules in it. I have one Sensor attached to the Snort RuleDB, and the other one to the Suricata RuleDB. One tool too rule them all…(LOTR->Lord of the Rules?).

A bit back to the –configure option. This will enter a ascii menu where you can add and edit RuleDBs and Sensors. For a RuleDB, you specify where to load the rules from (Currently just filesystem, but http/https is scheduled for next release), name, comment, etc. For a Sensor, you specify name, comment, what RuleDB to use, where to write rules to (a file, for writing all rules to a file or a dir, for writing all rules out into their original filename in that dir), where to write sid-msg.map file, etc.

Once a RuleDB is set up, you can eater from the menu load rules into the RuleDB or from command line. Once a Sensor is setup, and the specified RuleDB has rules in it, you can start to play with the Sensor rules. If you choose to write out rules from the menu straight away, it will turn on all rules that are default turned on by vendor (VRT/ET/ETPRO etc.). When you are back at command line, you can start turning on/off rules. One way I start, is to disable categories that I normally would not enabled in my setup, example:

polman.pl -i SensorA -m "(dos|games|icmp_info|pop3|rpc|scada|scan|snmp|sql|voip)"

This will search through all the rules based on the filename that the rules was loaded from. So from only the “sql” entry in my regexp above, it would typically show you all rules that where in the files: emerging-sql.rules, mysql.rules and sql.rules.
You will then be faced with some questions… Disable them all? or Enable them all? (With this particular search, I disable all the rules). There is also a third option, and that is to go through the rules, rule for rule, to make a conscious decision after you read the raw rule. If you choose to enable all rules, or based on rule for rule, you can change behavior of the rules to. Most rules are set to action “alert“, but if you choose to enable rules, you will have the option to change action for rules (to alert,log,pass,drop,reject,sdrop,default or current). The option “default” and “current” may need some explanation… Default will set the rule to the default action that was set by the vendor. Current will leave the rule(s) as they are defined for the sensor (say you have set an alert rule to drop before, the rule will keep the current action for the rule, which is drop).

There are some powerful searches built into polman. You can search in field classtype, metadata and msg at the same time. You can also search in “fields” ‘default enabled/disabled’ (Vendor state of the rule) and category (which is default filename where the rule was loaded from).
Example:
Say you want to search all rules that VRT classifies in their most secure config:
polman.pl -i SensorA -p "policy security-ips drop"
(You can then enable them all, and set action to drop).
Say you want to limit your search, and you only want to search for Zeus/Zbot traffic…
polman.pl -i SensorA -p "policy security-ips drop" -s "(Zeus|Zbot)"
Now you can enable them all, and set action to drop 😉

If you have a sid, or a list of sids, you can enable (-e) or disable (-d) them rather easy…
polman.pl -i SensorA -e sid1,sid2,sid3,....,sidN

To load rules into a RuleDB from command line:
polman.pl -r RuleDB1 -u

To write out rules for a sensor to a file (or files if a dir is specified):
polman.pl -i SensorA -w
This also writes out the sid-msg.map file…

There is nothing wrong about reusing the same Sensor rules across multiple sensor. Indeed that is one of the reasons that I choose the name policy manager, as they don’t need to be looked upon as Sensors that you define, but also Policies (I have thought about naming of Sensor/Policies a lot). In the future I hope to finish the implementation of Thresholding and Suppression too, so that you can edit them quickly from command line.

Thoughts and feedback are welcome on this blog or here!
Project on github here.
An example on how to use polman here.

OISF Suricata 1.1.0 beta 1 debian package for Ubuntu 10.04

I also got time to put together a package for the latest version of Suricata, namely 1.1 beta1.

My plan was to stick to a stable version when OISF released 1.0.3, but they skipped that, and went for a 1.1 release instead.
As I also try to help out where I can, I don’t mind running beta software, and reporting bugs etc. when and if I can. I’ll probably pack beta2 and so on until OISF hits a stable release, and then I’ll stick with that in my gamelinux PPA. So until then, I hope you try out Suricata with me on the quest for a stable release 🙂

Read more about suricata 1.1 beta 1 here.

Sourcefire daq-0.5 and Snort-2.9.0.3 debian packages for Ubuntu 10.04

Well, I did get a small hour to play today, so I packed updated versions for snort and daq, namely Snort-2.9.0.3 and daq-0.5.

You can read some more details about my last build of the packages here.

My PPA can be found here.

Comments and suggestions are welcome 🙂

Sourcefire daq-0.4 and Snort-2.9.0.2 debian packages for Ubuntu 10.04

Moving to the new Snort 2.9 version, it added dependencies on a new library, namely DAQ(Data Acquisition library) for packet I/O.

So the little extra of packaging a new deb (daq) and check snort-debian files that they where compliant to the new version, made me debianize Suricata instead, as I saw that as quicker way to get an IDS up and running on my new firewall at home.

Now that I have suricata in place, plus some extra time last night, and I see people struggling trying to install/upgrade to Snort 2.9 on Ubuntu, I could not help my self trying to be helpful, again…

So I made debian packages and put them in my Ubuntu 10.04 Lucid PPA on launchpad. I started a new clean debian package for Snort. Its not yet packed with “debian-easy-features”, so it just installs Snort, makes the directories and adds some default configuration files. I will improve this as I go.

DAQ is built with:

Build AFPacket DAQ module.. : yes
Build Dump DAQ module…… : yes
Build IPFW DAQ module…… : yes
Build IPQ DAQ module……. : no
Build NFQ DAQ module……. : no
Build PCAP DAQ module…… : yes

And Snort is compiled with:

–enable-perfprofiling
–enable-ipv6
–enable-sourcefire
–enable-dynamicplugin
–enable-targetbased
–enable-zlib
–enable-ppm
–enable-gre
–enable-mpls
–enable-decoder-preprocessor-rules
–without-mysql
–without-postgresql

So, if you add my PPA, you apt-get install snort version 2.9.0.2. Pronto though, Snort 2.9.0.3 will be out, and I’ll upgrade accordingly. Suricata will also soon be out in 1.0.3, hopefully this week. Maybe we get fresh releases from this Santa for both engines 🙂

Until then,

-*> Snort! <*-
Version 2.9.0.2 IPv6 GRE (Build 92)
By Martin Roesch & The Snort Team: http://www.snort.org/snort/snort-team
Copyright (C) 1998-2010 Sourcefire, Inc., et al.
Using libpcap version 1.0.0
Using PCRE version: 7.8 2008-09-05
Using ZLIB version: 1.2.3.3

nftracker – The Network File Tracker…

To fulfill my dream of automatic carving of files from network traffic, I wrote nftracker. The software is not 100% done, but well enough to deserve a blog post and to get a wider audience for testing! Some more file signatures could be added, especially for “Content-Type: ” in http or smtp traffic.

( I know I could have done something similar just writing snort/suricata rules. I could even write a snort preprocessor.. But hey! )

I also want to graph info from nftracker, such as how many files of type X traverse my network today, last week, month, year, etc..

A common first question from people is: Does it also carve out the files?
Answer: No

At this point, I just want to know whats on the wire. It would be cool to also carve out the file and dump it to disk (patches are welcome 😛 ), but for now I use other tools to do this. First of all, I use OpenFPC to do full packet capture. Mostly I have been using tcpxtract and I have also tested xtract.py. I see it as a bigger task to take on TCP reassembly and carving out the file correct, especially when I already have the pcap of the session, I can handle that offline. I also recommend xplico btw.

Default, nftracker logs to /var/log/nftracker-csv.log. The logfile looks like this:

# timestamp,[ session ],FILE_TYPE
# timestamp,proto,src_ip,src_port,dst_ip,dst_port,FILE_TYPE
1291893772,6,85.19.221.54,42696,217.147.81.2,80,exe
1292119164,6,217.69.134.176,51630,85.19.221.54,80,pdf
1292142613,6,85.19.221.54,59406,78.46.89.231,80,png
1292144009,6,85.19.221.54,34695,78.46.89.231,80,png
1292149647,6,85.19.221.54,43602,160.68.205.242,80,cws
1292414981,6,220.181.51.117,17942,85.19.221.54,80,pdf
1292427913,6,67.195.115.110,47998,85.19.221.54,80,pdf
1292435336,6,194.8.74.53,2206,85.19.221.54,80,html

I hope the tool is useful for someone, ideas/comments and such can be mailed to me.
I hope you try it out!

multicap – multi interface networkstream dump daemon

Two weeks ago, I was made aware of a new tool to do packet captures with that looks promising. The initial commit seems to be from 2010-10-27 from the looks of the git repo found here.

To test it,
git clone git://git.carnivore.it/multicap.git
cd multicap
autoreconf -i
./configure
make
sudo ./multicap -w /tmp/ -c $PWD/multicap.conf.dist

You will find your pcaps under /tmp/var/log/multicap/.

I specially like the possibilities with this tool, that I can read/interpret from the config file. You can do “multi-sniffing”, writing to different logfiles filtered on BPF, specify different interfaces, snaplength, log rotation… Take a look at the configfile to see what I mean.
This is a tool to keep an eye on!
The project is young it seems, as passing –help option to multicap does not say anything…
Looking at the code, I know why 🙂

// show_version(NULL);

Suricata 1.0.2 Debian/Ubuntu package

In stead of compiling Suricata over and over again on different hosts I have… I just made a debian package for my Ubuntu Lucid 10.04 systems.

Its a simple build, and Ill hopefully update it with time to incorporate different usage and install help etc.
Right now its just aimed at being a simple IDS using libpcap.

You can find suricata and other cool NSM stuff at my gamelinux PPA found here.

apt-get install suricata
cd /etc/suricata/ && wget http://rules.emergingthreats.net/open/suricata/emerging.rules.tar.gz
vim /etc/default/suricata
vim /etc/suricata/suricata.yaml
/etc/init.d/suricata start

Feedback and thoughts are welcome and needed 🙂 !