Archive for the ‘Python’ Category

Sanitising WSA export dates

2016/06/14 Comments off

As AV solutions go, Webroot’s Secure Anywhere (WSA) does a decent enough job of protecting against known and unknown threats; but I’ve always has disagreements with the administrative web interface for device management. As a work around if I’ve needed to extensively analyse the endpoints in any way I’ve typically exported the data from the interface to manipulate the data using typical toolkits (grep/Excel/etc.).

There’s still a problem with the exported data in terms of easy manipulation, namely the the chosen date format; which is frankly bizarre given it’s generated by a digital platform in the first place – Example: November 30 2015 16:25. Anyone that has spent any time sorting data sets by date will immediately see problems with this format.

Released today, simply reads the standard WSA “export to CSV” file, modifies the date format of the relevant fields and creates a new *-sanitised.csv file. The dates are more easily machine sortable, in the format YYYY-MM-DD HH:MM.

user@waitean-asus:~/Webroot# ./
Script sanitises the date format from Webroot Secure Anywhere’s “Export to CSV” output

script expects a single parameter, the filename of the original .csv file

script will create a single csv file with more sensible date format

./ exportToCSV.csv

user@waitean-asus:~/Webroot# ./ WebrootExampleExport.csv
[*] Opening file: WebrootExampleExport.csv
[*] Updating date fields….
100 records processed…
200 records processed…
300 records processed…
400 records processed…
500 records processed…
[*] Processing complete. 510 corrected and written to WebrootExampleExport-Sanitised.csv

The tool is basic enough, but if you regularly encounter WSA and haven’t already created a similar tool to work with the data, this script may (hopefully) prevent you from pulling your hair out.

–Andrew Waite

P.S. if you’re a developer, please take the time to review ISO 8601 to stop these tools be needed in the future.

Categories: Malware, Python

Starting with Kippo (SSH Honeypot)

As I started life as a Linux server admin I’m only too aware that many attackers see remote access functionality as a way into a system, and as SSH is the de facto standard for Linux access it is a prime target for attack. The stats collected by DShield give an indication to the extent of the problem.

As a result I’ve had the Kippo honeypot is something that I’ve had on my radar for a while. For a number of reasons I hadn’t found time to implement the system in a live environment, but a recent post on the Diatel blog suggested that installation may be quick and pain free.

Kippo is described by it’s author (Desaster) as:

Kippo is a medium interaction SSH honeypot designed to log brute force attacks and, most importantly, the entire shell interaction performed by the attacker.

Kippo is inspired, but not based on Kojoney.

Installation for me was painless, running a Debian system I downloaded the latest archive to disk, unpacked and installed the pyton-twisted package (I hadn’t read Mig5’s comment until after install so now need to go back and live on the bleeding edge…). I did hit a couple of problems when trying to start up the system, which is as simple as invoking ./

  1. First, I was logged in as root when I first tried to start the system (not clever I know, was testing…). Kippo encounters an error when started by a root user. As Desaster rightly states, it’s not wise to run Kippo as a root user anyway and running as a regular user resolves the issue.
  2. Second, when running as a normal user I got a ‘meaningful’ error of “Failed to load application: ‘NoneType’ object has no attribute ‘get’.” A quick piece of Google-fu lead me to this ticket, which explained Kippo was missing the file kippo.cfg, as explained copying kippo.cfg.dist to kippo.cfg correct the issue and produced a fully functional system.

There are a couple of key files that can be edited to change the feel of the system that is provided to malicious users:

  • kippo.cfg contains runtime information including log location, fake hostname etc.
  • kippo.tac contains an array ‘users’, which lists the username/password combination which the emulated SSH login will accept as ‘valid’.
  • The honeyfs/ directory goes so far as to allow you to create a ‘real’ filesystem for the malicious user to interact with, potentially copying a live server’s filesystem to the directory to help camouflage the emulated system (after sensitive data is removed/sanitised obviously….). I haven’t tried this myself yet but is definitely on my to-do list.

From initial testing I’ve got high hopes for Kippo becoming a mainstay in my honeypot toolbox; the interaction session provided to a malicious user is reasonably convincing at first glance, and I particularly like the trick to keep users logged in after they think they’ve sent an ‘exit’ command to close the session, it could get some interesting results.

For post compromise analysis Kippo also provides some an interesting utility, utils/ This allows you to replay a malicious terminal session in real-time, typos and all, to truely provide a feel for the malicious users interaction with the session. To help whet your apetite whilst I wait for someone to target my kippo installation, Kippo has a few demo’s of the playlog capabilities from compromise attempts. Get your demos here.

–Andrew Waite

Categories: Honeypot, InfoSec, Kippo, Python

I’ve been a bit lax in writing this post; around a month ago Miguel Jacq got in contact to let me know about a couple of errors he encountered when running InfoSanity’s with a small data set. Basically if your log file did not include any submissions, or was for a period shorter than 24hours the script would crash out, not the biggest problem as most will be working with larger data sets but annoying non the less.

Not only did Miguel let me know about the issues, he was also gracious enough to provide a fix, the updated script can be found here. An example of the script in action is below:

cat /opt/dionaea/var/log/dionaea.log| python

Statistics engine written by Andrew Waite –

Number of submissions: 84
Number of unique samples: 39
Number of unique source IPs: 65

First sample seen: 2010-06-08 08:25:39.569003
Last sample seen: 2010-06-21 15:24:37.105594
System Uptime: 13 days, 6:58:57.536591
Average daily submissions: 6

Most recent submissions:
2010-06-21 15:24:37.105594,, emulate://, 56b8047f0f50238b62fa386ef109174e
2010-06-21 15:18:08.347568,, tftp://, fd28c5e1c38caa35bf5e1987e6167f4c
2010-06-21 15:17:08.391267,, tftp://, bb39f29fad85db12d9cf7195da0e1bfe
2010-06-21 06:29:03.565988,, tftp://, fd28c5e1c38caa35bf5e1987e6167f4c
2010-06-20 23:34:15.967299,,, 094e2eae3644691711771699f4947536

— Andrew Waite

Amun statistics

Amun has been running away quite happily in my lab since initial install. From a statistic perspective my wor has been made really easy as Miguel Cabrerizo has previously taken one of the InfoSanity statistic scripts written for Nepenthes and Dionaea and adapted it to parse Amun’s submission.log files.

Results generated from the script in my environment are below, if you’re wanting to get an overview of submissions from another Amun sensor the script has been uploaded alongside the other InfoSanity resources and is available here.

~$ cat /opt/amun/logs/submissions.log* | ./

Statistics engine written by Andrew Waite ( modified by Miguel Cabrerizo (

Number of submissions      : 25
Number of unique samples   : 25
Number of unique source IPs: 18

Origin of the malware:
Ukraine :     1
None :     7
Poland :     2
Romania :     1
United States :     8
Russian Federation :     2
Hungary :     1
Norway :     1
Bulgaria :     2

Vulnerabilities exploited:
MS08067 :    13
DCOM :    12

Most recent submissions:
2010-05-31, 11:37:22,, 63.exe, acf5c09d547417fe53c163ec09199cab, MS08067
2010-05-30, 19:23:09,, 63.exe, 89b578839f1c39f79d48e5f9e70b5e2f, MS08067
2010-05-28, 10:27:03,, 63.exe, f7c4f677218070ab52d422b3c018a4ba, MS08067
2010-05-27, 16:23:14,, ssms.exe, 1f8a826b2ae94daa78f6542ad4ef173b, DCOM
2010-05-24, 19:46:35,, 63.exe, 53979f1820886f089a75689ed15ecf6e, MS08067

A comment on a recent post asked for a comparison between different honeypots, while this is far from conclusive and only focuses on a single aspect of the technologies one of InfoSanity’s Nepenthes sensors ‘saw’ more attacks in the last 24hrs than my Amun installation did in the almost three weeks shown above. As both are running within the same, small, IP allocation I think I’m safe in assuming that one IP isn’t actually receiving a disproportionate level of interest from the badguys and bots that are out there.

— Andrew Waite

Determining connection source from honeyd.log – cymruwhois version

2010/05/03 1 comment

InfoSanity’s script has been useful for analysing the initial findings from a HoneyD installation, but one of weaknesses identified in the geolocation database used by the script was that a large proportion of the source IP addresses connecting to the honeypot environment weren’t none within the database. Markus pointed me in the direction of the cymruwhois (discussed previously)python module as an alternative. I’ve re-written the initial script, below:

from cymruwhois import Client
import sys

logfile = open('/var/log/honeypot/honeyd.log', 'r')
source = []
for line in logfile:
    source.append(line.split(' ')[3])

src_country = []
src_count = []


for res in results:
    country = results[res].cc
        pos = src_country.index( country )
        src_count[pos] += 1
        src_country.append( country )
        src_count.append( 1 )

for i in range( 0, ( len( src_country ) - 1 ) ):
    sys.stdout.write( "%s:\t%i\n" %( src_country[i], src_count[i] ) )

So far this has resulted in far fewer unknown source locations, 249 using geoip compared to 3 using cymruwhois. The downside unfortunately is performance, the cymruwhois communicates with a remote host to gather information compared with the geolocation database that is already stored locally on the machine. Both perform some local caching of results/data however so I would expect the performane difference to decrease as larger datasets are analysed.

Using the newer script, based on the same 24hr data set, the top ten host countries communicating with InfoSanity’s honeyd environment are:

RU:     397
US:     234
TW:     179
BR:     158
CN:     123
RO:     107
DE:     101
IT:     96
JP:     91
AR:     86

— Andrew Waite

Team Cymru Whois

2010/05/03 1 comment

Since posting my Python whois class it’s lead to a (relatively) high volume of search hits pointing people to it. So I’d like to apologise for inflicting my code on other people. After a recent post with the script I was pointed in the direction Team Cymru’s whois service and accompanying python script. If you’ve not come across the stuff released by Team-Cymru I would strongly suggest that you take a look. I always manage to find some interesting new info, three overall sections Monitoring, Services and Reading Room.

Making my life easier, Justin Azoff has released a Python module hosted on github for the service. Using the client is incredible simple as the sample code included in the package shows:

>>> import socket
>>> ip = socket.gethostbyname("")
>>> from cymruwhois import Client
>>> c=Client()
>>> r=c.lookup(ip)
>>> print r.asn
>>> print r.owner
GOOGLE - Google Inc.

Overall Justin’s client works faster than my own attempt, especially has it has functions specifically designed for bulk lookups. If you’re working with IP, whois or geolocation data I’d suggest giving the cymruwhois utility a look. Thanks to Justin and the Team Cymru people for releasing tools and info that make my work easier.

–Andrew Waite

Categories: InfoSec, Python, Tool-Kit

New dionaea statistics script

Following on from my work with gathering statistics from the Honeypot systems that I run I have released a limited alpha of a new script/tool that I am working on. The tool provides access to common result sets from the sqlite database, without the requirement for remembering the database architecture  and entering lengthy SQL statements by hand.

Disclaimer first: the tool doesn’t do anything outrageously new, and most of the SQL queries have been borrowed from Markus’ post on SQL logging with Dionaea when the feature was first introduced. However I have found the script makes my analysis of the honeypot logs simpler and quicker, and I’ve a positive reaction from a limited few that have had a copy of the script before this post. Hopefully it will be of use others.

Usage is relatively simple, shown below:

Dionaea database query collection
Author: Andrew Waite –

Inspiration from article:

/path/to/python –query #
Where # is:
1:      Port Attack Frequency
2:      Attacks over a day
3:      Popular Malware Downloads
4:      Busy Attackers
5:      Popular Download Locations
6:      Connections in last 24 hours

The script can be found here. There is still a good level of work to be undertaken to tidy up the output, potentially allowing for output in different formats, and I also want to add additional and more complex queries as time progresses. If you have any success,  failure, comments or suggests please let me know.

— Andrew Waite

Categories: Dionaea, Honeypot, InfoSec, Python