Friday, 23 July 2010

Python

As regular readers will know here in the Sausage Factory our primary forensics tool is Encase. From time to time however we need to try out other tools to validate our results. Recently I wanted to utilise two python scripts widely discussed elsewhere and as a result had to figure out the mechanics of getting these scripts to run on a forensic workstation running Windows 7. I thought I'd share the process with you. Now some of you are highly geeky programmer types who write and run scripts for breakfast - if thats you turn away now. This blog post is in no way definitive and is intended for python newbies wishing to run python scripts in their forensicating but who until now didn't know how.

First off we need to install and configure Python

  • Download Python - I downloaded Python 2.7 Window X86-64 installer for my Windows 7 64 bit box
  • Run the installer
  • Right click on the Computer icon, select properties, select Advanced system settings and click on the Environment Variables button.
  • In the System Variables pane you will have a variable entitled Path, select it and click on edit
  • Add to the entries already there ;C:\Python27 (assuming you installed Python 2.7 to the default location)

The two scripts I wanted to run were David Kovar's analyzeMFT and the $USNJRNL parser written by Seth Nazzaro. They are designed to parse MFTs and USN Change Journals respectively which can be copied out of an image or made available via VFS or PDE. More about analyzeMFT can be found at the author's blog. Detailing how I ran these scripts will give a clear indication of how to run these, and many other python scripts, and utilise their output.

analyzeMFT
Download script by visiting http://www.integriography.com/ and right clicking on the Downloaded Here link in the Downloads section (for the source code) and saving the download as a text file. Once downloaded change the file extension to .py.

Save it somewhere and then run IDLE (installed with Python) and open the analyzeMFT.py script. Locate the words noGUI = False and edit to read noGUI = True and save.

To run

  • open command prompt
  • at prompt type Python C:\Path_to_the_script\analyzeMFT.py -f U:\Path_to_your_extracted_or_mounted_MFT\$MFT -o $MFT_parsed
  • The above command runs the script against your extracted or mounted $MFT and outputs the results to a file $MFT_parsed
  • Open $MFT_parsed using the text import wizard in Excel selecting the text format for each column.

Thanks to David Kovar for making this script available.

$USNJRNL•$J Parser
This script can be downloaded at http://code.google.com/p/parser-usnjrnl/.

To run

  • open command prompt
  • at prompt type Python C:\Path_to_the_script\UsnJrnl.py -f U:\Path_to_your_extracted_or_mounted_USNJRNL•$J\USNJRNL•$ -o Output_file -c
  • The above command runs the script against your extracted or mounted $USNJRNL•$J and outputs the results to Output_file.csv

Notes
Typing at the command prompt Python path_to_script.py wil give some help about a scripts options. For example Python UsnJrnl.py results in the output

Usage: UsnJrnl.py [options]
Options:
-h, --help show this help message and exit
-f INFILENAME, --infile=INFILENAME
input file name
-o OUTFILENAME, --outfile=OUTFILENAME
output file name (no extension)
-c, --csv create Comma-Separated Values Output File
-t, --tsv create Tab-Separated Values Output File
-s, --std write to stdout

I have installed Python 2.7. There are other (and later) versions available including some that are not completely open source. It is also possible to install Python modules to provide a GUI. I have not installed these - takes the fun out of running scripts!


6 comments:

Markus said...

Hi,

i'm very interested in parsing the UsnJrnl file aswell but unfortunately I receive the following error message after following your steps, when I arrive at the "script execute" path:

('Unexpected error:', )
Traceback (most recent call last):
File "C:\TEMP\UsnJrnl-24NOV09.py", line 279, in
main(sys.argv[1:])
File "C:\TEMP\UsnJrnl-24NOV09.py", line 79, in main
parsefile_small(it, options)
File "C:\TEMP\UsnJrnl-24NOV09.py", line 189, in parsefile_small
sys.stderr.write ("\nLength of data is %s \n", len(data))
TypeError: function takes exactly 1 argument (2 given)




I'm running Python 2.7 on Windows 7 (latest Patches etc) and I'm trying to analyze an Windows XP Service Pack 3 UsnJrnl File.


Am I doing something wrong? I hope you can help me :)


sincerely yours

Markus

DC1743 said...

Hi Markus,

I am not a python expert but I will try and help. What command did you enter at the command prompt?

Regards Richard

Markus said...

Hi,


before I start with further information I would like to thank you in advance for your help!

I'm actually writing my master thesis and actually I'm trying a few things out around the UsnJrnl file.

Since I'm not a python expert too, I've decided to post a reply here and on the script developers page first, before I spend more hours in understanding and fixing the code. (maybe I'm just doing a few things wrong or maybe I forgot something? I don't know actually)


---

First of all I installed Python 2.7 from the official Python project page and after this I've downloaded the .py script which was mentioned in your blog article.

I've uploaded a copy for verification purposes here:

http://dl.dropbox.com/u/494981/for/UsnJrnl-24NOV09.py



This script was executed on the following file:
http://dl.dropbox.com/u/494981/for/UsnJrnl-J

Its an extracted UsnJrnl file out of a Windows XP (SP3) installation out of an VirtualBox environment. The Source image was snapped by the use of the dd tool under helix, and the file was extracted later with the help of the TSK & Autopsy tools.


The script execution command was simply :
UsnJrnl-24NOV09.py -f UsnJrnl-J -o UsnJrnlOutput -c

unfortunately the posted error message occurs.


I appreciate any kind of help :-)

sincerely

Markus

DC1743 said...

Markus,

First off, the enscript I referred to in the http://forensicsfromthesausagefactory.blogspot.com/2010/08/usn-change-journal.html post wont parse your change journal either.

I suspect the file is corrupted in some way and is not in the form the scripts expect. This is possibly due to tools used to extract it.

With a hex editor delete the first 58 bytes of your file, then run the script using the output option -o output -s

This parses most of the file out to the console, you could pipe the output into a file.

R

Markus said...

Hi,

thank you very much for your answer and help.


I've solved the parsing issue almost. The error was caused by a bad input file. It seems that the USNJRNL File was damaged in its structure during the extraction.

After re-extracting the USNJRNL file with an different tool, I was able to parse the file, but the script is still crashing during the parsing process.

At least I have a csv File now and with it a point to continue and gain new motivation :)


sincerely

Markus

Gogo said...

Markus, I had a similar issue while parsing my USNJRNL file. What tool di you use to successfully extract it?

I tried with FTK Imager but it won't see the file, then I tried with Pro Discover Basic and it succeeded but then the script did not like the input (which is clearly corrupted in the first bytes).

Thanks
Paolo