AFAR - Automatic File Analyze and Reporting is a script that takes a list of files as input and runs them through a couple of tests and generates a report that can be used as a starting point for further malware analyzes. This tool main usefulness is to automate malware analyzes. Tests are done in Virtual machines. At the moment it is only written to work with VMware Fusion on Mac OS X. It uses three virtual machines for different tests. My setup is the following:
- Debian with Cuckoo Sandbox. I use Cuckoo Sandbox 2.0-dev which uses Volatility and Suricata. The VM is configured with my cuckoo-tools script.
- REMnux with SIFT installed. The VM is installed with my remnux-tools script.
- Windows 10 with LECmd.
Tools used by AFAR in REMnux includes but are not limited to:
- Density Scout
- PDF tools by Didier Stevens includes pdfid.py, pdf-parser.py
I get lists with CSV files with malware stopped by anti virus and wanted to analyze the files. To do this I wrote a script to take this information and call a script written by a colleague. The next problem was that I know had a bunch of files that was flagged as malware. How to get more information about their characteristics since AV vendors usually are bad at writing technical descriptions of malware? I then submitted the files to Cuckoo and had to look through a page for each file submitted. Heres where AFAR comes in. It submits the suspicious files to Cuckoo and also does some analyzes in REMnux and Windows if available. Then you can take a cup or two of coffee and wait for the result which gives a one page overview of the files with links to more information.
By default some of the tools will submit the files being analyzed to Virustotal and potentially other services. If you don’t want this to happen please disconnect your computer from the internet during script execution. It’s on my todo-list to control this with a command line option.
Processing of input files
If you use the AFAR with three VMs as listed above the steps are as follow:
- Start Cuckoo and REMnux (if you know you will have a .lnk file and add the -W option Windows is started to).
- Prepare files. This step unpacks zip files and creates a file structure under WORKDIR.
- Copy files to Cuckoo and REMnux and submit all files to Cuckoo.
- Depending of file type do different types of tests of the file in REMnux and Windows. Collect logs from each test.
- Wait for Cuckoo to finish. Remember to drink coffee during this step.
- Retrieve reports from Cuckoo and unpack them too the WORKDIR structure.
- Generate summary report.
Installation and configuration
Checkout the code from Github and copy the default configuration to config.cfg(or use the -c switch if you have more then one configuration).
git clone https://github.com/reuteras/afar.git cp config.cfg-default config.cfg $EDITOR config.cfg
I should update this section with more information about configuration that are needed in the different VMs. Some noteworthy changes that I remember are:
- Windows: Activate administrator account and set a password. Also change configuration to make it possible to run Powershell.
- Debian and REMnux. Make sure that the user used to login can do sudo without entering a password (NOPASSWD option in /etc/sudoers).
There are probably more changes needed and I will add them when I remember what I’ve changed that have an effect on AFAR.
Since there are many changes in the code at the moment the best information about the program is to look at the built in help. Or use the force and read the code. At the moment the help output looks like this:
./afar.sh [-h] [-v] [-o] [-w] [-c config] [-C] [-R] [-W] [-Z] file1 ... fileN -c config Load config file. Default is config.cfg. -h Show help -o Open summary when done -p Paus before stopping and deleting VM -r Run report generation again and exit -v Verbose -w Start Windows directly -Z Remove WORKDIR without questions -C Don't use Cuckoo -R Don't use REMnux -W Don't use Windows
A typical invocation for me is:
./afar.sh -o -Z test/*
In your specified WORKDIR you will get a folder per submitted file and extra folders if there are any zip-files since they are unpacked (tries with empty password and the passwords “virus” and “infected”. The folders are numbered from 1 counting upwards. There is also a file named index.html with the summary report. The folder cuckoo contains a text file with the last status from the Cuckoo Sandbox API. In each directory there is couple of files and directories. For a file that isn’t a duplicate there is usually a minimum of the following:
- only created if there is a script for that file type
- 2_file/ - directory that contains the original file
- 3_cuckoo/ - Directory with the total Cuckoo report and other outputs for the file
- 4_cuckoo_report.html - Cuckoo report web page.
- 9_sha256.txt - File with the files sha256.
If the file is a duplicate the contents will be:
- [email protected] - Link to the other files Cuckoo report
- [email protected] - The number (10 in this case) is a link do the duplicate that has been analyzed
- 6_duplicate - File indicates that this is a duplicate
Depending on file type there will be other report files or directories present. I’m trying to follow a naming scheme where a file named pdf-parser-f-w.txt indicates that the command pdf-parser was executed with the flags -f -w.
Probably. If you dare to use this script and find bugs please file a issue report at Github.
- Look into the possibility to control what data are sent to the internet with a command line option to AFAR.
- Full internet access as is the current standard. If you like to exit via VPN you can use that for all of your traffic.
- Only send hashes to the internet (Virustotal). No access to the internet for Cuckoo.
- No net
- Add support for Cuckoo’s url scanning.
- Code cleanup and make sure the code is secure.
- Write a script that monitors the status of analyzes running in Cuckoo to make it possible to estimate when the report is finished.
- Look at the test executed on files. At the moment it’s the result of a quick and dirty look through the REMnux documentation. Write scripts for more file types (jar, ps1 and more). Find test files for each type of file. The following sources are some that can be useful.
- Better reporting. Besides the file and signature parts from the Cuckoo Sandbox reports for each file the summary page only adds information about matches from Yara-Rules.
- More tools to look at
- Move todo to issues in Github.