Autopsy 3.0.5 - Forensics Browser for Digital Investigations

Digital investigation is a nested process with backtracking, where – in first stage - we find and test hypotheses that could possibly answer questions about security incident or basically digital scene. This  can be easily understood by any data-analyst regardless of security knowledge, as the general process is not  unique and can be successfully applied for any investigations or data extraction. Lastly I have seen an introduction presentation showing how the big-data analysis is cooperating with standard security model. On one hand, we have security analyst who performs daily-routine incident response process, and tuning. This typically is based on hot and warm data – all delivered by standard SIEM/NBAD technology. The same guy or team is cooperating with data-analyst (who does not have any security knowledge) and operate on level of data-warehouse and cold data. On this stage, new abnormal patterns can be spotted, especially with adequate skills in data-mining and things like that. As you can possibly see right now, this model looks pretty straight forward and can be deployed in big environments, combining different fields and personnel. I will cover this subject later. There are basically two types of investigations (on the level of system  status), live or post-mortem analysis. The so-called ‘live forensics’ is a little bit tricky, and is performed on non-trusted environment, on suspect system, and should be performed with sound methodology, leaving as little fingerprints as possible. On the other hand we have ‘dead analysis’ which should and is run on trusted, prepared laboratory, under control. There are of course, many sub-processes, stages and guidelines, but this was or will be covered in separate posts. I am very happy to call this field of knowledge (specialization) digital-archeology.

In this article, I am presenting and discussing features delivered with Autopsy Forensics Browser – a front end for the TSK (The Sleuth Kit). This tool is essential for forensics investigations for both linux and windows images (NTFS, FAT, HFS+, Ext3, UFS). Please visit project homepage: http://www.sleuthkit.org/. Below definitions for both, dead and live analysis is taken from Autopsy homepage.

Autopsy - main screen

A dead analysis occurs when a dedicated analysis system is used to examine the data from a suspect system. When this occurs, Autopsy and The Sleuth Kit are run in a trusted environment, typically in a lab. Autopsy and TSK provides support for raw, Expert Witness, and AFF file formats.

A live analysis occurs when the suspect system is being analyzed while it is running. In this case, Autopsy and The Sleuth Kit are run from a CD in an untrusted environment. This is frequently used during incident response while the incident is being confirmed. Following confirmation, the system is acquired and a dead analysis performed.

Creating, setting new case and adding evidence images

First of all, we start our investigation by creating new case. There we have Case Wizard, asking us for standard investigator/case names. We need to also fill the Base Case directory path (all stored in one place) and choose which images should be added. We can acquire image/disk with Image Wizard, also having multiple images in one case. Old-school DD and E01 formats are supported. As it is said, during adding, Autopsy will create internal database for image and findings. Additional options are available, such as ‘search in unallocated’ and features that can speed-up process when disabled.

Ingest features

After the evidence data is added, search-modules will start working in the background (delivering results over time). As it can be read on the homepage, the Ingest modules analyze files in a prioritized order, so that files in a user’s directory are analyzed before files in other folders. Recent Activity module, will extract user activity in the operating system and web history. Hash lookup, with combination of NSRL give us a great tool to get rid of false-positive and knows files. Also standard module will calculate hash for every file. Exif (exchangeable image file format) parser and archive extractor are worth to be mentioned. Additionally there is a keyword search feature – for both manual and automatic search.

Additional tips:
 - When clicking on file on directory listing, we can switch to the location of the file in the file system
 - Quick searches can be maintained with keyword search in the right corner of Autopsy

Autopsy workflow

To analyze system artifacts and Autopsy findings we need to follow bottom-down analyze philosophy. It means that firstly we look at tree view, showing system files, results .etc and then going deeper into the file system details with directory listing, and low-level data. It is pretty standard method of analysis, the most intuitive and simple, also commercial solutions use it.

Autopsy - workflow schema

In the most general pane/view there are three trees. The first is called “images’ and here any evidence files/disks/images can be added and review. What is more for each image, the full structure of volumes is presented with detailed information and general file system listings. As mentioned before, for each case more than one image can be added. Then we have Views panels, showing extracted information grouped by different category such as file types, documents and recent files. Another, and last pane shows results which basically  present all parsed information from files system. There are web history content (here we have bookmarks, cookies, downloads and history), keywords and hash-sets hits, installed programs and attached hardware. At the end of the flow we get the most important and detailed information from ‘low-level view’. Dependably on the file tagged, there are result/hex view, image/media view, string and text view giving comprehensive data  parsed from files. What is more you can manage you view, setting them in another ‘look’ and move forward/backward during your analysis, switching between opened windows and panes.

Autopsy Analysis Features
There are many interesting and advanced features delivered with Autopsy, and still the project is being developed.  Below describing functions of each.

Timeline: generally it shows how many changes, events occur over time for each activity on the files system. There is a histogram representation, with zoom in/out capability, and each file can be checked with more detailed information. Really awesome, straight forward feature. This cannot be compared to any commercial stuff out there! Really good in terms of filtering out ‘noise’ in file-system, and checking only relevant files, if time frame is known. MAC times included, searches being run in unallocated space. Timeline - great feature Web/Email content: Autopsy supports Firefox, Chrome, IE and Safari browsers. It looks for different types of information such bookmarks, cookies or history of downloads. Registry: there is some work done to help with registry identification (accessed documents and USB devices), but still no parser for ntuser.dat file. The same can be added about lnk files, which only identify recently accessed docs. For deeper analysis, additional tools and parsers need to be used. Keyword search and indexing: results from this feature are presented in ‘Keyword hits’. As mentioned on homepage there it powerful text indexing engine used, as well as mechanism of text extraction (furthermore all filres with text files are indexed). By default, Autopsy searches for regular expression such as; email addresses, phone numbers, IP addresses and URL. Additional keyword list can be added (for default automatic searches), also ad-hoc search is available. Other valuable: EXIF analysis, media( can review video without additional extensions), images, thumbnail viewer, file system analysis, unicode strings extraction from unallocated space and unknown file types. Reporting: in terms on evidence handling there is a mechanism for tagging/bookmarking all found files and content. Report can be created in different formats (with notes), and also extracting files is possible. Different formats available. Known ‘Bad Hashes Files” – after enabling hash-lookup, supplying Autopsy with NSRL list and user set of known good/bad files, tool will provide and identify files as good or bad. Nice feature in terms of looking of known threat (unknown not, because there will be to many false-positives). Also lookups based on hashes are a little bit old-school right now, especially when there is only one hash for one file. Of course, when determining if the file has a proper extension, then we do not talk about hashes, but signatures. Media – in terms of  video/picture review of suspect image, the ‘images’ or ‘thumbnail’ option would be perfect. Author says, that there is great support for drivers, also making the replying and  loading much faster. Sound included. File Type: All known types of files are sorted out, and additionally added to specific prepared groups (for greater visibility). What is good, is the fact that autopsy will check if the signature is the same as the declared file extension. When adding the possibility of unknown hashes, it presents great feature of tracking suspected files!  Deleted information: Autopsy will  try to recover all deleted files, tag them, and appropriately list in the particular view. This is always a great idea, to present where the file was, and when was deleted. Another essential feature. Content: Every content – irrespective of type, extension – will be check in terms of strings, ASCII .etc, and presented in different views: raw, hex, text, or images.  This analysis is based on meta-data information, brought by any file. File search by hash and attributes: Autopsy gives something like a script for searching files based on meta-data. Documents can be found based on md5, hashes, names, size, MAC times and 'known/unknown' status. Documentation: Sufficient help and infromation is available in help menu in Autopsy itself, also wiki,  data for developers is also here and here.

Autopsy - Media View

To summarize my simple and short description, I would like to share my humble opinion about Autopsy. I have some experience in commercial forensics tools, and I find Autopsy really sufficient, with great features (many not available in commercial), and thoughtful solution. I like its  simplicity,  great dynamic development (Carrier, Venema, Farmer), many ideal ideas, support for investigators and efficiency. There are many essential features in this forensics browser, which really help in extraction evidence and tracking potential findings or hypothesis. When performing investigations I really feel that this particular tool was designed by professional forensics guy, with great experience and academic approach at the same time. There are probably tons of things that I have forgotten to write about, but I am sure that you will find anything you are looking for in Autopsy during digital case. Personally, I will continue testing, and using Autopsy for my personal investigations. What need to be mentioned, that for full and really comprehensive investigation one needs additional tools(parsers!),knowledge, and processes.  Really happy with this solution, and looking forward for new versions and outstanding ideas!

Proxy Authentication - investigating requests and responses

When describing next-generation threats protection we need to focus on adding additional layer of protection to our defense-in-depth architecture. Later on, I will try to present what I understand by saying ‘next-generation threat’, and why the ‘old-school’ (pattern – matching techniques, AV, IPS, …) approach is not sufficient for this moment. In short, lets monitor outgoing traffic from our company, and be suspicious about every flow – … replying it.  There are several interesting products out there, that provide methods of checking outbound connections, filtering them out, checking malicious activity and delivering also sandbox technology ‘on the wire’.

For this moment, I have prepared two flow-diagrams presenting how internal host (client) is authenticating itself on proxy to ‘get’ something from the internet (server). I took into account, two methods of authentication, the first on relying on NTLM challenge-response, and the second, also NTLM but with support from domain controller. Why I am trying to present it? It seems that the understating  how the traffic is going through the proxy to the outside world is crucial in network forensics investigations. Imagine the situation, that you  have spotted some GET/POST queries to CnC server, and now you need to verify how your system responded to them, basing on proxy logs, captured traffic (outgoing), and hopefully flows (firewall, router outside!). This is possibly helpful, generally to understand how does it work, but also to check if our proxy is susceptible to some kind of obfuscation, defragmentation, and – most important – up to date.


Flow Diagram 1 - NTLM challenge-response authentication

First picture presents NTLM challenge-response authentication method. I have also added step-numbers, to make it easier to read and compare with the description below. These are steps during proxy-authentication, divided into packests:

Firstly, client sends a proxy request for www.intruderit.blogspot.com, using following message (Packet 1):

---------------------------------------------------------- Packet 1 -------------------------------------------------------------------
GET www.intruderit.blogspot.com:80 HTTP/1.0
User-Agent:Mozilla/4.0 (compatible ... )
proxy-Connection: Keep-Alive
Pragma: no-cache

In the response, Proxy sends a “Access Required” message with error code 407 – asking for authentication, then tears down the connection. Access to proxy is denied.

---------------------------------------------------------- Packet 2 -------------------------------------------------------------------
HTTP/1.1 407 Proxy Authentication Required
proxy-Authenticate: NTLM
Connection: close
proxy-Connection: close
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
The client, re-establish the TCP connection – sends request again with NTLM authentication:

---------------------------------------------------------- Packet 3 -------------------------------------------------------------------
GET www.intruderit.blogspot.com:80 HTTP/1.0
User-Agent:Mozilla/4.0 (compatible ...)
proxy-Connection: Keep-Alive
Pragma: no-cache
proxy-Authorization: NTLM + BASE64_Login + HASH

The proxy responds with the following message indicating the denied access and an authentication challenge for the client:

---------------------------------------------------------- Packet 4 -------------------------------------------------------------------
HTTP/1.1 407 ProxyAuthentication Required
proxy-Authenticate: NTLM (…)
Connection: Keep-Alive
proxy-Connection: Keep-Alive
Pragma: no-cache
Content-Type: text/html

---------------------------------------------------------- Packet 5 -------------------------------------------------------------------
GET www.intruderit.blogspot.com:80 HTTP/1.0
User-Agent:Mozilla/4.0 (compatible ...)
proxy-Connection: Keep-Alive
Pragma: no-cache
proxy-Authorization: NTLM + (…) + RESPONSE

---------------------------------------------------------- Packet 6 -------------------------------------------------------------------
HTTP/1.1 200 Connection established


The application data can be exchanged after the NTLM authentication is finished. Without doubts, the way investigations should be run, depends on the specific and current set in client’s infrastructure. There are plenty of different configurations, and proper approach should be chosen during examination. Basing on documentation (here), we can read:

...returns an HTTP reply with status 407 (Proxy Authentication Required). The user agent (browser) receives the 407 reply and then attempts to locate the users credentials. Sometimes this means a background lookup, sometimes a popup prompt for the user to enter a name and password. The name and password are encoded, and sent in the Authorization header for subsequent requests to the proxy.
 

From security point of view, there are several possibilities when tracking outgoing malicious GETs. It the user-agent has already authenticated itself correctly, and malicious URL is technically valid, proxy can block malicious request (if proxy has proper policy set up). In the HTTP proxying, not all requests require authentication. Standard authentication are TCP based, so proxy would only need one authentication on the begging of new session. Subsequent HTTP request would not require authentication. On the other hand if user login/credentials are used ‘correctly’ by malware, new session can be set up again anyway. If the GET request is obfuscated, fragmented, or somehow malformed, proxy will immediately block it. Possibly, we won’t see any responses (allow/deny) from proxy, when the firewall block the communication.

Flow Diagram 2 - authentication with DC

Second picture, present proxy authentication, when there is a domain controller available. For more information for this particular sample please follow this URL.  There is a great article, presenting also samples and other important facts. By the way, quick reminder, check URI/URL difference here.

To be definatelly sure, if the traffic is blocked or not, firstly proxy logs should be checked. For verification flows can be delivered from FW/Router behind the proxy, for further confirmation, and possibly revealing some potential holes. Contacting proxy guys, or networkers is always a good idea.