Proxy Authentication - investigating requests and responses

When describing next-generation threats protection we need to focus on adding additional layer of protection to our defense-in-depth architecture. Later on, I will try to present what I understand by saying ‘next-generation threat’, and why the ‘old-school’ (pattern – matching techniques, AV, IPS, …) approach is not sufficient for this moment. In short, lets monitor outgoing traffic from our company, and be suspicious about every flow – … replying it.  There are several interesting products out there, that provide methods of checking outbound connections, filtering them out, checking malicious activity and delivering also sandbox technology ‘on the wire’.

For this moment, I have prepared two flow-diagrams presenting how internal host (client) is authenticating itself on proxy to ‘get’ something from the internet (server). I took into account, two methods of authentication, the first on relying on NTLM challenge-response, and the second, also NTLM but with support from domain controller. Why I am trying to present it? It seems that the understating  how the traffic is going through the proxy to the outside world is crucial in network forensics investigations. Imagine the situation, that you  have spotted some GET/POST queries to CnC server, and now you need to verify how your system responded to them, basing on proxy logs, captured traffic (outgoing), and hopefully flows (firewall, router outside!). This is possibly helpful, generally to understand how does it work, but also to check if our proxy is susceptible to some kind of obfuscation, defragmentation, and – most important – up to date.


Flow Diagram 1 - NTLM challenge-response authentication

First picture presents NTLM challenge-response authentication method. I have also added step-numbers, to make it easier to read and compare with the description below. These are steps during proxy-authentication, divided into packests:

Firstly, client sends a proxy request for www.intruderit.blogspot.com, using following message (Packet 1):

---------------------------------------------------------- Packet 1 -------------------------------------------------------------------
GET www.intruderit.blogspot.com:80 HTTP/1.0
User-Agent:Mozilla/4.0 (compatible ... )
proxy-Connection: Keep-Alive
Pragma: no-cache

In the response, Proxy sends a “Access Required” message with error code 407 – asking for authentication, then tears down the connection. Access to proxy is denied.

---------------------------------------------------------- Packet 2 -------------------------------------------------------------------
HTTP/1.1 407 Proxy Authentication Required
proxy-Authenticate: NTLM
Connection: close
proxy-Connection: close
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
The client, re-establish the TCP connection – sends request again with NTLM authentication:

---------------------------------------------------------- Packet 3 -------------------------------------------------------------------
GET www.intruderit.blogspot.com:80 HTTP/1.0
User-Agent:Mozilla/4.0 (compatible ...)
proxy-Connection: Keep-Alive
Pragma: no-cache
proxy-Authorization: NTLM + BASE64_Login + HASH

The proxy responds with the following message indicating the denied access and an authentication challenge for the client:

---------------------------------------------------------- Packet 4 -------------------------------------------------------------------
HTTP/1.1 407 ProxyAuthentication Required
proxy-Authenticate: NTLM (…)
Connection: Keep-Alive
proxy-Connection: Keep-Alive
Pragma: no-cache
Content-Type: text/html

---------------------------------------------------------- Packet 5 -------------------------------------------------------------------
GET www.intruderit.blogspot.com:80 HTTP/1.0
User-Agent:Mozilla/4.0 (compatible ...)
proxy-Connection: Keep-Alive
Pragma: no-cache
proxy-Authorization: NTLM + (…) + RESPONSE

---------------------------------------------------------- Packet 6 -------------------------------------------------------------------
HTTP/1.1 200 Connection established


The application data can be exchanged after the NTLM authentication is finished. Without doubts, the way investigations should be run, depends on the specific and current set in client’s infrastructure. There are plenty of different configurations, and proper approach should be chosen during examination. Basing on documentation (here), we can read:

...returns an HTTP reply with status 407 (Proxy Authentication Required). The user agent (browser) receives the 407 reply and then attempts to locate the users credentials. Sometimes this means a background lookup, sometimes a popup prompt for the user to enter a name and password. The name and password are encoded, and sent in the Authorization header for subsequent requests to the proxy.
 

From security point of view, there are several possibilities when tracking outgoing malicious GETs. It the user-agent has already authenticated itself correctly, and malicious URL is technically valid, proxy can block malicious request (if proxy has proper policy set up). In the HTTP proxying, not all requests require authentication. Standard authentication are TCP based, so proxy would only need one authentication on the begging of new session. Subsequent HTTP request would not require authentication. On the other hand if user login/credentials are used ‘correctly’ by malware, new session can be set up again anyway. If the GET request is obfuscated, fragmented, or somehow malformed, proxy will immediately block it. Possibly, we won’t see any responses (allow/deny) from proxy, when the firewall block the communication.

Flow Diagram 2 - authentication with DC

Second picture, present proxy authentication, when there is a domain controller available. For more information for this particular sample please follow this URL.  There is a great article, presenting also samples and other important facts. By the way, quick reminder, check URI/URL difference here.

To be definatelly sure, if the traffic is blocked or not, firstly proxy logs should be checked. For verification flows can be delivered from FW/Router behind the proxy, for further confirmation, and possibly revealing some potential holes. Contacting proxy guys, or networkers is always a good idea.