So for the past couple of days CrashOverron and I have been researching different proxies and how we can log them using Apache. I will be looking at how to identify and log when a proxy is in use, what the log will look like, and draw some general conclusions.
Using the power of Apache’s ‘log_config_module’ the server can look out for certain common ‘signatures’ that the proxies can leave.
If you are familiar with proxy servers already you can skip to section 2.
Proxy Server Recap
I will mainly be focusing on ‘Forward Proxies’ and because I am using Apache, I will be looking at logging when a proxy visits the server.
If you’re new to proxies, the concept is very simple. When a HTTP request is made your public IP is sent along with the header. This means that you can easily be tracked down and reported to your ISP. A proxy is a server that will take your request and forward it to its original destination, the benefit being that the host is set to the proxy server’s IP address, not yours.
Normal Conn: {Host}->{Destination} > Header “host:” = {Host}
Proxy Conn: {Host}->REQ->{Proxy}->FWD REQ->{Destination} > Header “host:” = {Proxy}
As you have now seen using a HTTP proxy can be an effective way to remain hidden to web server logs. The aim was to look at the most popular proxies and determine common ‘signatures’, thus enabling the logging of details about the proxy.
Transparent Proxy Servers
The first type of proxy is a transparent one, these are nice and easy to log and send the originating IP along with the forwarded request. Before we begin it’s good to know a little about the httpd.conf typeset.
The typeset follows a common rule of denoting with a ‘%’. For example in figure 1 you can see %h which is representative of host address, or %r being the request string. At the end you see that we have referenced header values directly with %{Header Name}. This will be the basis for using Apache to log proxy visits and or users.
A transparent proxy sends an additional HTTP header called ‘X-Forwarded-For’. This is a field that contains the value of the IP address the request is being forwarded for. Clearly this isn’t going to help at all in keeping you anonymous to the web server, however has it’s purpose in filtering networks and other uses. So now we know we can reference headers directly to be logged and that transparent proxies such as Squid (Default) can be caught, and their users logged.
LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" "%{X-Forwarded-For}i"" combined
So if we add the above to the httpd.conf file the majority of transparent proxies will be logged successfully. For even better results we can also specify an individual log for only proxies, thus meaning we don’t have to trawl through 1000’s of legitimate requests to find the proxy ones.
It is worth noting that it because of the ‘combined’ value that we can reference header fields directly.
Give httpd a quick restart, take your pick of transparent proxies and give it a whirl. There may be some minor tweaking needed, or you may want to add more rules based on more obscure proxy software. Each will likely have its own signature, however it does appear the most prominent among transparent proxies is the X-Forwarded-For.
Anonymous Proxies & The Feds
We have seen previously how ineffective a transparent proxy is when it comes to concealing your identity; this is where anonymous proxy servers come into play. An anonymous proxy does not send an additional header that discloses its users IP address thus it is classified as anonymous. It is essential to keep in mind though that being anonymous does not mean undetectable.
The majority of anonymous proxies send a different HTTP header this time called Via. This header field contains information about which software the request has been sent Via. Imagine we have a user who has configured Squid to act in an anonymous manner, there will still be a header called Via that will look something like this ‘Via: snort v.1.4’. Now we really encounter the issue of it still being detectable, this being an issue really is dependent on the reason for using a proxy.
Imagine you were using this to view content restricted in your country, chances are that the website will automatically detect that header value and just refuse your connection attempts. Let’s consider that we are performing less than legal activities this could alert the web admin and if the feds locate the proxy server, whoever the owner is will most likely have to give up the logs.
So now to catch the anonymous proxies using Apache, this will be almost exactly the same as the last log with different header values accessed.
LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" "%{X-Forwarded-For}i" ”%{Via}” " combined
As you can see we have simply added in the Via header field name, also like with the transparent proxy we could very easily have Apache handle a separate log file for these kinds of connections.
To conclude an anonymous proxy is of much more benefit than a transparent proxy for most situations, however it is still detectable and thus block-able. Also it would be wise to keep in mind that with any proxy server, regardless of type, your requests could easily be watched by the server admin, so make sure you trust the admin.
High Elite Proxies
The last breed of the forwarding proxies that we will look at is what have been dubbed ‘High Elite’ servers. Now it may sound cool, but remember you could very easily get a VPS for £5 a month and run your own (A tut in the making?). The reason it is named as such is because it is either impossible or very difficult to determine that the request came from a proxy server.
When you use a High Elite proxy no additional headers are sent to the web server, as far as the logs look and an auditor can see these r
equests just came from any normal system. With this choice we have achieved almost full anonymity and perfectly normal requests making detecting the server difficult.
As with everything though in computing, there are still some issues affecting High Elite proxies, the first and main problem is that by buying access to High Elite server online, some are on watch lists. If the server IP is marked down on watch lists as a proxy server then we no longer have the beauty of being undetectable. This issue can be beat quite easily by simply purchasing your own server, and keeping its services hidden.
The last issue is with anonymity, just like with anonymous proxies the proxy server it self could still be compromised and the logs read thus showing the primary and initial user. This again can be avoided easily by simply not keeping any logs on the server, have a clean install or a VM and don’t log anything a86;
Conclusions
So that is the three main species of forwarding proxy servers, each have their use and reasons for existence, however guessing by my audience the high elite and anonymous servers are preferable. So before I finish up and sign off, I challenge you, the reader…
• Install and configure Squid as a transparent & anonymous server
• Be able to log both of these types with Apache
• Why not take a crack at a High Elite, there is information widely available.
Using the power of Apache’s ‘log_config_module’ the server can look out for certain common ‘signatures’ that the proxies can leave.
If you are familiar with proxy servers already you can skip to section 2.
Proxy Server Recap
I will mainly be focusing on ‘Forward Proxies’ and because I am using Apache, I will be looking at logging when a proxy visits the server.
If you’re new to proxies, the concept is very simple. When a HTTP request is made your public IP is sent along with the header. This means that you can easily be tracked down and reported to your ISP. A proxy is a server that will take your request and forward it to its original destination, the benefit being that the host is set to the proxy server’s IP address, not yours.
Normal Conn: {Host}->{Destination} > Header “host:” = {Host}
Proxy Conn: {Host}->REQ->{Proxy}->FWD REQ->{Destination} > Header “host:” = {Proxy}
As you have now seen using a HTTP proxy can be an effective way to remain hidden to web server logs. The aim was to look at the most popular proxies and determine common ‘signatures’, thus enabling the logging of details about the proxy.
Transparent Proxy Servers
The first type of proxy is a transparent one, these are nice and easy to log and send the originating IP along with the forwarded request. Before we begin it’s good to know a little about the httpd.conf typeset.
The typeset follows a common rule of denoting with a ‘%’. For example in figure 1 you can see %h which is representative of host address, or %r being the request string. At the end you see that we have referenced header values directly with %{Header Name}. This will be the basis for using Apache to log proxy visits and or users.
A transparent proxy sends an additional HTTP header called ‘X-Forwarded-For’. This is a field that contains the value of the IP address the request is being forwarded for. Clearly this isn’t going to help at all in keeping you anonymous to the web server, however has it’s purpose in filtering networks and other uses. So now we know we can reference headers directly to be logged and that transparent proxies such as Squid (Default) can be caught, and their users logged.
LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" "%{X-Forwarded-For}i"" combined
So if we add the above to the httpd.conf file the majority of transparent proxies will be logged successfully. For even better results we can also specify an individual log for only proxies, thus meaning we don’t have to trawl through 1000’s of legitimate requests to find the proxy ones.
It is worth noting that it because of the ‘combined’ value that we can reference header fields directly.
Give httpd a quick restart, take your pick of transparent proxies and give it a whirl. There may be some minor tweaking needed, or you may want to add more rules based on more obscure proxy software. Each will likely have its own signature, however it does appear the most prominent among transparent proxies is the X-Forwarded-For.
Anonymous Proxies & The Feds
We have seen previously how ineffective a transparent proxy is when it comes to concealing your identity; this is where anonymous proxy servers come into play. An anonymous proxy does not send an additional header that discloses its users IP address thus it is classified as anonymous. It is essential to keep in mind though that being anonymous does not mean undetectable.
The majority of anonymous proxies send a different HTTP header this time called Via. This header field contains information about which software the request has been sent Via. Imagine we have a user who has configured Squid to act in an anonymous manner, there will still be a header called Via that will look something like this ‘Via: snort v.1.4’. Now we really encounter the issue of it still being detectable, this being an issue really is dependent on the reason for using a proxy.
Imagine you were using this to view content restricted in your country, chances are that the website will automatically detect that header value and just refuse your connection attempts. Let’s consider that we are performing less than legal activities this could alert the web admin and if the feds locate the proxy server, whoever the owner is will most likely have to give up the logs.
So now to catch the anonymous proxies using Apache, this will be almost exactly the same as the last log with different header values accessed.
LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" "%{X-Forwarded-For}i" ”%{Via}” " combined
As you can see we have simply added in the Via header field name, also like with the transparent proxy we could very easily have Apache handle a separate log file for these kinds of connections.
To conclude an anonymous proxy is of much more benefit than a transparent proxy for most situations, however it is still detectable and thus block-able. Also it would be wise to keep in mind that with any proxy server, regardless of type, your requests could easily be watched by the server admin, so make sure you trust the admin.
High Elite Proxies
The last breed of the forwarding proxies that we will look at is what have been dubbed ‘High Elite’ servers. Now it may sound cool, but remember you could very easily get a VPS for £5 a month and run your own (A tut in the making?). The reason it is named as such is because it is either impossible or very difficult to determine that the request came from a proxy server.
When you use a High Elite proxy no additional headers are sent to the web server, as far as the logs look and an auditor can see these r
equests just came from any normal system. With this choice we have achieved almost full anonymity and perfectly normal requests making detecting the server difficult.
As with everything though in computing, there are still some issues affecting High Elite proxies, the first and main problem is that by buying access to High Elite server online, some are on watch lists. If the server IP is marked down on watch lists as a proxy server then we no longer have the beauty of being undetectable. This issue can be beat quite easily by simply purchasing your own server, and keeping its services hidden.
The last issue is with anonymity, just like with anonymous proxies the proxy server it self could still be compromised and the logs read thus showing the primary and initial user. This again can be avoided easily by simply not keeping any logs on the server, have a clean install or a VM and don’t log anything a86;
Conclusions
So that is the three main species of forwarding proxy servers, each have their use and reasons for existence, however guessing by my audience the high elite and anonymous servers are preferable. So before I finish up and sign off, I challenge you, the reader…
• Install and configure Squid as a transparent & anonymous server
• Be able to log both of these types with Apache
• Why not take a crack at a High Elite, there is information widely available.
No comments:
Post a Comment