Repeated requests on our server?

Posted by pitty.platsch on Server Fault See other posts from Server Fault or by pitty.platsch
Published on 2012-10-24T23:31:53Z Indexed on 2012/10/25 11:03 UTC
Read the original article Hit count: 193

I encountered something strange in the access log of our Apache server which I cannot explain. Requests for webpages that I or my colleagues do from the office's Windows network get repeated by another IP (that we don't know) a couple of seconds later.

The user agent repeating our requests is

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.2)

Has anyone an idea?

Update: I've got some more information now.

  • The referrer of the replicate is set to the URL I requested before and it's not the exact same request as the protocol version is changed from 'HTTP/1.1' to 'HTTP/1.0'.
  • The IP is not just one, it's just one of a subnet (80.40.134.*).
  • It's just the first request to a resource that's get repeated, so it seems the "spy" is building up some kind of cache of visited places.
  • The repeater is also picky. I tried randomly URLs with different HTTP status codes and different file patterns. 301s and 200s are redone, 404s not. Image extensions seem to be ignored.

While doing my tests I discovered that this behavior seems to be common as I found other clients visiting just after the first requests:

66.249.73.184 - - [25/Oct/2012:10:51:33 +0100] "GET /foobar/ HTTP/1.1" 200 10952 "-" "Mediapartners-Google"

50.17.125.180 - - [25/Oct/2012:10:51:33 +0100] "GET /foobar/ HTTP/1.1" 200 41312 "-" "Mozilla/5.0 (compatible; proximic; +http://www.proximic.com/info/spider.php)"

I wasn't aware about this practice, so I don't see it that much as a threat anymore. I still want to find out who this is, so any further help is appreciated. I'll try later if this also happens if I query some other server where I have access to the access logs and will update here then.

© Server Fault or respective owner

Related posts about apache2

Related posts about log-files