Robots.txt never downloaded but some blocked URLs in GWT
- by Zistoloen
There is something I don't understand in Google Webmaster Tools (GWT) for my Wordpress site.
In menu "Blocked URLs", it mention that my robots.txt has never been downloaded but there are some blocked URLs. It's kind of weird and not logical. Am i missing something?
User-agent : *
Disallow: /*?
Disallow: /wp-login.php
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content
Allow: /wp-content/uploads
Disallow: */trackback
Disallow: /*/feed
Disallow: /*/comments
Disallow: /cgi-bin
Disallow: /*.php$
Disallow: /*.inc$
Disallow: /*.gz$
Disallow: /*.cgi$
Disallow: /author/*
I'm afraid my robots.txt doesn't block several URLs I want to block.
