Does the Instant Preview in Google webmaster tools takes Robot.txt in account?

Posted by rockyraw on Pro Webmasters See other posts from Pro Webmasters or by rockyraw
Published on 2014-04-03T17:52:19Z Indexed on 2014/06/03 15:58 UTC
Read the original article Hit count: 189

Is that the way to go If I want to visually see what the googlebot see? I'm trying to check a folder which I have just blocked in my robots.txt.

  • If I fetch the folder as google bot, It fetches ok, so that doesn't tell me nothing about whether the block is working
  • I know there's a tool to check for blocking, but it is dependent on the input of the robots.txt

Therefore I've tried the Instant preview, and I don't get a preview for what the bot sees ("pre-render), so I think that means that it's because the robots.txt blocks it; however - I don't see the bot tried beforehand to access my updated robots.txt, so I'm not sure how does it know that this folder is blocked? (it does preview another new folder, that is not blocked)

© Pro Webmasters or respective owner

Related posts about seo

Related posts about google