How to test robots.txt in googlebot to find out what is being indexed

Posted by Amar Jarubula on Pro Webmasters See other posts from Pro Webmasters or by Amar Jarubula
Published on 2011-11-18T18:13:41Z Indexed on 2011/11/20 10:25 UTC
Read the original article Hit count: 262

Filed under:
|
|

This question is a continuation for this answer How to check if googlebot will index a given url?
As was told I did go to the Webmaster Tools and tested contents of my robots.txt file. However this is just giving me the info if that content is good enough or not. However for my scenario I need to test whether disallowing some patterns is being indexed or not. For example I have something like this below in my robots.txt

disallow:/pattern*     

My understanding is the URLs with word pattern should not crawled, but how do I test this pattern is enforced while indexing the website?

© Pro Webmasters or respective owner

Related posts about google

Related posts about seo