Blocking Just the Parent Domain via robots.txt

Posted by Bryan Hadaway on Pro Webmasters See other posts from Pro Webmasters or by Bryan Hadaway
Published on 2012-09-06T12:29:40Z Indexed on 2012/09/06 15:50 UTC
Read the original article Hit count: 408

Filed under:

Let's say you have a parent domain:

parent.com

and children subdomains under that parent domain:

child1.com

child2.com

child3.com

Is there a way to use just the following within parent.com:

User-agent: *
Disallow: /

Considering each child has their own robots.txt stating:

User-agent: *
Allow: /

Or is the parent robots.txt still going to have to make an exception for every single subdomain:

User-agent: *
Disallow: /
Allow: /child1/
Allow: /child2/
Allow: /child3/

Obviously this is important and tricky territory SEO wise so I'm looking to learn the definitive and safe, best practice method here to sharpen my skills.

Thanks, Bryan

© Pro Webmasters or respective owner

Related posts about seo