How to prevent majestic 12 from indexing a site

Posted by matnagel on Server Fault See other posts from Server Fault or by matnagel
Published on 2010-05-15T23:56:01Z Indexed on 2010/05/16 0:01 UTC
Read the original article Hit count: 244

Filed under:

search-engine

|

apache2

We experience a lot of traffic and server load on a web server.

All I can find out is majestic12 accessing pages all the time.

I wonder how I can prevent majestic12 from indexing the site

Do they respect any robots.txt entry and how do I write such an enty?

© Server Fault or respective owner

Related posts about search-engine

Search Engine Optimisation - Google Replaces Yahoo As T-Mobile's Default Search Engine

as seen on Ezine Articles - Search for 'Ezine Articles'
Competition has been stoked in the mobile search engine optimisation (SEO) sector with the announcement that Google is to replace Yahoo as T-Mobile's default engine. While T-Mobile USA has replaced Yahoo with Google as its default search on mobile phones such as RIM BlackBerrys, some Yahoo-based services… >>> More
Search Engine Optimization - The Importance of Page Optimization in Search Engine Optimization

as seen on Ezine Articles - Search for 'Ezine Articles'
In order for your website to rank well, your internal linking structure is critical to your success. This is covered some of the theory for this in various articles and blogs about Page Structure of a website, which said how you should map out the physical linking structure, but in this guide I will… >>> More
How to Improve Search Engine Ranking by On-Page Search Engine Optimization

as seen on Ezine Articles - Search for 'Ezine Articles'
There are two types of search engine optimization (SEO) techniques: on-page and off-page SEO. With on-page SEO, you have full control of all the techniques and methods you can use on your own sites. With off-page SEO, you will try some techniques to get other site owners to refer to your site which… >>> More
Search Engine Optimization For Great Search Engine Placement!

as seen on Ezine Articles - Search for 'Ezine Articles'
Do you do enough search engine optimization to get the search engine placement that you want for the keywords that you want to rank for? If not then read on and I will give you information that you need to know to start getting those rankings that you want, and start receiving traffic! There are a… >>> More
Search Engine Optimization - Use of META Tag in Search Engine Marketing

as seen on Ezine Articles - Search for 'Ezine Articles'
Want your WEBPAGE to land in the 1st PAGES of Google and Yahoo search engines? Then, make use of this powerful META TAG in your webpage/s, and you will have the result you have been aching for ages. >>> More

Related posts about apache2

apache2: Could not open configuration file /etc/apache2/apache2.conf: Permission denied

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I recently upgraded Ubuntu to the latest LTS edition on my work laptop, which I use as a LAMP development platform. The upgrade was from 12.4 to 14.4. Now I'm having trouble getting apache up and running again. Here is the output from an attempt: antonc@antonc-laptop:/etc/apache2$ sudo service apache2… >>> More
Apache segfault glibc segfault

as seen on Server Fault - Search for 'Server Fault'
I keep getting (about every 5-6 hours) this segfault in apache: [Tue Jun 26 12:43:10 2012] [notice] child pid 26810 exit signal Aborted (6) *** glibc detected *** /usr/sbin/apache2: free(): invalid pointer: 0xb68c2628 *** ======= Backtrace: ========= /lib/i386-linux-gnu/libc.so.6(+0x6ff22)[0xb75aef22] /lib/i386-linux-gnu/libc… >>> More
Localhost not working after installing PHP on Mountain Lion

as seen on Server Fault - Search for 'Server Fault'
I've installed php using brew install php54 --with-mysql, I've set up all the path correctly. which php will give me /usr/local/bin/php php -v will give me PHP 5.4.8 (cli) (built: Nov 20 2012 09:29:31) php --ini will give me: Configuration File (php.ini) Path: /usr/local/etc/php/5.4 Loaded Configuration… >>> More
Apache 2 Virtual Hosts no working on OSX 10.6

as seen on Server Fault - Search for 'Server Fault'
This is my first MacBook and I'm trying to get virtual hosts up and running so as it's going to be my dev machine. I've got apache/php/mysql running fine, the problem is that what ever address I go to I just get one of the virtual hosts I've setup. I can't even get to the root site anymore. I had… >>> More
apache2 error Could not open configuration file /etc/apache2/conf.d/: No such file or directory

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I have just upgraded my Ubuntu 13.10 and apache2 is not working. When I try to start the apache2 server it is printing following errors: * Starting web server apache2 * The apache2 configtest failed. Output of config test was: apache2: Syntax error on line 263 of /etc/apache2/apache2.conf: Could… >>> More