Can I tell sitecrawlers to visit a certain page?

Posted by Ace on Stack Overflow See other posts from Stack Overflow or by Ace
Published on 2010-06-14T13:49:46Z Indexed on 2010/06/14 13:52 UTC
Read the original article Hit count: 150

Filed under:

drupal

|

indexing

|

seo

|

robots.txt

Hi there!

I have this drupal website that revolves around a document database. By design you can only find these documents by searching the site. But I want all the results to be indexed by Googlebot and other crawlers, so I was thinking, what if I make a page that lists all the documents, and then tell the robots to visit the page to index all my documents..?

Is this possible, or is there a better way to do it?

© Stack Overflow or respective owner

Related posts about drupal

Commercial Drupal Modules & Themes

as seen on Just Skins - Search for 'Just Skins'
A discussion at Drupal.org forums prompted me to give my input about commercial ecosystem around Open Source Content Management Systems. WordPress and Joomla have been growing rapidly since past few years. But, growth rate of Drupal seems to be almost flat. Despite being the most powerful CMS around… >>> More
better to wait Drupal 7 to start learning Drupal, or no big difference with Drupal 6?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi friends, I'm going to start some projects in next weeks. I currently have my custom cms, but i wanna go for opensource cms solution from now on. and as i researched Drupal is COOL for any kind of project from small scope to most complex ones. I have never used Drupal before, I know nothing about… >>> More
Drupal 6: Drupal Themer gives same candidate name for different type of content types

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi friends, I'm a drupal newbie... I have different type of contents like News, Events, etc. and their content is different. News detail page has title-content text-date. but Events detail page has title-date-content text-location-speaker-etc. So I need different layout page for these different… >>> More
drupal: how to generate doc files in drupal is there any possibility to that

as seen on Stack Overflow - Search for 'Stack Overflow'
how to generate doc file in drupal files.. do we have module for that... as pdf generation module... >>> More
Drupal 6 vs Drupal 7 performance

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi all. I want to start new project and stuck without idea what version to use. I have huge expirience with D6, and also one project (module developement) for D7. It looks like D7 slower, have bigger memory consumption and also have a lack of documentation by the moment. I don't need new CCK, Views… >>> More

Related posts about indexing

IIS6 Indexing Service indexing asp.net codebehind (.aspx.cs) files

as seen on Server Fault - Search for 'Server Fault'
I've setup a few catalogs on an Windows Server 2003 IIS6 install, each tracking files within a website. In the Properties - Generation Dialog for each catalog, 'Index files with unknown extensions' is turned OFF. 'Inherit above settings from Service' in that dialog is also turned off. However, the… >>> More
Outlook 2007 OST File Indexing and OneNote 2007 Indexing are Broken

as seen on Super User - Search for 'Super User'
I'm running Outlook 2007 under Windows 7 Home Premium RTM. My OST file was previously being properly indexed but eventually searches significantly slowed down so I suspected a problem. Searching and indexing appears broken in OneNote 2007 as well as search time is now significantly longer. I brought… >>> More
Indexing vs. no indexing when inserting records

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a few questions about whether or not it would be best to not use indexing. BACKGROUND: My records have a timestamp attribute, and the records will be inserted in order of their timestamps (i.e., inserted chronologically). QUESTIONS: If I DON'T use indexing is it typical for the database… >>> More
SQL SERVER – Transcript of Learning SQL Server Performance: Indexing Basics – Interview of Vinod Kumar by Pinal Dave

as seen on SQL Authority - Search for 'SQL Authority'
Recently I just wrote a blog post on about Learning SQL Server Performance: Indexing Basics and I received lots of request that if we can share some insight into the course. Here is 200 seconds interview of Vinod Kumar I took right after completing the course. We have few free codes to watch the course… >>> More
Can the Windows Indexing Service restart an app pool if it doesn't index the Web.Config?

as seen on Stack Overflow - Search for 'Stack Overflow'
I am having a slight debate with a colleague of mine on this subject. Scenario: A web site has a sub directory that is indexed using the Windows Indexing Service. There is not a web.config in this directory and the indexing service is not indexing the parent directory which is the site root. Something… >>> More