Can I use a Google Appliance/Mini to crawl and index sites I don't own?
- by SkippyFire
Maybe this is a stupid question, but...
I am working with this company and they said they needed to get "permission" to crawl other people's sites. They have a Google Search Appliance And some Google Minis and want to point them at other sites to aggregate content. The end result will be something like a targeted search engine. (All the indexed sites relate to a specific topic)
The only thing they will be doing is:
Indexing Content from the other sites/domains
Providing search functionality on their own site that searches the indexed content (like Google, displaying summaries and not the full content)
The search results will provide links back to the original content
Their intent is not malicious in nature, and is to provide a single site/resource for people to reference on their given topic.
Is there anything illegal or fishy about this process?
