Anonymous users support vs Google bot

Posted by Andy on Stack Overflow See other posts from Stack Overflow or by Andy
Published on 2010-05-12T17:29:52Z Indexed on 2010/05/12 17:34 UTC
Read the original article Hit count: 191

Filed under:

authentication

|

users

|

googlebot

I have a User class in my web app that represents a user currently logged in. Every time a user vists a page, a User instance is populated based on authentication data supplied in cookies.

A User instance is created even if an anonymous user logs in - and a corresponding new record is created in the User table in the database.

This approach allows me to save some state info for the current user regardless of its type.

The problem however with this approach is the Google bot, and other non-human web organisms crawling my pages. Every time a bot starts to walk around the site, thousands of useless records will be created in the database, each of them only to be used for a single page.

Question: what is the best trade off? How to support anonymous users, save their state, and don't get too much overhead because of cookieless bots?

© Stack Overflow or respective owner

Related posts about authentication

Configuring Fed Authentication Methods in OIF / IdP

as seen on Oracle Blogs - Search for 'Oracle Blogs'
In this article, I will provide examples on how to configure OIF/IdP to map OAM Authentication Schemes to Federation Authentication Methods, based on the concepts introduced in my previous entry. I will show examples for the three protocols supported by OIF: SAML 2.0 SSO SAML… >>> More
windows authentication vs sql server authentication for asp.net forms authentication site

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a database and a site having forms authentication. It is working fine with VS2008. This time, I am using "Trusted_connection =True". But when it is opened from outside or directly from browser then I am getting error "Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'." I know this is due… >>> More
Disable authentication on subfolder(s) of an ASP.NET app using windows authentication

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, Is it possible to disable windows authentication on one or more subfolders of an ASP.net application using windows authentication? >>> More
AutoCompleteExtender - authentication failure (forms authentication)

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm using the AutoCompleteExtender from the AJAX control toolkit on my aspx page - I have it wired up to a WCF service that is returning a string array and everything works happily. If I change my service definition to include a demand for the caller to be authenticated, like so: <OperationContract()… >>> More
Ruby on Rails , functionalities having twitter authentication and email authentication (autlogic and

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Experts, I would like to ask your guidance on how to authenticate using email add/ the basic log-in while having also a twitter log-in, twitter authentication works fine but if having an alternative log-in like using basic sign-up and email log-in part wont work.... any ideas please...? >>> More

Related posts about users

rake migration aborted: could not find table 'roles'

as seen on Stack Overflow - Search for 'Stack Overflow'
I just inherited code that I'm attempting to run the migrations for but I keep getting a rake aborted error. I've come across others that have what appears to be similar issues, but most involved Heroku and I'm trying to run this locally (to start.) I've tried troubleshooting using both PostgreSQL… >>> More
Including an embedded framework using a cross-project-reference: Header no such file or directory

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to create a Cocoa framework by using a cross-project reference in Xcode. I have 2 projects: one for the framework; one for the application that will use the framework. This framework is not intended to be stored within the system; it is an embedded framework that lives within the application… >>> More
Spork servers super slow (>3m) to start for RSpec & Cucumber BDD

as seen on Stack Overflow - Search for 'Stack Overflow'
I recently installed a fresh development setup on my laptop and now notice that my instances of spork take several minutes to start up. This is also most likely of the RSpec and Cucumber tests start up times running super slow. I ran in diagnostic mode with the -d flag and received the output below… >>> More
Sinatra and XML POST request

as seen on Stack Overflow - Search for 'Stack Overflow'
I don't know is it my mistake or no. So i have that code: <code> post '/singin/get_token' do content_type :xml puts request.body.read puts xmlRequest xmlRequest = REXML::Document.new(request.body.read) ... </code> And when i post something like that: <code> <?xml… >>> More
Exception with RubyAMF and Ruby 1.9 although code works

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm getting an exception with RubyAMF using Ruby 1.9 and Rails 2.3.5. Although code afterward executes normally I'm not very comfortable with seeing such exception in the log file. Do you know what is causing it: >>>>>>>> RubyAMF >>>>>>>>> #<RubyAMF::Actions::PrepareAction:0x0000010139ff48>… >>> More