Efficiently Serving Dynamic Content in Google App Engine
- by awegawef
My app on google app engine returns content items (just text) and comments on them. It works like this (pseudo-ish code):
query: get keys of latest content #query to datastore
for each item in content
if item_dict in memcache:
use item_dict
else:
build_item_dict(item) #by fetching from datastore
store item_dict in memcache
send all item_dicts to template
Sorry if the code isn't understandable. I get all of the content dictionaries and send them to the template, which uses them to create the webpage.
My problem is that if the memcache has expired, for each item I want to display, I have to (1) lookup item in memcache, (2) since no memcache exists I must fetch item from the datastore, and (3) store the item in memcache. These calls build up quickly.
I don't set an expire time for the entries to the memcache, so this really only happens once in the morning, but the webpage takes long enough to load (~1 sec) that the browser reports it as not existing. Regularly, my webpages take about 50ms to load.
This approach works decently for frequent visits, but it has its flaws as shown above. How can I remedy this? The entries are dynamic enough that I don't think it would be in my best interest to cache my initial request.
Thanks in advance
