Bulkloading schema less entities on Google App Engine

Posted by Rahul on Stack Overflow See other posts from Stack Overflow or by Rahul
Published on 2010-05-24T23:46:27Z Indexed on 2010/05/24 23:51 UTC
Read the original article Hit count: 265

Filed under:

google-app-engine

|

bulk-load

The new bulkloader added into SDK 1.3.4 works great for models that have a schema. For models inheriting db.Expando (or loosely defined schemas) i would like to understand what i would have to do to bulk upload them.

I defined a custom connector, that implemented the ConnectorInterface and converted my data to the python dict required. How can i use this dict to define entities that get uploaded to the data store ?

In the documentation there seems to be a post_import_function that can be used to return the entities that get uploaded. Is there an example on how this function is used ?

© Stack Overflow or respective owner

Related posts about google-app-engine

unittest import error with virtualenv + google-app-engine-django

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm working with google-app-engine-django + zipped django. Just running "python manage.py test" succeeded without error. But with virtualenv, test was failed with "import unittest error". same error with Django 1.1. - OSX 10.5.6 - google-app-engine-django (r101 via svn) : r100 was failed with launcher… >>> More
Could not import Django settings into Google App Engine

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello all you Google App Engine experts, I have used Django a little before but am new to Google App Engine and am trying to use it's development web server with Django for the first time. I don't know if this is relevent but I previously had Django 1.1 and Python 2.6 on my Windows XP and even… >>> More
How does Google App Engine precompile Java ?

as seen on Stack Overflow - Search for 'Stack Overflow'
App Engine uses a "precompilation" process with the Java bytecode of an app to enhance the performance of the app in the Java runtime environment. Precompiled code functions identically to the original bytecode. Is there any detailed information what this does? >>> More
Creating a session scoped bean in Google App Engine using Spring 2.5

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am trying to create a session bean in spring mvc. I am having the following error message when I run my google app engine server in my local box: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'siteController' defined in ServletContext resource [/WEB-INF/springapp-servlet… >>> More
Google App Engine UI Widgets

as seen on Stack Overflow - Search for 'Stack Overflow'
Are there any UI widgets available to the python side of Google App Engine? I'd like something like the collapsed/expanded views of Google Groups threads. Are these type things limited to the GWT side? >>> More

Related posts about bulk-load

Using SQLXML Bulk Load in .NET Environment - Error with One to Many relationship on Complex Type

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I have an error when I am importing an XML file using SQLXMLBulkLoad, wondering if anyone could help. Error: Data mapping to column 'Attribute' was already found in the data. Make sure that no two schema definitions map to the same column Full files and details can be found here http://www.experts-exchange… >>> More
STA threads with SQLXMLBULKLOAD

as seen on Stack Overflow - Search for 'Stack Overflow'
If I have N STA .NET Threads each performing an independent bulk load operation on a different database using the SQLXMLBulkLoad dll (which requires calling threads to be STA), is it possible for all bulk loads to be happening at the same time, or are they implicitly serialized due to the STA COM… >>> More
MySQL to AppEngine

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Nick! How are you? I'm from Brazil and study at FATEC (college located in Brazil). I'm trying to learn about AppEngine. Now, I'm trying to load a large database from MySQL to AppEngine to perform some queries, but I don't know how i can do it. I did some testing with CSV files,but is there any… >>> More
Bulkloading schema less entities on Google App Engine

as seen on Stack Overflow - Search for 'Stack Overflow'
The new bulkloader added into SDK 1.3.4 works great for models that have a schema. For models inheriting db.Expando (or loosely defined schemas) i would like to understand what i would have to do to bulk upload them. I defined a custom connector, that implemented the ConnectorInterface and converted… >>> More
SQL Server 2008 BULK INSERT causes more reads than writes. Why?

as seen on Stack Overflow - Search for 'Stack Overflow'
I've huge a table (a few billion rows) with a clustered index and two non-clustered indices. A BULK INSERT operation produces 112000 reads and only 383 writes (duration 19948ms). It's very confusing to me. Why do reads exceed writes? How can I reduce it? update query insert bulk DenormalizedPrice4… >>> More