Mobile App Data Syncronization
- by Matt Rogish
Let's say I have a mobile app that uses HTML5 SQLite DB (and/or the HTML5 key-value store). Assets (media files, PDFs, etc.) are stored locally on the mobile device.
Luckily enough, the mobile device is a read-only copy of the "centralized" storage, so the mobile device won't have to propagate changes upstream.
However, as the server changes assets (creates new ones, modifies existing, deletes old ones) I need to propagate those changes back to the mobile app.
Assume that server changes are grouped into changesets (version number n) that contain some information (added element XYZ, deleted id = 45, etc.) and that the mobile device has limited CPU/bandwidth, so most of the processing has to take place on the server.
I can think of a couple of methods to do this. All have trade-offs and at this point, I'm unsure which is the right course of action...
Method 1:
For change set n, store the "diff" of the current n and previous n-1. When a client with version y asks if there have been any changes, send the change sets from version y up to the current version.
e.g.
added item 334, contents: xxx. Deleted picture 44. Deleted PDF 11. Changed 33.
added picture 99.
Characteristics:
Diffs take up space, although in theory would be kept small. However, all diffs must be kept around indefinitely (should a v1 app have not been updated for a year, must apply v2..v100).
High latency devices (mobile apps) will incur a penalty to send lots of small files (assume cannot be zipped or tarr'd up into one file)
Very few server CPU resources required, as all it does is send the client a list of files
"Dumb" - if I change an item in change set 3, and change it to something else in 4, the client is going to perform both actions, even though #3 is rendered moot by #4. Or, if an asset is added in #4 and removed in #5 - the client will download a file just to delete it later.
Method 2:
Very similar to method 1 except on the server, do some sort of a diff between the change sets represented by the app version and server version. Package that up and send that single change set to the client.
Characteristics:
Client-efficient: The client only has to process one file, duplicate or irrelevant changes are stripped out.
Server CPU/space intensive. The change sets must be diff'd and then written out to a file that is then sent to the client. Makes diff server scalability an issue. Possibly ways to cache the results and re-use them, but in the wild there's likely to be a lot of different versions so the diff re-use has a limit
Diff algorithm is complicated. The change sets must be structured in such a way that an efficient and effective diff can be performed.
Method 3:
Instead of keeping diffs, write out the entire versioned asset collection to a mobile-database import file. When client requests an update, send the entire database to client and have them update their assets appropriately.
Characteristics:
Conceptually simple -- easy to develop and deploy
Very inefficient as the client database is restored every update. If only one new thing was added, the whole database is refreshed.
Server space and CPU efficient. Only the latest version DB needs kept around and the server just throws the file to the client.
Others?? Thoughts? Thanks!!
