ASP.NET and HTML5 Local Storage
- by Stephen Walther
My favorite feature of HTML5, hands-down, is HTML5 local storage (aka DOM storage). By taking advantage of HTML5 local storage, you can dramatically improve the performance of your data-driven ASP.NET applications by caching data in the browser persistently. Think of HTML5 local storage like browser cookies, but much better. Like cookies, local storage is persistent. When you add something to browser local storage, it remains there when the user returns to the website (possibly days or months later). Importantly, unlike the cookie storage limitation of 4KB, you can store up to 10 megabytes in HTML5 local storage. Because HTML5 local storage works with the latest versions of all modern browsers (IE, Firefox, Chrome, Safari), you can start taking advantage of this HTML5 feature in your applications right now. Why use HTML5 Local Storage? I use HTML5 Local Storage in the JavaScript Reference application: http://Superexpert.com/JavaScriptReference The JavaScript Reference application is an HTML5 app that provides an interactive reference for all of the syntax elements of JavaScript (You can read more about the application and download the source code for the application here). When you open the application for the first time, all of the entries are transferred from the server to the browser (all 300+ entries). All of the entries are stored in local storage. When you open the application in the future, only changes are transferred from the server to the browser. The benefit of this approach is that the application performs extremely fast. When you click the details link to view details on a particular entry, the entry details appear instantly because all of the entries are stored on the client machine. When you perform key-up searches, by typing in the filter textbox, matching entries are displayed very quickly because the entries are being filtered on the local machine. This approach can have a dramatic effect on the performance of any interactive data-driven web application. Interacting with data on the client is almost always faster than interacting with the same data on the server. Retrieving Data from the Server In the JavaScript Reference application, I use Microsoft WCF Data Services to expose data to the browser. WCF Data Services generates a REST interface for your data automatically. Here are the steps: Create your database tables in Microsoft SQL Server. For example, I created a database named ReferenceDB and a database table named Entities. Use the Entity Framework to generate your data model. For example, I used the Entity Framework to generate a class named ReferenceDBEntities and a class named Entities. Expose your data through WCF Data Services. I added a WCF Data Service to my project and modified the data service class to look like this: using System.Data.Services;
using System.Data.Services.Common;
using System.Web;
using JavaScriptReference.Models;
namespace JavaScriptReference.Services
{
[System.ServiceModel.ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public class EntryService : DataService<ReferenceDBEntities> {
// This method is called only once to initialize service-wide policies.
public static void InitializeService(DataServiceConfiguration config) {
config.UseVerboseErrors = true;
config.SetEntitySetAccessRule("*", EntitySetRights.All);
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
}
// Define a change interceptor for the Products entity set.
[ChangeInterceptor("Entries")]
public void OnChangeEntries(Entry entry, UpdateOperations operations) {
if (!HttpContext.Current.Request.IsAuthenticated) {
throw new DataServiceException("Cannot update reference unless authenticated.");
}
}
}
}
The WCF data service is named EntryService. Notice that it derives from DataService<ReferenceEntitites>. Because it derives from DataService<ReferenceEntities>, the data service exposes the contents of the ReferenceEntitiesDB database.
In the code above, I defined a ChangeInterceptor to prevent un-authenticated users from making changes to the database. Anyone can retrieve data through the service, but only authenticated users are allowed to make changes.
After you expose data through a WCF Data Service, you can use jQuery to retrieve the data by performing an Ajax call. For example, I am using an Ajax call that looks something like this to retrieve the JavaScript entries from the EntryService.svc data service:
$.ajax({
dataType: "json",
url: “/Services/EntryService.svc/Entries”,
success: function (result) {
var data = callback(result["d"]);
}
});
Notice that you must unwrap the data using result[“d”]. After you unwrap the data, you have a JavaScript array of the entries.
I’m transferring all 300+ entries from the server to the client when the application is opened for the first time. In other words, I transfer the entire database from the server to the client, once and only once, when the application is opened for the first time.
The data is transferred using JSON. Here is a fragment:
{
"d" : [
{
"__metadata": {
"uri": "http://superexpert.com/javascriptreference/Services/EntryService.svc/Entries(1)", "type": "ReferenceDBModel.Entry"
}, "Id": 1, "Name": "Global", "Browsers": "ff3_6,ie8,ie9,c8,sf5,es3,es5", "Syntax": "object", "ShortDescription": "Contains global variables and functions", "FullDescription": "<p>\nThe Global object is determined by the host environment. In web browsers, the Global object is the same as the windows object.\n</p>\n<p>\nYou can use the keyword <code>this</code> to refer to the Global object when in the global context (outside of any function).\n</p>\n<p>\nThe Global object holds all global variables and functions. For example, the following code demonstrates that the global <code>movieTitle</code> variable refers to the same thing as <code>window.movieTitle</code> and <code>this.movieTitle</code>.\n</p>\n<pre>\nvar movieTitle = \"Star Wars\";\nconsole.log(movieTitle === this.movieTitle); // true\nconsole.log(movieTitle === window.movieTitle); // true\n</pre>\n", "LastUpdated": "634298578273756641", "IsDeleted": false, "OwnerId": null
}, {
"__metadata": {
"uri": "http://superexpert.com/javascriptreference/Services/EntryService.svc/Entries(2)", "type": "ReferenceDBModel.Entry"
}, "Id": 2, "Name": "eval(string)", "Browsers": "ff3_6,ie8,ie9,c8,sf5,es3,es5", "Syntax": "function", "ShortDescription": "Evaluates and executes JavaScript code dynamically", "FullDescription": "<p>\nThe following code evaluates and executes the string \"3+5\" at runtime.\n</p>\n<pre>\nvar result = eval(\"3+5\");\nconsole.log(result); // returns 8\n</pre>\n<p>\nYou can rewrite the code above like this:\n</p>\n<pre>\nvar result;\neval(\"result = 3+5\");\nconsole.log(result);\n</pre>", "LastUpdated": "634298580913817644", "IsDeleted": false, "OwnerId": 1
}
…
]}
I worried about the amount of time that it would take to transfer the records. According to Google Chome, it takes about 5 seconds to retrieve all 300+ records on a broadband connection over the Internet. 5 seconds is a small price to pay to avoid performing any server fetches of the data in the future.
And here are the estimated times using different types of connections using Fiddler:
Notice that using a modem, it takes 33 seconds to download the database. 33 seconds is a significant chunk of time. So, I would not use the approach of transferring the entire database up front if you expect a significant portion of your website audience to connect to your website with a modem.
Adding Data to HTML5 Local Storage
After the JavaScript entries are retrieved from the server, the entries are stored in HTML5 local storage. Here’s the reference documentation for HTML5 storage for Internet Explorer:
http://msdn.microsoft.com/en-us/library/cc197062(VS.85).aspx
You access local storage by accessing the windows.localStorage object in JavaScript. This object contains key/value pairs. For example, you can use the following JavaScript code to add a new item to local storage:
<script type="text/javascript">
window.localStorage.setItem("message", "Hello World!");
</script>
You can use the Google Chrome Storage tab in the Developer Tools (hit CTRL-SHIFT I in Chrome) to view items added to local storage:
After you add an item to local storage, you can read it at any time in the future by using the window.localStorage.getItem() method:
<script type="text/javascript">
window.localStorage.setItem("message", "Hello World!");
</script>
You only can add strings to local storage and not JavaScript objects such as arrays. Therefore, before adding a JavaScript object to local storage, you need to convert it into a JSON string. In the JavaScript Reference application, I use a wrapper around local storage that looks something like this:
function Storage() {
this.get = function (name) {
return JSON.parse(window.localStorage.getItem(name));
};
this.set = function (name, value) {
window.localStorage.setItem(name, JSON.stringify(value));
};
this.clear = function () {
window.localStorage.clear();
};
}
If you use the wrapper above, then you can add arbitrary JavaScript objects to local storage like this:
var store = new Storage();
// Add array to storage
var products = [
{name:"Fish", price:2.33},
{name:"Bacon", price:1.33}
];
store.set("products", products);
// Retrieve items from storage
var products = store.get("products");
Modern browsers support the JSON object natively. If you need the script above to work with older browsers then you should download the JSON2.js library from:
https://github.com/douglascrockford/JSON-js
The JSON2 library will use the native JSON object if a browser already supports JSON.
Merging Server Changes with Browser Local Storage
When you first open the JavaScript Reference application, the entire database of JavaScript entries is transferred from the server to the browser. Two items are added to local storage: entries and entriesLastUpdated. The first item contains the entire entries database (a big JSON string of entries). The second item, a timestamp, represents the version of the entries.
Whenever you open the JavaScript Reference in the future, the entriesLastUpdated timestamp is passed to the server. Only records that have been deleted, updated, or added since entriesLastUpdated are transferred to the browser.
The OData query to get the latest updates looks like this:
http://superexpert.com/javascriptreference/Services/EntryService.svc/Entries?$filter=(LastUpdated%20gt%20634301199890494792L)
If you remove URL encoding, the query looks like this:
http://superexpert.com/javascriptreference/Services/EntryService.svc/Entries?$filter=(LastUpdated gt 634301199890494792L)
This query returns only those entries where the value of LastUpdated > 634301199890494792 (the version timestamp).
The changes – new JavaScript entries, deleted entries, and updated entries – are merged with the existing entries in local storage. The JavaScript code for performing the merge is contained in the EntriesHelper.js file. The merge() method looks like this:
merge: function (oldEntries, newEntries) {
// concat (this performs the add)
oldEntries = oldEntries || [];
var mergedEntries = oldEntries.concat(newEntries);
// sort
this.sortByIdThenLastUpdated(mergedEntries);
// prune duplicates (this performs the update)
mergedEntries = this.pruneDuplicates(mergedEntries);
// delete
mergedEntries = this.removeIsDeleted(mergedEntries);
// Sort
this.sortByName(mergedEntries);
return mergedEntries;
},
The contents of local storage are then updated with the merged entries.
I spent several hours writing the merge() method (much longer than I expected). I found two resources to be extremely useful.
First, I wrote extensive unit tests for the merge() method. I wrote the unit tests using server-side JavaScript. I describe this approach to writing unit tests in this blog entry. The unit tests are included in the JavaScript Reference source code.
Second, I found the following blog entry to be super useful (thanks Nick!):
http://nicksnettravels.builttoroam.com/post/2010/08/03/OData-Synchronization-with-WCF-Data-Services.aspx
One big challenge that I encountered involved timestamps. I originally tried to store an actual UTC time as the value of the entriesLastUpdated item. I quickly discovered that trying to work with dates in JSON turned out to be a big can of worms that I did not want to open. Next, I tried to use a SQL timestamp column. However, I learned that OData cannot handle the timestamp data type when doing a filter query. Therefore, I ended up using a bigint column in SQL and manually creating the value when a record is updated.
I overrode the SaveChanges() method to look something like this:
public override int SaveChanges(SaveOptions options) {
var changes = this.ObjectStateManager.GetObjectStateEntries(
EntityState.Modified |
EntityState.Added |
EntityState.Deleted);
foreach (var change in changes) {
var entity = change.Entity as IEntityTracking;
if (entity != null) {
entity.LastUpdated = DateTime.Now.Ticks;
}
}
return base.SaveChanges(options);
}
Notice that I assign Date.Now.Ticks to the entity.LastUpdated property whenever an entry is modified, added, or deleted.
Summary
After building the JavaScript Reference application, I am convinced that HTML5 local storage can have a dramatic impact on the performance of any data-driven web application. If you are building a web application that involves extensive interaction with data then I recommend that you take advantage of this new feature included in the HTML5 standard.
