It’s not something new that Mobile Phones and Mobile Applications comes into our daily life, and they come not only as a tool for communication and entertainment. Many enterprises adopt mobile technologies to give their field workers ability to fulfill operations that previously could be done only in office.

Enterprise mobile applications are recognized as a separate class of mobile application. Microsoft outlines a set of features important to enterprise users and providing tools and practices to fulfill them.

These features are:

• Offline data synchronization;
• Enterprise Sign-On;
• Connect to on-premises data;
• Push notifications.

Support for these features is in the core of Microsoft Azure Mobile App offering for enterprises (former Azure Mobile Services). As Microsoft is a developer-oriented company, it provides an SDK (here), tutorials and documentation (here) on how to get started, so we will skip this part.

In this blog post, I’d like to focus your attention on the implementation approach and design decisions that should be taken in account to be aligned with Microsoft approach when implementing Offline data synchronization. Hopefully, that will be interesting and helpful also for those who are building Mobile Apps outside of Microsoft technology stack.

The Problem

You have a worker in the field, doing an inspection of underground facilities, or working in conditions when mobile connectivity is either low or otherwise unstable. How can you ensure that your employee has the right data at the right time? – Make data available at your employee’s device.

On the high level, any offline data solution for mobile applications relies on keeping data in the local data storage of the mobile device. But devil is in the details and there are some questions:

• How to get latest data from the server to local data store without loading full production database?
• How to submit updates from a local data store to the server?
• How to handle data deleted by other users?
• How to handle conflicts?

The Sample

We will follow the Task entity, a Note entity, and a Task REST Service to see how they both evolve on their way to Offline Synchronization.

Figure 1. Data Model for presented sample

On the Server side, there is a REST service that manages two resources – Task and Note. REST Service accepts requests with GET, POST (insert), PATCH (update), DELETE HTTP verbs. As a request body REST services accepts entities in JSON format.

Prerequisite

The Offline Synchronization approach adopted in Azure Mobile Services SDK relies on modifications made to the original entity. The framework practically enforces following system fields to the entity that should be managed by both mobile client and mobile server:

• CreatedAt: date time offset – the timestamp of entity creation;
• UpdatedAt: date time offset – the timestamp of entity update;
• Deleted: bool – the indicator that entity is to be deleted (Soft Delete feature)
• Version: byte[] – the version of the entity.

Workflows

The whole Offline Synchronization process is orchestrated by Mobile Application and can be separated in few concepts that we will review in details.

• Local Storage – how data is persisted on mobile device;
• Push – process of sending changes made locally to the server;
• Pull – process of retrieving data from the server to local storage;
• Purge – process of cleaning-up local storage.

Local Storage

Obviously, Local Storage in a mobile application holds a copy of data from the server. But apart from that, it has another important purpose – act as a tracker for all changes done to local entities. When the time comes, you should be able to retrieve a list of all changes done to all entities that are stored locally.

Let’s see the example.

1. You add a Task locally on your mobile application;
2. Then you decide to add a Note to your Task;
3. Then you also decide to modify the Task entity, because you dislike the original description.

The previous example will result in two insert transactions persisted to the Local Storage. The minimal required information to store each transactions are {TransacitonNo} / {Operation} / {Entity} / {EntityId}.

Push

Push is the process of delivering changes accumulated by Local Storage tracker to the server. The Push is pretty straightforward – for each transaction a corresponding HTTP request is generated with appropriate HTTP verb.

Figure 2. Data Push form Mobile Device to Mobile Service

Insert

POST http://localhost/{EntityName}

Header: POST http://localhost/Task

Body:

{

    “Id”: “123456”,

    “Caption”: “Buy milk”

}

Update

PATCH http://localhost/{EntityName}/{id}

Header: PATCH http://localhost/Task/123456

Body:

{

    “Id”: “123456”,

    “Caption”: “Buy milk and eggs”

}

Delete

DELETE http://localhost/{EntityName}/{id}

Header: DELETE http://localhost/Task/123456

It is essential that Push is executed for each entity in transaction log – this ensures data consistency for related entities.

Pull

Pull is the process of retrieving changes from the server and delivering them to the mobile application local storage.

Figure 3. Data Pull from Mobile Service to Mobile Device

The first rule of Pull process - Push your data before executing Pull requests. This rule prevents a scenario where an update is done to a local storage, but a Pull operation would overwrite local changes and potentially leave the data in an inconsistent state.

As you might suspect, Mobile Application sends a GET HTTP request to retrieve data stored on the server.

GET http://localhost/{EntityName}

But that actually is not quite true. The big issue with this approach is:

• It pulls all data and may result in large bills for mobile data usage;
• The unfiltered request may result in a large response, which is very sensitive to the unstable connection.

Azure Mobile Services SDK natively adopts the OData protocol, and with it, it is pretty easy to retrieve only latest changes done to the server database. Here is a sample of a full Pull request sent from mobile application:

GET http://localhost/{EntityName}?$filter=(__updatedAt >= datetimeoffset({timestamp}))&$orderby=__updatedAt&$skip=0&$top=50&__includeDeleted=true&__systemproperties=__updatedAt,__deleted

Let’s analyze it in details:

$filter=(__updatedAt >= datetimeoffset({timestamp}))

Here, a previously enforced UpdatedAt entity property is utilized to get only modified records since last Pull request, reducing the amount of retrieved data to the minimum. The timestamp parameter is persisted in mobile application local storage and is used as a parameter in consequent calls.

$orderby=__updatedAt

When receiving ordered data, the mobile application may safely assume that the last received entity holds the last change, so the UpdatedAt entity property may be persisted as a timestamp in device Local Storage for the next call. Also, this is part of paging mechanism implementation (see next).

$skip=0&$top=50

These two parameters implement paging, which helps to keep each message traveling between the server and mobile application be short. This way, if message randomly fails, it can be repeated with less effort. The mobile application continuously increases skip parameter until zero entities are received in response.

$__includeDeleted=true

This parameter is part of the Soft Delete feature of mobile service. Occasionally, there is a need to delete some data from the system. How to ensure that data is deleted (not visible to users), but all other mobile devices are notified about removal and are able to delete this entity? The enforced Deleted property does the trick. Instead of actual Delete operation, entities are marked as Deleted = true. This way, other mobile devices will be notified.

Purge

Purge process for mobile application means – start a new life

Sometimes, when data is in inconsistent state in the mobile device, or it accumulated a lot of obsolete data (if Soft Delete feature is not used), Purge process allows to clean up the local storage.

But remember: Push your data before executing Purge; otherwise you will just erase all changes done by your user.

Conflicts

The topic of conflicts, when several users attempt to modify the same entity is an important one. The safe conflict handling is ensured by the use yet another enforced property Version.  The approach proposed by Azure Mobile Services SDK is based on Optimistic Concurrency strategy.

The mobile client application receives Version of each entity when it Pulls data from the server. When Push is executed (and Version field is provided) the server may check the Version property against the stored and decide on rejecting the update – in this case, a HTTP Error Code 409 (Conflict) is returned. Now, it is up to client application what to do with the Error. Azure Mobile Client SDK allows developers to provide handler to handle conflicts in desired way:

• Handler may force the client version to the server, overwriting server changes;
• Handler may overwrite the client version with server version;
• Handler may block further Push operation execution.

Approach for conflict resolution depends on specific application requirements and should be decided by developers.

Summary

The Microsoft Azure Mobile Services SDK wraps all described approach into simple and pretty straightforward API – available both for Client and Server side. But I hope you might find this information useful when either assessing the use of Azure Mobile Services in your solution or implementing your own Mobile Application outside of Microsoft stack.

About the author: Dmitrijs Jesilevskis is an experienced software developer and architect, focusing on Microsoft and Microsoft Azure technology stack, BizTalk integrations and SOA approach. He is interested in analysis, design and implementation of complex Enterprise solution based on Microsoft technologies.