What is Open Data and why to use it?

Open Data is any information the government agencies and municipalities collect for their own needs but then offer for others to use. Open Data is usually available in machine-readable formats and can be used as it is or be a part of third party solutions, including commercial products.

By using Open Data or combining different sources of data, developers can enrich functionality of their mobile and Web applications. For example, food safety inspection results can be used by dinner planning app to provide better recommendation about restaurants to visit.

While all good practices for working with remote data sets are relevant for Open Data as well, there are some specifics, as I learned developing my Open Data apps for Windows Phone and Windows 8.

Importance of data caching

Cache for Open Data is not just a traffic saver for users on metered networks, but also a way to improve app’s usability. By displaying cached data first and downloading (and then presenting) most recent data asynchronously, startup time for app can be reduced significantly.

Most of Open Data sources have well defined update interval, for example daily, every Thursday, etc. Knowing these intervals, developers can tune cache lifetime to minimize unnecessary data requests.

To simplify building of network-optimized Windows 8/Windows Phone apps developers can use already existing libraries, like AgFX or Akavache.

Speaking of data caching and updating, an important point to remember is the modern app lifecycle. Mobile application can stay suspended for days and when app is resumed this is developer’s responsibility to update UI if new data is available.

Error handling

Most of the times, government and municipal Open Data services are quite reliable. However, if they are down and application is not able to retrieve new data, it should not result in application crash. Instead, application can use cache to display last available information and notify user about temporary unavailability of the service.

Application should indicate whether data is outdated and when it was retrieved.

Complex data sources

When application works with multiple data sources, custom Web or Cloud server can be used to merge or correlate data and expose it through simplified Web API. Same approach is useful when data is published in form of Excel or Word documents – parsing files on server and exposing data via HTTP.

Obviously, publishing of Excel files is the easiest approach for data owners, but it adds unnecessary complexity for data consumption by mobile apps. When necessary data is published as Excel spreadsheet only, developers should try to contact corresponding Open Data curators and request data in alternative formats.

Licensing

Open Data is free to use and usually licensed with just a few conditions. These conditions may vary from agency to agency bust the most common requirement is to acknowledge data source by providing some statement like “Contains public sector Datasets made available under the City of Toronto’s Open Data License”.

This statement presented in your app rises citizens’ awareness about Open Data initiatives and increases chances of publishing additional datasets, so make it visible!