Data is Messy

posted in: technology | 0



Interconnecting urban data-sets can be messy but the Tombolo Digital Connector can help by taking care of some of the heavy lifting.

As we compare a handwritten scribble and a well typeset paper, we can get the image that with digitisation the world is becoming more structured and understandable. With the increased availability of open urban data, we can get our hopes up for a world where decision making is autonomous or at least informed by real data. While both points are true, they can be misleading and give false hopes of the status of our journey toward digital models of cities.

Although individual data sources can be clean and well-structured on their own, combining them together can become messy. One data-source may be in distributed as an Excel sheet while another is in GeoJSON format. One data-source might have attributes gathered at postcode level while another is at the street level. Some data-sets may be distributed as folder of zip-files while other are exposed through an API.

In order to facilitate the job of working with heterogeneous urban data-sources, the Tombolo project has developed the Digital Connector, a software that makes it easier to interconnect data from multiple sources. When using the Digital Connector, the user describes the data they would like to work with and the software takes care of connecting to the appropriate data end-points, harmonising incompatible data formats, aggregating and/or disaggregating between different spatial granularities and delivering a single joined data output.

The Digital Connector will be open-sourced later this year. Stay tuned.

Follow Börkur Sigurbjörnsson:

Latest posts from

Leave a Reply