People working with data often say they spend 80% of their time getting it into the right shape for analysis. This doesn’t sound like fun and in the Tombolo team we’re trying to make this aspect of using data easier.
Even as a data scientist I think it’s fair to say that most people don’t like staring at reams and reams of numbers. Okay, looking at the raw data can help you check that the information represents what you expect, or it can help you confirm that it’s what you need, but staring at rows in a table or looking at some sort of data hierarchy is not likely to provide enormous amounts of insight.
This is why we adopt statistical summaries, models and data visualisations. In one way or another, taking the average of a column, generating a graph (or map) or performing some sort of modelling helps us to extract the insight that we need from data. You may not have considered it, but if you’ve ever used data to make decisions you’ve probably used statistical summaries, models or visualisations to do so.
Most importantly, that’s the best part – the part where we explore and generate insights. Getting the average values, visualising them on a map and even understanding how they relate to (or predict) other variables that you’re interested in. It’s not the process of finding, downloading, cleaning, preparing and munging data that’s fun. And nobody loves documenting their process so that someone else can check or repeat what they’ve done. It’s the focus on insight and how that can help you make better decisions.
This is why we’ve created the Tombolo Digital Connector (TDC). The TDC helps you to get to the point where you can generate insights by making the process of obtaining, formatting and combining data as easy and reproducible as possible. We’re currently using the Tombolo Digital connector to download, format and combine data about traffic, health, the environment and even the economy. Much of this data comes from different sources including the Office for National Statistics, the Department for Transport, OpenStreetMap, Twitter and even the London Air Quality Network. The best thing is, all of our analyses and processes are fully transparent and reproducible.
Why not try turning your data preparation into something much simpler? With the Tombolo Digital Connector, you can focus your time on the fun stuff like generating insight and it’s all completely free.
Fancy a look at the Digital Connector? Check out our Github account and don’t forget to contribute!