The trouble with data

I a bit of a data freak. I like my data, preferably correct. I feel like James Bond as he approaches the bar. How do you like your data? "Accurate, not Cut". That would be me. I don't take it re-worked, re-cut or in extracts. I take it raw so I know what it is, what it means, what it tells me, what flaws it has.

The problem with data is that people more often than not do not understand what they are looking at. You asked for a piece of analysis and someone sends you a chart and you take it as it is. You believe you have done the right question and the person has interpreted what you were after the write way, extracted what you intended. A whole bunch of assumptions which my time working inside an organization (other than in the client facing side of the business) has taught me are mostly flawed.

When we start looking at a problem, most likely we don't really know what we are after. We feel that there is some data that illustrates our story but really we may not even be asking for the right things. So we get Finance or Ops, or HR to run whatever data we feel is the right one and after what is usually not a short turn around and a probably cumbersome project on the other side, we get some nicely formatted charts. My usually response "kindly send through the raw data". I can feel the mouse shaking on the other side. Why does she want the raw data. She does not even understand what this data is. Haven't I answered your question? Well yes, partially, but I am conscious I may not know the question I am asking is the right one, and therefore I want to look at the data and just let the screen tell me a story, rather than have a pre-done story and find the facts to go along. Because of my typical outsider perspective in many problems, it is not uncommon that an analysis will take many iterations until we even get the data set aligned what I am really after.

So now you have this raw data, you know there are different questions to be asked, you start drawing new conclusions. Data is king right, so your analysis is by now looking unique and truly interesting. You draft charts around, you even put together some slides telling this story. What does it all mean. Well, look at all this data. And suddenly someone asks a basic question that immediately highlights the data is not correct. The analysis becomes flawed even if it is still perfectly valid, you lose an incredible amount of credibility and you go back to square one. Checks and balances. That is something that often does not exist with data producers. Because they do not have the full picture, the people that pull out the reports are more often than not unable to evaluate the acccuracy of the data they are providing. I know, ludicrous. By no means should this be even possible, that is why you have people extracting reports rather than machines. But it is the case in many places. So as an analytical thinker, before analysing, stop and think whether what you are looking at even makes sense. I have sent data sets back within 30 seconds of receipt by simply asking "kindly clarify why the total does not match the total in report x that you sent last week". This usually gives rise to a whole other round of iterations that leads to new data sets and ultimately an accurate and complete data set.

Is this it then? Well, only partially. Data sets some times give you but a side of the story, or but the icing on the cake. If you get data that is aggregated at a too high level, than chances are you need to go back and forward trying to figure out why trends changed and what are the drivers as the data has insufficient detail. Here, there is a fine balance. I always err on the side of too much data, but I recognise data sets are just so large that we do have to filter out fields and it is sometimes more efficient to ask for additions than to deal with spreadsheets that do not load on any common person excel. Or that block each time you add values. You don't want to deal with the wheel that shows up when excel is "thinking". So shooting at common fields but keeping an open mind to what may be missing is important, so you can dig deeper into a few points.

But there is the digging dipper and the digging laterally. Sometimes, the data may only tell one side, and as we look at different sources we may reach complementary, more insightful (though sometimes contradictory) outcomes. Data is king, but there is a queen in the game as well, how to chose then? There is no chosing really, there is only then understanding what different sources tell you and why they may be represented in different ways. That will broaden the understanding and avoid undermining by different players.

The thing with data, is that people make decisions on the basis of it. It is called management information, under the assumption it informs management of the decisions to be done. However, in large organizations, MI can be built in many different ways and tell many different stories. The key importance to know different sources, so you understand the different stories, and can challenge each and any one of them.