Productivity and the data economy: what’s the problem?
Creating meaning from data across the economy will require hard, organisational change in many firms
The creation, collection, storage, transmission, and use of information has been going on for millennia. Around 10,000 years ago the societies of the First Agricultural Revolution began to plant and harvest crops, recording and passing on what they learned. They might have just remembered and verbally communicated the information they were accumulating; they could have put it down in carvings or clay; or perhaps they had social rituals, like harvest celebrations. Regardless of the method, passing accurate information between people using the social media of the time meant that they could get better at producing food, leading to bigger populations than were possible with hunting and gathering.
Better information helps a society to know more about what to do, and when. If it can regularly record and maintain information about events and change, it can develop a better grasp of things that happen, gradually associating them into explanations of the world. And with better explanations comes more human agency — affecting the universe in ways that improve the conditions for human life, encouraging flourishing and reducing risk.
Unlike our ancestors thousands of years ago, we now store and transmit most of our information in electricity — ‘data’ — rather than clay, rituals, or paper. And that’s because Claude Shannon, a twentieth century mathematician and juggler, showed how technology that uses electricity — telephones, computers and wires— could be used to keep and send information in great quantities with almost no decay. The world of almost flawless information communication that Shannon showed was possible in his 1948 paper, A Mathematical Theory of Communication, is the one that we now live in.
But being able to store and transmit information almost perfectly does not answer the question of how to use it. Shannon conceptualised communication as an engineering problem that could be solved through focusing on syntactic information — held in symbols like letters and numbers — and not about how semantic information — meaning — motivates action among whoever receives it.
Meaning from more?
In 1949, Warren Weaver, a mathematician, helped Shannon to write a book-length explanation of his 1948 paper in an attempt to popularise its ideas and clarify what it was really about. Over the first twenty pages or so Weaver distinguished three levels of information problems:
- Level one, the technical problem: how to accurately communicate information symbols.
- Level two, the semantic problem: whether the communicated symbols convey the desired meaning.
- Level three, the effectiveness problem: how effectively the meaning received from the communicated symbols affects the behaviour of the receiver.
Level one was solved by Shannon and led to enormous amounts of data now being created, collected, stored, and transmitted. Since the early 2000s, digital formats have been used to store the majority of the world’s information, and according to the International Monetary Fund up to two billion people may now be using cloud computing to store digital information.
The level two and level three problems are not subject to solutions that are easily transferable across time and place, unlike with level one. Transmitting an information symbol just means being able to accurately represent it to the receiver, and these days that tends to mean it being displayed on a screen. If I draw a squiggle on my computer and it is shown on yours, you will know that it is something that I drew but no more than that.
The level two problem of creating meaning starts to be solved by the sender and the receiver sharing a common understanding of how the symbols transmitted should be interpreted. While learning a language you’re being taught the same perception of the letters, words, and cultural content as its other speakers. And without that interpretative code, there is no point in using the language to communicate.
But having a common system for interpretation doesn’t fully solve the level two problem, as the depth of information held in symbols can be unstable or unclear. You’re likely an English-speaker as you’re reading this, so we share a common understanding of the words I’m using. But if I write here that ‘Making money from stocks and shares is easy and I don’t know why more people don’t do it’, you’ll probably want to ask me about what I mean. And that requires organising an extra communication channel with me — perhaps a single phone call, but if the context for the information changes or one of us starts to think about it differently, maybe we’ll need regular ones. Perhaps other people should be invited, too.
Organising for meaning
The question of how to create meaning — semantic and effective information — from data is the problem left to us by Claude Shannon, and the answer to it is in organising human beings around it. But there’s mounting evidence that doing so is a slog, and might be becoming tougher because Shannon’s solution has led to the creation of so much syntactic information that is difficult to search through and use.
In a survey in 2019 of executives at large US companies by New Vantage Partners, a management consultancy, 77 percent said that it was a significant challenge for them to make data and advanced computation — the process of selecting meaningful signals from the noise — a part of their daily operations, and only a third thought that they were running ‘data-driven’ organisations. This is despite the benefits available: the economist Erik Brynjolfsson and his co-authors found that using data analytics can give large, publicly traded American corporations a 5–6 percent uplift in their productivity.
Are Ideas Getting Harder to Find? a recent academic paper, discovered that annual research productivity in the US economy has declined by five percent on average since the 1930s, and that it now takes 18 times more effort to produce the improvements in the computer chip predicted by Moore’s Law than it did in the 1970s. The paper argues that this is because researchers now have more information to search through before they discover something interesting, and that knowledge domains have become so specialised that only experts can work on them. Another academic paper, Declining business dynamism among our best opportunities: the role of the burden of knowledge, seems to confirm this: founders of new US firms now spend more time on research and development than than they used to; and big firms that can create specialist teams to work on discrete knowledge discovery problems are more innovative. New evidence on China and Germany suggests that the problem might be global.
And what about the public sector? It’s much harder to measure how the organisation of civil servants around data affects their productivity with it, but it would be surprising if things were much better than in the private sector. There’s almost no guidance on how to do it, for a start. A recent report from the OECD tells governments that they need to make decisions with data, but makes no headway in showing them how. And that’s because there has been almost no research into how public sector organisations should assign control rights over data — who gets to decide what information is collected, how it is stored, and where it is used — and what the effects are. We’ve got many case studies in Britain and elsewhere on public bodies doing well with data; and years of work on data standards, data sharing, data stewardship, data privacy, and the like, but nothing on how giving one person more power than another over data affects their incentives for doing useful things with it.
Meaning from machines?
Advanced computation — machine learning and the like — has the potential to make firms and public bodies more efficient, and help researchers to create new products and services by searching through information in new ways. But it will also make the problems of organising around information, harder. A forthcoming paper from Nan Jia of the University of Southern California, and other authors, suggests that managers with emotional awareness and relationship-building skills are better at using artificial intelligence, perhaps because they help their staff cope with the uncertainty involved. Two academics at MIT argue that advanced computing is becoming less of a general purpose technology, meaning that companies are now faced with training their staff on new platforms, or even making them in-house. And Gina Neff of the Oxford Internet Institute, and co-authors, have shown that the labour of clerical staff in hospitals in Denmark and the US is as important to the chain of creating meaning from data, as anyone else.
It’s going to take years of investment and testing to solve these organisational problems in ways that are simple enough for lots of firms to use. Training staff, understanding new technology, and trialling new management processes is expensive and often only improves through error. But to raise productivity across the economy and boost growth, good practices need to spread from the firms at the technology frontier to those doing more mundane things. And that could be a long grind.
