Where AI wins.
What AI systems do better than humans.
Hello friends,
tl;dr: AI excels at managing large quantities of explicated data, doing complicated data analysis, and following well-defined rules — but only if it is designed to highlight edge cases and potential patterns for human interpretation and meaning-making. The most valuable AI workflows to build will be the ones where AI systems and humans work interdependently on what they respectively do well.Caveat: This is thinking-in-progress, not a polished thought-product. Send me your comments (reply to this email or find my contact information here). The second time I used a Waymo self-driving car was last week, because I’m secretly a techno-optimist under my hardened techno-cynic exterior. It’s exciting to imagine AI (built by Google or Tesla or whoever) freeing humans from the drudgery of driving and sitting in traffic! But it is also discomforting to think of an AI system already set loose to navigate unpredictable driving environments — especially people in cars, on streets, on bicycles — and potentially displace professional drivers. (The Waymo ride to Good Machine was completely uneventful.)
I have similar ambivalent #vibes more generally amid the excitement about AI, so I’ve been thinking about what it means to do good AI strategy. AI strategy = choosing what kinds of AI models and AI products to invest in, and how to write AI policy and regulation. Good AI strategy makes investments in AI more likely to be productive, safe, aligned, and to support human flourishing.
(Also, I’m working a lot on strategies for not-knowing, and AI already looks like a major injection of different kinds of not-knowing into lots of Important Areas for Strategic Thinking, from geopolitics, to how capital is allocated, to how work is designed, to how organizations are structured.)
To have good AI strategy, we need to make reasoned decisions about
What humans can do which AI systems cannot — these are tasks which should be done by humans.
What AI systems can do much better than humans — these are tasks which AI systems should be built to do.
I answered the first question a few weeks ago by proposing that one thing humans can do which AI systems can’t do at all (yet) is meaning-making: Deciding what things mean by assigning inherently subjective relative and absolute value to them. I also gave some concrete examples which show that though AI systems can’t make meaning, human meaning-making work is essential for making and using AI systems. That essay addresses why machines that seem able to mimic human behavior indistinguishably — e.g., in this recent study that’s been getting a lot of attention — should not be thought of as being able to make meaning the way humans do. (Meaning-making as I define it here is usually folded into sensemaking; in the essay, I also explain why we should call out meaning-making specifically.)
This time, I want to focus on the second question: “What types of tasks can AI systems do much better than humans?” The answer seems obvious on the surface: AI systems are generally better than humans in 1) data management (storage, search, and retrieval), 2) data analysis (pattern-finding), and 3) rule-following.
Poking at the hidden assumptions in each of these three areas shows why we need to have a more nuanced set of principles about the conditions under which tasks in these three areas are suitable for AI systems to do in place of humans. A good entry point to this is human failings.
Human failings and AI capabilities
Humans have highly constrained storage and processing capacity. We are also unpredictably biased, lazy, and inconsistent, depending on anything from the weather, to our mood, to recently announced news.
Even with training, individual humans can store only tiny amounts of data in their brains before they become unreliable at storage (“Did I read that paper already?”), searching (“I totally forgot that paper which turned out to be very relevant”), and retrieval (“I swear I’ve read a paper about this exact thing but I can’t remember how to find it”). Our limited processing capacity prevents us from doing large-scale, high-dimensional, complicated analysis without machine assistance, while bias often leads us to find patterns which aren’t there (or ignore patterns that are there). And our biases, laziness, and inconsistency make us unreliable followers of rules.
Our failings reveal where AI systems might outperform us.
Data management
AI systems are potentially way better than humans at storing, searching for, and retrieving data.
“AI systems” is a broad term. The top-of-mind Large Language Models (LLMs), which so far are notoriously bad at the kind of search/retrieval that we are benchmarking them against — but they are only one of many possible types of AI system. Machine learning is another aspect of AI development, and data management systems driven by ML developments over the last 20 years have led to huge improvements in data management. Google’s consumer search product is the best-known of these and it made the corpus of internet unstructured data searchable (and making other types of data like scholarly publications, patents, and geographic data more searchable too).
Stuff like Google Search is a type of AI system, and they are the current iteration in a long sequence of attempts to use the best available technology to improve our individual storage and retrieval capacity: Digital storage becomes ever more reliable, storage prices continue to decline, and search algorithms seem to become ever more effective and efficient.
But the correct conclusion to draw is not that AI systems are better than humans at all data management tasks involving a lot of data.
Why? Because any system that doesn’t rely on humans as the storage medium can only store and search for and retrieve content that has been explicated — i.e., has been converted into a string of symbols (like words, numbers, or bits). Not all content can be converted into a string of symbols with present understanding and technology, and some may not be explicatable at all.1
The typeset text in a hardcopy book can be scanned, OCRed, and accurately converted to a string of symbols (alphanumeric characters) that can easily be searched. But if the handwritten marginalia in the same book are faint and hard-to-decipher, they may not be recognizable by existing OCR technology. The scanned book pages may be stored but the marginalia can’t be reliably searched until they can be accurately converted to searchable strings of symbols. If you have a bunch of books that are important partly because of the marginalia in them, maybe it’s not a good idea to digitize them and rely solely on an AI system to manage those digitized books.
AI systems have a clear advantage over humans for data management, but only for data that is explicatable and already explicated.
AI systems become relatively better than humans for data management (storage, search, and retrieval) the larger the dataset size (amount of data stored) and complicatedness (number of types of data stored) — but only where the data to be managed is both explicatable and already explicated.
Where important parts of the content to be managed are still tacit and not yet explicated, using AI systems for data management leads to lost information — information that is valuable for analysis which hasn’t been captured digitally or is unfindable and unretrievable.
Data analysis
A lot of data analysis consists of looking for meaningful patterns in a dataset by breaking the data down before processing the constituent parts to understand patterns of association — the usual intention is to identify causal mechanisms. “Pattern” is a broad term here, covering everything from lifestyle habits associated with diseases, to corporate actions associated with concealed fraud, to structural characteristics of molecules associated with how they register on insect sensoria. Examples of pattern-seeking data analysis include regression analysis, mathematical optimization, agent-based modeling, and neural networks.
AI systems can be built which break down and process more data faster, in more different and more complex ways than humans can — by running operations in parallel and/or in series, with interoperating layers of analytic process, or by using complex simulations and modeling. In combination, this could let AI systems analyze datasets that are too large for humans to feasibly work on, and detect patterns of association that are either too weak or too obscured by noise for humans to detect unaided.
Again, the instinctive leap — that AI systems are better than humans for large-scale, complicated data analysis — isn’t the right one. This is because AI systems cannot yet decide on their own what patterns are meaningful ones.
AI systems must be instructed that particular patterns are meaningful (e.g., when an automated AI medical diagnostic system is designed flag patients for treatment when images of their tissue samples show experimentally supported cellular markers of a particular disease), or they must be designed to detect potential patterns and pass them on to humans for interpretation as being meaningful (e.g., when an AI medical research support system is designed to identify features in images of stained tissue from patients that seem associated with particular disease diagnoses, and to highlight those features to researchers for further investigation).
AI systems become relatively better than humans for data analysis as datasets get larger and more complicated, and as patterns in the datasets become weaker or noisier — but only where a) patternings are pre-established to be meaningful by humans and/or b) the analyses are designed to highlight potentially meaningful patterns for human meaning-making.
If AI systems for data analysis don’t work to patterns previously established to be meaningful by humans, the results of their analysis are potentially and unpredictably flawed.2
Rule-following
AI systems are potentially much better rule-followers than humans, because we can design machines to be more hardworking, conscientious, and consistent than humans.
“Rules” here means generally instructions for actions to be taken. Examples include extracting to a summary document all instances of particular phrases in a corpus of digital/hardcopy documents and audio/video recordings obtained through legal discovery, or navigating a vehicle from between 2 points while avoiding collisions and obeying a predetermined set of traffic rules. An AI system can be designed to work as much and whenever its designers want, and to comply precisely and consistently with the rules it is instructed to follow, even if those rules are extremely complicated.
AI systems are limited mainly by our ability to provide them with good rules to follow. Because AI systems can’t make meaning yet, they only have unambiguous advantage over humans when the rules they follow don’t require any additional meaning-making to be followed precisely — i.e., when they are specifically and intentionally designed to not require discretion to be properly followed, or when their interpretation evolves with changing conditions.
Discretion in rule-following can only be avoided when a lot of meaning-making has been done in creating the rule to be followed: When the conditions of applicability for a rule are both fully specifiable and fully specified (i.e., all edge-cases are accounted for, and there are no unanticipated cases), and the actions the rule calls for are also fully specified. In real life, it is probably rare for any rule to have fully specifiable and fully specified conditions of applicability, so AI systems should explicitly default to surfacing unanticipated cases for human meaning-making.
AI systems become relatively better than humans at rule-following as the rules become more complicated and/or tedious, and the more fully specifiable and fully specified the rules — but only if they are designed to explicitly surface unanticipated edge-cases for human meaning-making.
We rely on (ideally well-trained human) discretion when we don’t know yet how to write fully specified rules. This can happen when we’re doing innovation work (i.e., learning how to do something new, where neither the process nor the definition of a successful outcome are known in advance) and when we confront unpredictably changing environments.
If rule-following AI systems are deployed without expert human meaning-making support for innovation work, they’re unlikely to come up with real innovations or new ways of doing things that are effective — maybe not such a big deal. The bigger problem is that rule-following AI systems deployed in highly uncertain environments without human meaning-making support are likely to misapply rules — and the consequences of doing that in high-stakes, high-uncertainty environments like warfare, justice, or health are usually significant.
These three propositions about where AI wins are summarized in this diagram:
Good AI strategy is a social and technical question
The deeper insight from these three propositions is that good AI strategy depends on understanding where and how AI systems are necessarily embedded in human meaning-making work, whether this is diagnosing which parts of a medical dataset to be managed by an AI system are explicable, or deciding whether traffic rules are well-defined enough to be followed reliably by an AI system without human support. In other words, AI systems must be understood as sociotechnical systems.
In turn, this implies that humans and human organizations need different training and capabilities to become sophisticated users and builders of AI systems. As more tasks are taken over by AI systems, workers will have to learn how to be better (more mindful, more sophisticated) at making explicit subjective judgments of value and defending them with evidence and reasoning.
But the harder meta-challenge of AI strategy is for leaders, managers, policymakers, and technologists, who must be able to make and defend subjective diagnoses about whether specific tasks depend on human meaning-making. They will also have to become proficient at decomposing complex work into tasks that AI systems can do better than humans and high-value meaning-making tasks that must be left to humans. The most valuable (and safest and most aligned) workflows to build will be the ones where AI systems and humans work interdependently but doing fundamentally different things which they respectively do well — and these will also be the hardest ones to build because they require rethinking how work is designed, how humans interact with software and machine systems, and retooling our organizations accordingly.
The barrier to more widespread, useful, safe, and aligned adoption of AI is not just whether the right technology exists — it is also whether the humans in the loop can learn how to build and use them as force multipliers for human meaning-making work.
See you next time,
VT
Harry Collins’s Tacit and Explicit Knowledge gives a useful overview of a framework for thinking about what kinds of knowledge are explicatable (convertable to symbolic strings) and what kinds of knowledge aren’t.
One mechanism by which this happens is spurious associations which seem, on principle to become more likely as model parameter size continues to increase — LLM hallucinations are potentially an example of this.
I liked your framing of AI systems being used in contexts where the meaning-making has already been done by humans.
For data management would that mean these are equivalent - "...but only where the data to be managed is both explicatable and already explicated." to "where the human meaning-making of the data has already been done"? For eg. in your example, the marginalia has been typed up separately.