PART ONE
1
When Human Intelligence Meets Artificial Intelligence
Late one afternoon, UCLA professor Burt Swanson was about to leave his office for home when an email caught his attention. Subject: “Interested in Meeting You.” It was from a professor named Todd who worked at a university across the country. Todd worked in a similar area to Swanson and wanted to meet up while he was visiting Los Angeles. He closed: “I copied my assistant, Amy, who can help with scheduling. Please reply all with time(s) and location(s) that are convenient for you if you are interested.”
Swanson provided Todd’s assistant with several dates and times as options. By the time he arrived home, Amy had written back. Todd was not available for any of the times Swanson had indicated. She asked that he propose new times. Swanson did. By early the next morning, Amy had confirmed a meeting. Several hours later, though, Amy wrote again, saying that Todd was no longer available at that time, and she suggested several other times. Swanson felt annoyed that Todd was making so many changes, especially since it was Swanson who was going out of his way to fulfill Todd’s request. Still, he picked a time. At the end of his email, he politely wrote that he would appreciate if they could stick to this newly agreed upon time. Much to his surprise, Amy responded immediately that the time Swanson selected was no longer available. She suggested more proposed times. Thoroughly frustrated, Swanson typed out a lengthy response to Amy expressing how unpleasant the experience of trying to accommodate Todd had been and that he was not available at other times. Amy never wrote back.
A few weeks later, Swanson was surprised to learn that Amy was not a person. She was an AI scheduling agent created by a company called x.ai. The product is used by companies around the world including Disney, Coca-Cola, and Nike. The easy conclusion to draw from this is that Swanson’s scheduling fiasco was caused by a poorly functioning AI. But it’s the wrong conclusion. Scenarios like the one described above are common when people begin to enter into relationships, however brief, with AI, bots, and machine learning algorithms. The problem isn’t the AI’s capability; it’s the lack of experience we have interacting with such machines. Because they mimic the functionality of humans, people tend to treat them like humans. Developing a digital mindset means overcoming that understandable error and knowing how to treat AI agents on their own terms as computers, even if they are programmed to present human-like characteristics.
New Rules of Interaction
Computational and machine learning algorithms perform an ever-increasing number of activities within organizations. Among them: They have fundamentally shifted the nature of Wall Street trading.1 They determine credit scores for existing and potential customers. They are used to screen applicants and assist in hiring. They enable chatbots to respond in real time to queries and suggest new courses of action for people with computer trouble, for those looking for new loans, and for workers who hope to find new information in their jobs.
The rapid scaling of computational power means that digital technologies have migrated from tools that people use to platforms upon which they interact.2 Now they’re beginning to migrate again, to agents with which people actively collaborate—like Amy the scheduler. If this change conjures images of working side by side with a robot on your team, you’re actually not far off. We already have integrated robots into many aspects of our lives. Think of the robot you interact with when you call for an airline reservation, the chatbot that helps you open a new bank account, the physical robots (controlled by digital automation tools) that workers on a high-tech product assembly line interact with by giving commands and receiving feedback. Each of these digitally controlled bots uses AI and machine learning techniques to evaluate data inputs, make suggestions for behaviors and actions, and learn from your responses in order to improve its performance in the next interaction.3
One important feature of a digital mindset is to understand that the keys to working successfully with machines are not the same as the skills needed to work successfully with humans. You might think that’s an obvious point. But in countless experiments, subjects who know that they’re interacting with machines instead of people overwhelmingly tend to treat the machines as if they are people.4 As we’ll elaborate shortly, that behavior causes problems for how we approach individual tasks and get work done. Developing a digital mindset means recognizing that the rules of interaction are not the same when you’re working with machines and that when you deploy machines to work for you and interact with friends, family, or customers on your behalf, all of those people are likely to treat the machines as people too.
In this chapter, we draw on our work with more than eight hundred people across multiple companies who have begun to interact regularly with robots, chatbots, and other AI-powered autonomous agents. We discuss how to effectively build relations with these digital technologies when they interact with us verbally and when they become members of our teams. The chapter is designed to help you build the skills to work effectively with machines by treating them as machines.
But before we can get there, you need to understand what artificial intelligence actually is and how AI agents “think.” You don’t need to know how to build your own AI, but you do need to know:
· What machine learning is
· How AI train on certain data sets
· How models are built from those data, and how the machine uses prediction techniques to determine how best to interact with you
Along the way we’ll define machine learning, neural networks, natural language processing, and computer vision. These are all building blocks for developing a digital mindset that can interact with machines as machines. You may be surprised at how simple some of this technology is on its surface, even if how it goes about doing that work is marvelously complex.
How artificial intelligence works
If you’re like most people, you used to think that artificial intelligence was far-fetched science fiction—maybe you think of movies like Star Wars, The Terminator, Minority Report, or Interstellar that depict autonomous computers or robots doing things that humans can’t totally explain or control. John McCarthy, who coined the term artificial intelligence in 1956, lamented that “as soon as it works, no one calls it AI anymore.”5 In other words, we tend to think of AI as something futuristic that hasn’t quite happened yet. But that’s not true. If you speak to Siri or are driving a car that lets you know when you are drifting out of lane, you are using AI. AI is found in many applications. We use AI in our daily lives, even if we don’t realize it.
The type of AI we use today focuses on one specific task. Think of the AI applications that routinely beat human chess champions. That specific AI is very good at playing chess and nothing else. Alibaba, one of the world’s largest e-commerce platforms, like Amazon uses AI to predict what customers might want to buy. We don’t yet have AI that can, for example, like Rosey in The Jetsons, discuss detailed decisions about housecleaning. There’s no AI to help solve a murder mystery the way Sonny does in the movie iRobot. In other words, for all its dazzle, AI has not yet achieved high enough levels of intelligence to converse and solve problems across a range of topics or issues. We also don’t have the type of AI in which computer-programmed robots take over the world and rule human beings. AI thinker Nick Bostrom defines this superintelligence as “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.”6 Not only would AI in this capacity need to surpass humans in every way, but it would likely also be capable of having emotions and relationships.
Don’t worry about any of that. Our focus is on single-task AI—what’s real now—and learning how to treat machines like machines. That requires a mindset shift. One of the reasons we may have difficulty treating machines as machines is that AI is defined as a machine displaying a kind of intelligence akin to that of humans. Machines mimic the “cognitive” functions of people by executing algorithms.7 Therefore, AI is a machine that perceives its environment and takes actions that maximize its chance of successfully achieving its goals. Today, those goals are programmed by humans.
It’s also important to remember that robots are not AI, though we often hear these two conflated. Robots are simply containers for AI: the AI is what is inside the robot making it run. For example, the software, data, and algorithms running behind Alexa make up the AI while the voice that speaks to us is just the personification of that AI. Similarly, the figures of steel that perform many industrial tasks, from building cars to packing boxes, are the robots—collections of metal and actuators and electrical circuits. AI is the software programmed to make that pile of stuff act like an arm and tighten a screw or pick up a box.
The AI ecosystem broadly encompasses data, tools, and statistical models.
The statistical models process large-volume data sets. Before processing, the data must be “cleaned” and converted into formats the algorithms understand. Cleaning involves fixing or removing incorrect, incomplete, or duplicate data. Data aren’t always collected in perfectly consistent pristine ways, and combining multiple data sources often results in duplications, incongruencies, and mislabeling. Say you’re combining different sources of demographic data and one includes “Population Data” and the other includes “PopDat.” They’re the same, but the computer doesn’t know it. You have to clean it to make it so the computer knows to treat them as the same group.
Today we see AI all around us. Here are a few examples of AI doing very specific activities that equal or exceed a human’s ability to do them, though the AI can do them more quickly and efficiently:8
· Cars are full of AI systems, from the computer that figures out when the anti-lock brakes should kick in to the computer that tunes the parameters of the fuel injection systems. Self-driving cars will contain robust AI systems that allow them to perceive and react to the world around them.
· Smartphones are practically an AI factory. When you navigate using your map application, receive customized music recommendations, check tomorrow’s weather, ask your phone a question, or perform dozens of other everyday activities, you’re using AI.
· Your email spam filter is a classic AI. It starts off loaded with intelligence about how to figure out what’s spam (“Congratulations, you’ve won $1,000,000”) and what’s not (“update on Thanksgiving plans”), and then it learns and tailors its intelligence to you as it gets experience with your particular preferences.
· Controllers like thermostats can use AI. For example, the Nest Thermostat adapts as it starts to figure out your typical routine and adjusts your house’s climate accordingly.
· Google Translate is impressively good at one narrow AI task. Voice recognition is another. Some apps mash those together, allowing you to speak a sentence in one language and have the phone spit out the same sentence in another.
· When your plane lands, it’s not a human that decides which gate it should go to. Just like it’s not a human that determined the price of your ticket.
· Google Search is one large AI brain with incredibly sophisticated methods for ranking pages and figuring out what to show you in particular. Same goes for Facebook’s Newsfeed.
And those are just consumer-facing examples. Sophisticated AI systems are more widely used in sectors and industries like military, manufacturing, and finance (algorithmic high-frequency AI traders account for more than half of equity shares traded on US markets), and in expert systems that, for example, help doctors make diagnoses.9
How Do Machines Learn?
Let’s start by understanding how AI really works. Key building blocks of AI are machine learning techniques, which are algorithms that derive predictions from data using statistics. While humans use complex natural languages and visual cues, computers work with numbers to generalize from examples and gain the ability to “learn” without being explicitly programmed.
Machine learning expert Marily Nika loves to explain machine learning by showing how it learns the difference between cats and dogs.10 First we label pictures “cat” or “dog.” Then we feed the algorithm the labeled pictures. (See figure 1-1.)
The machine reads the patterns of pixels in each labeled picture and stores it as an example of the label: this pattern equals cat, and that pattern equals dog. Of course, not all patterns of pixels that make up a cat in a picture are going to be the same. Maybe the cat is head-on in one picture and side-viewed in another. So the computer needs a lot of pictures of cats and dogs to store many patterns and get good at identifying a cat versus a dog.
Nika adds that we can correct the computer when it incorrectly says, “This is a cat.” The computer makes a note of its mistake. It has learned not to include that pattern as a “cat.” That ability to adapt is why we use the term “intelligence” to describe this learning process.
FIGURE 1-1
Teaching a model how to identify dogs and cats
Source: Marily Nika, “An Intro to AI/ML and Deep Learning,” https://marilynika.medium.com/an-intro-to-ai-ml-and-deep-learning-ffd2f2fbf1e.
The mathematics to do this have been around for a long time, but it took the availability of vast amounts of data and much higher computer processing power to make the math useful in the real world.11
Let’s look at how machine learning works through an example provided by Mark Robins, head of corporate AI strategy at Intel, whose face you can see depicted in figure 1-2.12 In a typical machine learning approach, some person identifies a set of features that uniquely represent one person’s face, like Mark’s. The distance between the eyes, the nose width, and eye socket depth are common examples. A machine learning algorithm takes these features and builds classification systems of them using various algorithms based on different kinds of statistical models. By repeating this process multiple times and being corrected (at least at first) by humans who know the face, the machine learning algorithm learns to associate a given pattern of features with a particular person.
As Robins observes, and many other experts have also noted, the difficulty with this approach is that it is not always obvious what features are most useful for determining one particular face.13 And even if we know that a feature is important, it may be hard to compute it. For example, in order to compute the distance between the eyes, you need to first be able to find the eyes in the image and calculate the distance based on how far the face is from the camera, which in and of itself can be complicated. Enter deep learning.
FIGURE 1-2
Classic machine learning vs. deep learning
Source: “The Difference Between Artificial Intelligence, Machine Learning, and Deep Learning,” Intel, https://www.intel.com/content/www/us/en/artificial-intelligence/posts/difference-between-ai-machine-learning-deep-learning.html.
Deep learning is a type of machine learning in which the algorithm doesn’t need to be told about the important features by a person. Instead, it is able to discover features on its own by using a neural network to examine the data themselves. The term comes from a mathematical object called an artificial neuron that “fires” if inputs exceed some threshold, just like a neuron in the brain fires. Artificial neurons can be arranged in layers, and deep learning has many layers of artificial neurons. Deep learning requires millions of parameters, which is why it has only become powerful recently, now that we have enough data for it to learn from and the processing power to do the very complex math it has to do in a reasonable amount of time.
FIGURE 1-3
Machine learning vs. deep learning
Source: “The Difference Between Artificial Intelligence, Machine Learning, and Deep Learning,” Intel, https://www.intel.com/content/www/us/en/artificial-intelligence/posts/difference-between-ai-machine-learning-deep-learning.html.
In the context of facial recognition, deep learning avoids having to try to relate various shapes in an image to prespecified features. Feed it enough “labeled data” (that is, images of known faces) and give it the right training, and a deep learning model will decide what the most relevant features are from the data on its own. This process dramatically improves the accuracy of the algorithm.14 When used with images deep learning is called computer vision.15 (See figure 1-3.)
Machine learning algorithms can also be applied to written text or spoken word to identify patterns and make correlations in much the same way. Natural language processing (NLP) is to spoken text what computer vision is to images.16 NLP works by following a script that classifies certain language into categories. It’s a similar winnowing through statistics. First, phrases are labeled based on some classification scheme. Then, the computer cleans the phrases, getting rid of stop words (“a,” “and,” “the,” “but,” “or,” and so forth) and punctuation. Then words are “lemmatized” (a technical way of saying that they are sorted by grouping variants of the same word together, such as “stopping,” “stopped,” and “stopper”). They are then stemmed, which simply means that the words are reduced to their word stem (“stop”). Finally, the cleaned data are “vectorized,” or turned into numbers that can be analyzed statistically.
Yelp combines such techniques and uses machine learning to help its staff compile, label, and categorize images and descriptions of restaurants more efficiently.17 Labels in broad categories like “menu,” “food,” “tacos,” and “sushi” are attached to images based on comments supplied by the users who uploaded the images. Additional validation of these extracted labels is done through crowdsourcing. Then the images and labels are used to train a type of deep learning neural network that does computer vision. Once it has learned what makes a menu, or food, or tacos, or sushi, the neural network will itself classify and label previously unlabeled images in a fraction of the time it would take an army of humans to do the same task. With an increase in the number of labeled images, Yelp can offer additional features, like tabbed photo browsing based on categories like “food,” “menus,” and “drink.”
As we discussed in the introduction, algorithms are sequences of instructions that can be used to solve a problem. Algorithms developed by programmers to instruct computers to do certain tasks are building blocks for our digital world. But developing these algorithms takes time. The advantage of using neural networks and similar techniques is that they can create new algorithms themselves. They recognize the rules guiding the patterns and are then programmed to generate new rules based on what they learn, and again, in a fraction of the time it would take humans to do the same.
So, for a machine learning algorithm to generate new rules, it needs to first be trained (usually by a human) to identify patterns and extract rules from those patterns. Applying labels to phrases is a good example of the way that a programmer would train a machine learning algorithm. This kind of training is often referred to as supervised learning. A phrase is labeled, say, positive, negative, or neutral. The algorithm can now identify the rules to classify a phrase as good or bad. It might decide a rule is that phrases with “worst” in them are negative, for example (though it’s more complicated than this; we are keeping this simple for explanation’s sake).
Once an algorithm has been trained through supervised learning it is ready for a more advanced phase, called unsupervised learning. This is where the algorithm automatically sorts through the Yelp pictures and reviews and classifies certain restaurants as good or bad on its own. Unsupervised learning is how Alibaba and Amazon figure out that two items are often bought together.
A third and even more advanced phase of learning is called reinforcement learning, when the machine learning algorithm is constantly corrected based on feedback. For an example of reinforcement learning, consider a few everyday examples that you’ve likely experienced. If a machine learning algorithm infers that sweatshirts and socks are frequently bought together, it serves an advertisement for socks to the buyer of a sweatshirt. But if that buyer doesn’t buy the socks, the algorithm takes that feedback and uses it to refine its model about who is likely to buy what. Similarly, if the driver of a self-driving car takes the wheel to make a correction, the algorithm learns that its model of the vehicle’s trajectory was off and takes this correction data into account in reconstituting the model.18
Developing a digital mindset also means accepting that, in many cases, machines are better than humans at making certain predictions and doing specific tasks. Advances in computational power paired with massive amounts of data generated in health-care systems make many clinical problems perfect candidates for AI applications. Researchers at Seoul National University Hospital and College of Medicine developed an AI algorithm called DLAD (Deep Learning–based Automatic Detection) to analyze chest radiographs and detect abnormal cell growth, such as potential cancers.19 In a four-year study, the hospital found that the AI was able to dramatically reduce the number of overlooked lung cancers on chest radiographs without a proportional increase in the number of follow-up chest CT examinations. For physicians and their patients, this technological breakthrough means adjusting expectations for diagnosis and practice. In another example, Google Health created a machine learning algorithm, LYNA (LYmph Node Assistant), to identify metastatic breast cancer tumors from lymph node biopsies.20 What was unique about this AI was that the algorithm could identify suspicious regions undistinguishable to the human eye. LYNA was tested on two data sets and was shown to classify a sample as cancerous or noncancerous correctly 99 percent of the time. When given to doctors to use in conjunction with their typical manual analysis of stained tissue samples, LYNA halved the average slide review time. Again, doctors had to integrate this new level into their practice. Although ultimately an advance to be celebrated—more accurate diagnostics in less time—workflows and human roles then need to be adjusted, and that’s not always easy. In health care, for example, it can feel threatening to have a machine contradict your diagnosis.21 That’s where the idea that machines aren’t human is important. It feels threatening, but code doesn’t make threats. It’s just a tool for us to harness.
What it really means when you hear that AI is getting more powerful (and when movies caution us that machines are about to take over our lives) is that more and more data are becoming available and we are acquiring more and more computer processing power. This also means that computer scientists are able to take advantage of more powerful processing to design better algorithms to train AI. Think of all the digital transactions conducted online daily, or the number of sensors being deployed into all the devices you own (today’s cars have more than one hundred sensors that monitor functions such as fuel level and tire pressure). Although those numbers have grown rapidly in the past decade, the march to even more powerful processing and more data is continuing unabated. Every year we produce more data than all the previous years combined.22 What’s state of the art today will soon be considered slow. Companies like IBM are working to develop quantum computing systems to power their Watson AI.23 If nothing else, you need to know that the future progress of AI will depend on a combination of data, computing power, and more advanced algorithms.
Get to Know Your Technology Stack
Have you played the game Jenga? Rectangular blocks are cross-stacked in a number of consecutive layers to create a tower of interdependent layers. You can extract some pieces at each layer without knocking over the tower, but if you weaken those lower layers too much the whole structure will come crumbling down.
AI is a lot like a Jenga tower. AI systems are built out of an increasingly complex and interdependent web of software, hardware, and databases. For example, to do something as seemingly simple as showing a webpage about your favorite musical artist when you ask Alexa to do it requires a mix of technologies that do everything from storing, accessing, and securing those data, to computing them and displaying them to the page’s viewer. All this requires a Jenga-type level of interdependence and cross-stacking with multiple layers.
In the world of IT, these layers are called a technology stack, which is simply all the hardware and software systems needed to develop and run a single application. Software developers can use a preconfigured technology stack (think of a Jenga set with all the pieces already included and set up for you) to develop a new application, or they can build their own technology stack (by finding and assembling their own blocks for the game).
Typically, a technology stack is built as two subsystems. The first is the “front-end” system or the “client side,” and comprises all of the technologies and data sources that users (you, your employees, your customers) will interact with. This might include the buttons you click or the forms you fill on a computer survey. The second system is the “back-end” system or the “server side,” which is the underlying infrastructure that powers the client experience. Technicians and engineers are concerned with the back end.
Linking these layers of the stack together are software applications known as middleware. Middleware operates as a hidden translation layer, almost like a house’s plumbing connecting the sink and the tub and other appliances into the same system; middleware enables communication between database and data. For example, when you submit a form on a web browser, the middleware retrieves content for the web page you’ll see based on the information you submitted. Many software developers describe middleware as “the software glue” that holds the various layers of the stack together—so that the Jenga structure won’t easily topple over. Figure 1-4 provides a visualization of a generic technology stack.
FIGURE 1-4
Generic technology stack
It’s important to know about a technology stack so that you have an appreciation for the many, many technologies required to make even simple things happen with digital data. Also, it should make you realize that if you decide you need new data, or a new kind of analysis, you’re going to have to do much more than simply ask a software developer to write a script for you. What might seem like a simple change can have dramatic implications for the lowest levels of the technology stack, like trying to swap out one of those blocks at the bottom of the Jenga tower. Certain applications in the stack favor certain programming languages and certain servers are optimized for particular kinds of data categorization, storage, and retrieval activities (which we’ll discuss in chapter 3). If you want to change anything high up in your technology stack, you may be limited by the arrangement of applications lower down. These layers are interdependent. Knowing that can help you to ask the right questions to assess whether a change that seems small is actually small or not.
Treat AI Like a Machine, Even If It Seems to Act Like a Human
We are accustomed to interacting with a computer in a visual way: buttons, drop-down lists, sliders, and other features allow us to give the computer commands. However, advances in AI are moving our interaction with digital tools to more natural-feeling and human-like interactions. What’s called a conversational user interface (UI) gives people the ability to act with digital tools through writing or talking that’s much more the way we interact with other people, like Burt Swanson’s “conversation” with Amy the assistant.24 When you say, “Hey Siri,” “Hello Alexa,” and “OK Google,” that’s a conversational UI.
The growth of tools controlled by conversational UIs is staggering. Every time you call an 800 number and are asked to spell your name, answer “Yes,” or say the last four numbers of your social security number you are interacting with an AI that uses conversational UI. Conversational bots have become ubiquitous in part because they make good business sense, and in part because they allow us to access services more efficiently and more conveniently.
For example, if you’ve booked a train trip through Amtrak, you’ve probably interacted with an AI chatbot. Its name is Julie, and it answers more than 5 million questions annually from more than 30 million passengers. You can book rail travel with Julie just by saying where you’re going and when. Julie can pre-fill forms on Amtrak’s scheduling tool and provide guidance through the rest of the booking process. Amtrak has seen an 800 percent return on their investment in Julie. Amtrak saves more than $1 million in customer service expenses each year by using Julie to field low-level, predictable questions.25 Bookings have increased by 25 percent, and bookings done through Julie generate 30 percent more revenue than bookings made through the website, because Julie is good at upselling customers!
One reason for Julie’s success is that Amtrak makes it clear to users that Julie is an AI agent, and they tell you why they’ve decided to use AI rather than connect you directly with a human. That means that people orient to it as a machine, not mistakenly as a human. They don’t expect too much from it, and they tend to ask questions in ways that elicit helpful answers. Amtrak’s decision may sound counterintuitive, since many companies try to pass off their chatbots as real people and it would seem that interacting with a machine as though it were a human should be precisely how to get the best results. A digital mindset requires a shift in how we think about our relationship to machines. Even as they become more human-ish, we need to think about them as machines—requiring explicit instructions and focused on narrow tasks.
x.ai, the company that made meeting scheduler Amy, enables you to schedule a meeting at work, or invite a friend to your kids’ basketball game by simply emailing Amy (or her counterpart, Andrew) with your request as though they were a live personal assistant. Yet Dennis Mortensen, the company’s CEO, observes that more than 90 percent of the inquiries that the company’s help desk receives are related to the fact that people are trying to use natural language with the bots and struggling to get good results. Perhaps that was why scheduling a simple meeting with a new acquaintance became so annoying to Professor Swanson, who kept trying to use colloquialisms and conventions from informal conversation. In addition to the way he talked, he made many perfectly valid assumptions about his interaction with Amy. He assumed Amy could understand his scheduling constraints and that “she” would be able to discern what his preferences were from the context of the conversation. Swanson was informal and casual—the bot doesn’t get that. It doesn’t understand that when asking for another person’s time, especially if they are doing you a favor, it’s not effective to frequently or suddenly change the meeting logistics. It turns out it’s harder than we think to interact casually with an intelligent robot.
Researchers have validated the idea that treating machines like machines works better than trying to be human with them. Stanford professor Clifford Nass and Harvard Business School professor Youngme Moon conducted a series of studies in which people interacted with anthropomorphic computer interfaces.26 (Anthropomorphism, or assigning human attributes to inanimate objects, is a major issue in AI research.) They found that individuals tend to overuse human social categories, applying gender stereotypes to computers and ethnically identifying with computer agents. Their findings also showed that people exhibit overlearned social behaviors such as politeness and reciprocity toward computers. Importantly, people tend to engage in these behaviors—treating robots and other intelligent agents as though they were people—even when they know they are interacting with computers, rather than humans. It seems that our collective impulse to relate with people often creeps into our interaction with machines.27
This problem of mistaking computers for humans is compounded when interacting with artificial agents via conversational UIs. Take for example a study we conducted with two companies who used AI assistants that provided answers to routine business queries. One used an anthropomorphized AI that was human-like. The other wasn’t.
Workers at the company who used the anthropomorphic agent routinely got mad at the agent when the agent did not return useful answers. They routinely said things like, “He sucks!” or “I would expect him to do better” when referring to the results given by the machine. Most importantly, their strategies to improve relations with the machine mirrored strategies they would use with other people in the office. They would ask their question more politely, they would rephrase into different words, or they would try to strategically time their questions for when they thought the agent would be, in one person’s terms, “not so busy.” None of these strategies was particularly successful.
In contrast, workers at the other company reported much greater satisfaction with their experience. They typed in search terms as though it were a computer and spelled things out in great detail to make sure that an AI, who could not “read between the lines” and pick up on nuance, would heed their preferences. The second group routinely remarked at how surprised they were when their queries were returned with useful or even surprising information and they chalked up any problems that arose to typical bugs with a computer.
For the foreseeable future, the data are clear: treating technologies—no matter how human-like or intelligent they appear—like technologies is key to success when interacting with machines. A big part of the problem is they set the expectations for users that they will respond in human-like ways, and they make us assume that they can infer our intentions, when they can do neither. Interacting successfully with a conversational UI requires a digital mindset that understands we are still some ways away from effective human-like interaction with the technology. Recognizing that an AI agent cannot accurately infer your intentions means that it’s important to spell out each step of the process and be clear about what you want to accomplish.
Build Trust to Team with Machines: Human-Agent Teaming
We typically think of technology as something that enables teams of people to interact with each other—think of a team working together via Skype, sharing an Excel spreadsheet, or accessing the same Oracle database. But with advances in machine learning, NLP, and AI, technologies are becoming more like “real” team members themselves that can interact with you, make suggestions, and act on decisions.
The US military is one of the most advanced users of human-agent teaming. They use it to help determine, for example, a convoy’s route through a battle zone, or to make tactical battle plans. Companies are also increasingly putting AI team members into their operations. Google is incorporating smart digital technologies into executive decision-making teams. Foxconn is using smart digital tools on shop-floor manufacturing teams, and banks are using smart digital technologies on underwriting teams.
Our ability to trust the machines in this “human-agent teaming” is a key factor for success working with machine team members, and it’s not always there.28 As much as we tend to humanize machines, we can also be wary of trusting them.
Think of the battlefield: How do you know if you can trust a digital technology to give you the best coordinates for an attack, or the safest route through an enemy encampment?
To answer this question, and to learn to trust AI team members, we have to return to some of the machine learning lessons we discussed earlier. Machine learning is the ability of computers to adjust their behavior based on the data to which they are exposed. In a battle zone, this could mean having a specific goal—such as minimizing the number of misses, as well as a set of rules that define a miss or a hit—to enable computers to adjust their decisions based on their experiences. This learning process requires a large amount of data that can be used for training. The important thing for you to remember is that when properly trained, AI is able to make accurate decisions with newly presented similar data and adjust its behavior when necessary.
Developing the digital mindset to work with an AI machine as a teammate means you trust that’s true even though the specific calculations that lead computers to make the decision, and which are based on the data, are mostly unknown. Also, you trust that the computers are able to use data to a greater extent than humans and may therefore achieve better results than humans.
For example, consider Arthur Samuel, one of the pioneers of machine learning, who taught a computer program to play checkers. His goal was to teach it to play better than he could, which was not something he could program it to do himself. As data, Samuel provided a large number of annotated games, with the “good moves” distinguished from the “bad moves” and a copy of the book Lees’ Guide to the Game of Draughts or Checkers to adjust the criteria for choosing moves. This data programmed the computer to choose moves checkers experts rated as “good” as often as possible.
In 1962, Samuel’s program beat the Connecticut checkers champion, who was the fourth top player in the nation. The key takeaway is that even though the computer program was applying preprogrammed rules, it could make decisions in a better way than the programmer could.
The stakes don’t seem so high when you’re playing checkers against a machine. Thus, people who use AI for gaming and other simple purposes don’t worry much about whether they can trust that the machine’s moves are accurate. In fact, most studies show that people just assume that AI-powered machines can crunch numbers and follow simple rules better than people can. Research by Jennifer Logg at Georgetown University bears this out.29 In her work, participants received advice about simple predictions about business success, song popularity, or romantic attraction. Some were told the advice came from a person, others that it came from an algorithm. In either case the advice was identical. Overwhelmingly people relied more on advice when they believed it came from an algorithm rather than from other people. They even showed a willingness to choose algorithmic advice over their own judgment.
Interestingly, willingness to defer to AI in decision-making was lower among participants with a less developed digital mindset and among experts who were simply less open to taking any advice. The accuracy of their judgments suffered as a result.
The trust in machines here was in low-stakes, fairly straightforward and numeric (based on accurate counting) tasks. And that typically results in better outcomes.
But the stakes are a lot higher when you’re collaborating with a machine on more complicated and less numerical predictions about, say, consumer behavior or to determine the best tactical plan for a military strike or to diagnose a disease. Trust in machines is still needed but harder to come by.
David Newman and colleagues from the University of Southern California conducted a series of experiments in which students and employees were told that they were receiving feedback on their performance from either a manager or an AI tool.30 The results showed that when people were being evaluated on what they perceived as “objective” criteria, like how many sales calls they closed in a certain amount of time, the AI did a better job at recognizing their performance than human managers—a similar outcome to Logg’s. But when they were being evaluated on what they perceived as “subjective” criteria, like how hard they worked to close the deal or whether or not they built a relationship with the potential customer, they felt that human managers were more accurate than the AI. Clearly, our trust in the accuracy of AI depends a great deal on the kinds of tasks the technology performs.31
Given these characteristics of AI, it is important to recognize that there are differences between trusting humans and trusting technology. AI’s complexity suggests a need for a leap of faith to trust in processes that cannot be directly observed or cognitively understood. Ella Glikson of Bar-Ilan University and Anita Woolley of Carnegie Mellon University have argued that learning to trust AI requires that at least two conditions are met:32
1. AI must be transparent. This means that people have to be able to understand how a set of algorithms works, or at least recognize the assumptions through which the algorithms are operating.
2. AI must be reliable. Low reliability significantly decreases trust and the way to restore trust is difficult and takes time.
Glikson and Woolley also argue that although anthropomorphic AI can increase trust to a point by making people liken the technology to a human counterpart, it also creates high levels of expectation that can be violated when the AI does not perform as expected.
To build on these assumptions about trust, a team of researchers at the US Army Research Laboratory led by D. Jessie Chen built a small ground robot (they called it the “Autonomous Squad Member”) that interacted directly with members of an infantry squad to make tactical battlefield decisions.33 Few situations require more trust in team members (people or machines) than combat—where the situation is changing rapidly, highly uncertain, and people’s lives are on the line. Because AI-powered robots can process incoming data more quickly than people can and calculate probabilities in real time, they have the potential to be key team members in such high-stakes situations, but only if the rest of the team feels comfortable trusting their recommendations.
To ensure transparency, the robot showed visual information at three levels. At the first level, the robot provided the team members the basic information about its current state and goals, intentions, and plans. At the second level, it described its reasoning process and the various contextual factors that it was considering when planning its actions. At the third level, the robot provided the team members with information regarding its projection of future states, predicted consequences, and the likelihood of success or failure of its recommendation.
When the robot made this kind of information transparent, team members reported higher levels of trust and a higher likelihood to take its recommendations than if the robot made a bunch of decisions and the team members didn’t know how or why. They also found that when team members took care to explain to the robot the reasoning behind their actions, the robot learned better. It turns out that being explicit and purposeful in how we communicate is key to a successful human-machine interaction whether you’re a human or a machine.
What does it mean for people interacting with machines to do their work? For engineers designing AI-powered machines that will be used as teammates, it means making their decision processes transparent.34 For the rest of us, developing a digital mindset that helps us to work alongside intelligent machines means understanding that trusting an AI-based agent is important and can be safe, but also understanding that doing so is different than trusting a person. When we interact with human colleagues we don’t know well, we use many proxies to determine if they’re trustworthy—for example, their level of education, where they work, what role they have, whether people we know trust them, whether they sound intelligent, and so forth. If we trust them and good things happen, we build a tendency to trust them again in the future. If bad things happen, we don’t.
Those same cues aren’t available to us when working with AI-powered machines. Instead, to determine if we can trust their decisions or recommendations, we first have to be cognizant of the type of task we’re asking them to do. If it’s more than simple calculation and classification, we need to ask to see the processes and routines the agents are using to make their decisions to determine if what they are doing is appropriate.35 If you’re responsible for managing people who are working with AI-powered machines as teammates, it is essential to make sure that you ask for and enable this kind of transparency so your employees or customers can have the confidence to work closely with the machines and follow their recommendations. And being transparent yourself when working with a machine is important to assure that the machine learns in a way that will improve your interactions with it over time.
Though you may think that such skills will become obsolete as digitally controlled, smart robots become more human-like in the way they interact with us, our own research, and the work of others, suggests that just the opposite is true. When robots act more like humans, we tend to treat them more like humans and that’s where the trouble begins.
GETTING TO 30 PERCENT
Working with Machines
Developing a digital mindset means accepting that machines are not people even if they increasingly look and sound human. Getting angry or talking politely at machines is not effective! Instead, you need to learn how machines work, which means understanding the basics of AI.
· AI is the computer inside the robot. Today’s cars, airplanes, thermostats, and even email filters all rely on artificial intelligence to perform specific tasks. When you interact with a machine by talking or writing it’s through what’s called a conversational user interface.
· Data, processing power, and algorithms are the three basic components that computer scientists use for developing AI.
· A technology stack is comprised of all the hardware and software systems needed to develop and run a single application. Typically, a technology stack includes a front-end system, a back-end system, and middleware.
· Machines mimic the “cognitive” functions of people by learning and problem-solving.
· Machine learning is a type of artificial intelligence that can generalize from examples and gain the ability to “learn” without being explicitly programmed by humans. Over time, machine learning enables computers to adjust their behavior to new situations based on the accumulated data to which they have been exposed.
· Advances in computational power paired with massive amounts of data mean that machines can make calculations much faster, more accurately, and at higher volumes than the human brain. For that reason, a digital mindset means becoming clear and intentional about the tasks you ask AI to perform.
· We cannot comprehend exactly how AI makes its calculations; we just give it instructions. A digital mindset means learning when it is and is not ok to trust the machine’s results and also when to check to see that the results make intuitive sense.
Understanding the basic ways that AI works gives you a vocabulary and set of concepts that helps gauge how little or how much to trust the machines you will increasingly work alongside. When collaborating with machines, remember that AI will not respond to your emotions but only to the explicit directions that you program it to follow.