NUAN Nuance Communications Inc

By Jared Council 

Microsoft Corp.'s $16 billion acquisition of Nuance Communications Inc. will bolster its strength in natural language processing, artificial intelligence that can interpret and respond to human language.

The technology has become ubiquitous in recent years. It powers Apple Inc.'s Siri and Amazon.com Inc.'s Alexa and automated call-center attendants. Voice is coming to rival text as a computing interface. Research firm Gartner predicts that by 2025, 50% of knowledge workers will use an NLP-powered virtual assistant daily, up from 5% in 2020.

Here's how natural language processing works.

Calculating the odds

Much of today's natural language processing calculates the probability of how a sequence of words should be interpreted among all the possible ways. For instance, it can tell whether the word "club" likely refers to a sandwich, the game of golf, or night life.

When and where did it originate?

The field traces its roots back to the 1950s and 1960s. Its pioneers include linguist Noam Chomsky and computer scientist John McCarthy, according to Michael Picheny, a research professor at New York University's Courant Institute Computer Science Department and Center for Data Science. In its early days, the process of getting computers to analyze and understand language involved hard-coded software rules and expert systems, according to Dr. Picheny, who's been working in the field since 1979.

Enter neural networks

The technology has advanced with the maturity of so-called neural networks, which began to take off in the early 2010s, according to Dr. Picheny. Neural nets mimic neurons in the human brain, and power an AI technique known as deep learning, as well as facial-recognition systems.

Whether it's trying to understand text or speech, a deep-learning model analyzes a sentence by first examining fundamental components known as tokens, which can be words or segments of words, Dr. Picheny said. Those tokens are analyzed in tandem with other words in the sentence to determine the likely usage of each word. A neural network trained on millions or billions of sentences can make a probabilistic assumption about which kind of bank is being talked about in the sentences "I went across the river to the bank" and "I went across the street to the bank, " for instance.

If a speaker is trying to book a flight with an NLP-powered automated system, the algorithm looks for words that are likely the origin city, the destination city as well as the departure and return times, Dr. Picheny said.

The use of NLP in voice assistants tends to involve additional steps, according to Bern Elliot, a research analyst at Gartner Inc. The system identifies the words based on the sounds and makes statistical guesses about what word was uttered based on surrounding words. Those words are translated to text and analyzed for intent, such as whether the word "I" or "eye" has been uttered. In communicating a response to the user, the text is then converted back to speech.

What are the challenges?

NLP systems currently in use are designed for very specific uses, and their performance tends to break down when they encounter something outside of that zone, said Ian Jacobs, an analyst at Forrester Research Inc. For example, if a chat- or voice-bot is helping a user make a flight reservation, that system may have trouble answering questions about reward points or the status of a refund.

It is also difficult to train NLP systems to predict which situations might require a simulation of empathy and convey that response in the choice of words and tone, which can affect the user experience, Mr. Jacobs said. "You don't want to say something to the system like, 'Yeah, my mom just died,' and then have the system say, 'Oh, great. Your mom just died.' It's got to mirror your tone as well," Mr. Jacobs said.

What does the future hold?

Academic and corporate researchers are striving to make NLP systems more general -- that is, able to handle more requests and inquiries without breaking down. Another goal is to make the systems more responsive to context. For instance, Amazon said in December that its Alexa technology can ask a clarifying question to understand requests it has never heard before. Right now, that capability is limited to a few applications, but they are likely to expand. Earlier this year, software company OpenAI LP showed that its NLP system, dubbed DALL-E, could generate images based on a user's request -- for instance, it could produce an image of "an armchair in the shape of an avocado" if instructed to.

Researchers see NLP as one of the most promising technologies for creating machines that can exhibit what is known as general intelligence by performing tasks and responding to a broad range of human requests without being explicitly trained to do so.

"Though scientists and researchers have done a lot of theoretical work on NLP in the past, we have only recently started seeing its real-world use cases," said Ritu Jyoti, who leads the AI practice at International Data Corp.

Write to Jared Council at jared.council@wsj.com

(END) Dow Jones Newswires

April 13, 2021 13:17 ET (17:17 GMT)

Copyright (c) 2021 Dow Jones & Company, Inc.