Apple buys Emotient emotion detection company

It looks likeApple hopesyour emotions are written all over your face.

As first reported last week by the Wall Street Journal, the technologygiant has acquired Emotient, a startup that uses artifical intelligence tospecializein "emotion detection."

CBC Radio technology columnistDan Misenerexplains the coming wave of emotionally awarecomputers.

What do we know about Emotient?

It's a company that specializes in what's called "automated facial expression recognition."Basically, that involvesteaching a computer to look at human facesand recognize emotions.

The company says its technology can recognize a number of different facial expressions joy, sadness, surprise, anger, fear, disgust, and contempt.

And they say they can identify those emotions in real-time, and in busy environments, like a retail space.

Details of the company's acquisition byApple like how much Apple paidhaven't been revealed, and Apple hasn't said anything about what it plans to do with the technology.

But we do know this is part of a growing field, called "affective computing," and there's a huge amount of money and attention going into it right now.

How does a computer recognize emotions in someone's face?

Michel Valstarisa computer science professor at the University of Nottingham, and he specializes in getting computers to recognize facial expressions.

He said these kinds of systems look at facial featuresand the geometry of a person's face.

"What kind of wrinkles are there, what kind of shape doesthe corner of your mouth have, how much white is there in the eyes," are facial features Valstar said a computer might analyze.

"Or you can go the other way, and you can look at the geometric shape of the face. Because when you smile, obviously the corners of your mouth go out and up. They will indicate to a systemthat you are smiling," he said.

• WATCH: Michel Valstar explains how computers recognize facial expressions

By looking at the textures and geometry of someone's face, and combining those with sophisticated pattern-matching, a computer can measure what are called the six "basic expressions" anger, disgust, fear, happiness, sadnessand surprise.

• Microsoft's new emotion-detecting AI can be hilariously bad at detecting emotions

What's driving the push to develop computers that can sense emotions?

The earliest adopters of this technology have been advertisers and market researchers.

Let's say you have a video of customers using your product, or walking past a retail display, or watching an advertisement. You can use software like Emotient to get a sense of how people feel about your product.

However, that brings up some obvious privacy and ethical questions.

It's one thing for a computer to analyze yourface if you know it's happening, and you've agreed to be part of a market research group.

But it's another thing for peoplesimply walking down the streetor browsing through a store to have their facesanalyzed without their knowledge or consent.

Who else might be interested in using emotionally-aware computers?

Valstarfocuses his research on medical conditions that alter expressive behavior, like Alzheimer's disease, depression, or chronic pain. And he said computers that can recognize facial reactions could be beneficial in his work.

"If you could measure how often somebody smiles and how intense people smile,but also whether they look at youoravert their gaze. Those are very strong objective signals that you could use in diagnosis and monitoring of people withvaryingmedical conditions," he said.

He also said facial expression technology could offer clinicians more objective measurements than they can achieve right now, because facial expressions can difficult to monitor, and interpreting them can be subjective.

How will Apple use theface-reading technology it's acquired?

It's unclear at this point, because Apple hasn't said what they plan to do with Emotient's technology.But many within the facial expression recognition community have been speculating.

One obvious possibility is that Apple willuse this technology to improve Siri, its personal assistant software. If Siri knows that you're happy, or sad, or tired, or bored, it could tailor its responses to you.

Another possibility is using the technology to tailor recommendations. Apple has a large content business itsellse-books, TV shows, movies, and music.Emotion detection could help Applerecommend something that suits your mood.

There are also rumours Apple is working on its own car. It's not hard to imagine this sort of technology monitoring drivers, especially for safety purposes.For example, the technology could ensure a driver'seyes are on the road andthey'repaying attention. Or it couldlookfor signs of drowsiness.

When asked aboutthe possibility of Emotient's technology being used inside an Apple car, Michel Valstar said, "I think this is getting to an areathat I'm not allowed to talk about."

But wherever it ends up,this is probably not the last we've heard about emotionally-aware computing justthe beginning.

• Microsoft's new emotion-detecting AI can be hilariously bad at detecting emotions
• Pepper, SoftBank's friendly robot, offers ardent attention, cool dance moves