Automated Emotion Detection Could Get to the Truth During Online, Video Conference Job Interviews

Automated emotion detection could expand the possibilities of video conferencing.

You can say whatever you want, contrive the most beautiful verbal diversion that ever reached human ears, but the camera doesn’t care. The camera sees right through you.

Because the camera can read your emotions. And this intelligent camera is trained on you as you are put through your paces by what is hopefully your next employer during an online interview. We’ve all been guilty of padding our resumes, at one time or another, and even stretching the truth a little during a job interview to give ourselves the best chance at the position.

But would you be brave enough to lie to a camera that reads your facial movements and a computer that uses that information to detect your mood, your anxiety, and hidden emotions? The camera in question is Affectiva’s emotion recognition technology, and it’s already available commercially for use in social media, smart appliances, and advertising.

It’s only a matter of time before it enters the online job interview. Yes, it’s terrifying. You’ve been warned.

Online Truth Seekers

Affectiva has deliberately stated that its product is not a lie detector, nor will they develop that side of its potential. It requires companies using its device to get express consent from anyone being subjected to its enquiring gaze.

But there’s clearly a gray area to be exploited here between detecting lies and detecting mood. After all, if you’re searching for an insight unintentionally displayed on your subject’s face, you’re chasing something they either can’t or won’t articulate.

And their model is, naturally, already attracting attention from other researchers and developers who may have other deployments in mind. Nothing untoward, but with a sharper inclination to discover hidden truths. Police forces, for instance, or financial institutions running routine risk assessments, insurers trying to get to the bottom of a claim, and online therapists and doctors who need to cut through the bluff.

Truth detection is, actually, a growing online market.

Companies like AC Global, for example, provide advanced, voice-based audio truth verification instruments for use by employers recruiting talent, particularly in other countries. What Affectiva’s emotion recognition tech has over those kinds of services, and especially the old-fashioned polygraph, is that it fits seamlessly into an online video call, which is an increasingly popular way to interview candidates for a job position.

Emotion Sensitive Interviews

In fact, if you were to employ Affectiva’s device during an interview it’s actually easier if it’s the online variety.

The presence of a video camera during an in-room job interview is confronting, and screams police interrogation. The use of a camera during an online job interview is a necessity. There’s no foreign object in this context–actually, it’s the applicant’s own webcam that’s in use.

The handy fact is the camera isn’t really what’s important–although there are obvious advantages to having as clear and steady a stream as possible.

Instead, the machine crunching comes from the Affectiva app. It’s a learning program that maps and tracks the features of a subject’s face, and matches the combination of movements between features with its database of 4.8 million other faces from 78 different countries to determine the appropriate mood and emotional reaction.

You can test its accuracy yourself at Affectiva’s website, although it’s displayed within the advertising context for which it’s explicitly intended. Bottom-line though, it’s smart enough to tell the difference between a smile, a smirk, and a fake smile.

And it’s not just blatant, unsubstantiated boasting the emotion tech can detect. It could also tell you how an applicant handles stress, how much confidence they possess, how engaged they are with the interview itself, and perhaps even their trustworthiness.

Online Interviews of the Future

Assuming Affectiva deems job interviews an appropriate use of its tech, or that some imitator does, it could give employers a novel new way of picking the right candidate. Firstly, they’d have to make sure the applicant understood the technology, and was comfortable with, or at least agreeable to, its use.

Once everyone’s on board the interview should flow as normally as any other online job interview.

The device has already been deployed in ooVoo’s social media video calling platform, where it can be used, unobtrusively, to “gauge the vibe of a chat room.” If it’s smooth enough to work in a relaxed social environment it shouldn’t interfere with a far more formal job interview setting.

So, the interviewee could ask their usual questions, “List your strengths”, “What personal experience will help you in this position,” “Tell us how you deal with multiple tasks at once,” etc.

Only now they have both the applicant’s verbal responses, and also an insight into how they felt while pondering and answering the question.

You could tell what made them uncomfortable or anxious, gauge if their interest waned, measure their confidence, and, when prompted by the device, ask follow-up questions to assess strengths or probe weaknesses.

It’s true much of this visual information can be picked up by a perceptive interviewer over a video call anyway. What Affectiva has created is a dedicated device that searches only for such insights, staring at an applicant without listening to the verbal answers, and without pausing to consider new questions.

It’s certainly enough to make anyone reconsider fudging their resume.

Subscribe to VC Daily