I'mbesideyou x AZ Challenge to quantify communication!

The spread of the new coronavirus infection ("COVID-19") has led to an increase in the number of companies adopting remote working. Digitization of business operations and the shift to online communication call for significant changes in every part of corporate activities.

One such area is human resource development and training. I'mbesideyou Inc. is a multimodal AI-based online communication video analysis service provider. The company kicked off a new collaborative project with AstraZeneca to support enterprise capability uplift.

We spoke with Shozo Kamiya of I’mbesideyou and Seiichi Hosoda and Yosuke Harumoto of AstraZeneca about the joint project.

PROJECT CASE

Remote working unveiled potential communication barriers

――First of all, can you give us an overview of the project and how AstraZeneca and I’mbesideyou worked together?

Seiichi Hosoda (Hosoda)：The project created an opportunity for our employees to look back on 1-on-1 meetings with the help of the multimodal AI video analysis.

We used I'mbesideyou, a multimodal AI online communication analysis service for the video analysis, with a particular analytical focus on “facial expressions” and “conversation volume”.

The aims of the project were to create self-learning opportunities and encourage users to review their coaching and teaching skills based on data analytics, which would result in upskilling of the target population, and to provide better training opportunities for the trainees who receive guidance from the users.

（MR：Medical Representative、DM：District Manager、RSD：Director）

――What was the background behind this project?

Hosoda ：Under the pandemic, majority of our work and communication processes offline were brought online, allowing us to work fully remotely.

In the meantime, our employees started voicing communication-related concerns from various parts of their work processes, such as feeling uneasy about communication approach and struggling with people management since changes that our staff are going through are less visible online.

Rather than new issues arising, latent issues have become apparent.

AstraZeneca has large salesforce, many of them Medical Representatives (MRs). Their main job is around communicating with customers. As we assign priority on capability uplift under the remote working environment, we felt that communication is the top issue we should be working on.

――How did you meet I'mbesideyou?

Hosoda ：We met through Liu-san, Director of i2.JP, which is a healthcare open innovation network. He had met Kamiya-san from I’mbesideyou through an event. He matched us up, with the hope to address AZ internal pain points and introduce solutions for AZ Sales Academy.

Shozo Kamiya (Kamiya): My encounter with AstraZeneca was very exciting.

Many potential partners approach us with a casual show of interest but often without progress. However, Liu-san and Hosoda-san were completely opposite. They were very engaged, asking themselves questions and proposing possible use cases.

They were quick to action - Project was kicked off soon after. I am grateful to i2.JP for connecting us to the right partner.

――When you met Kamiya-san, did you already have a clear view about the use case of the tool?

Hosoda ：To tell the truth, I didn’t. We thought it would be best if we could analyze the communication between MRs and physicians, but this goal is quite difficult to meet. It took a lot of thinking to determine how we can leverage the tool.

In the course of the discussion, we learned that Kamiya-san had a considerable passion for education with an abundance of use cases in people development.

With our employees working remotely, AZ was required to update its people development scheme. So, we decided to first test this solution in 1-on-1 interviews between managers and staff, with the goal of improving coaching and teaching skills.

Our Oncology team is highly innovation driven. Harumoto-san is a sales academy lead there. We chose Oncology team as our sounding board.

The point of the pilot was to use “no judgment”. The tool was offered not for performance evaluation but for self-learning.

――What was your impression, Harumoto-san, when you heard about this project?

Yosuke Harumoto (Harumoto)：I was very excited to apply the solution in communication between managers and staff.

I used to manage a team of MRs for three and a half years in the past.

As a manager, I regularly held 1-on-1 interviews with my MRs, but to be honest, I was always unsure, wondering in the back of my mind, if the way I communicate with them based on my own instincts, is correct.

But there was no way to be sure, and no way to know how peers in my role were communicating in the interviews.

I was certain there were managers out there who have similar concerns as mine. I thought this tool may offer solutions to them and their concerns.

――You yourself had struggled with team management. Was the project kicked off smoothly?

Harumoto：Actually, that was the point we struggled with.

What used to be a closed interview was redesigned by introducing the video analysis. MRs or interviewees assumed that their response in the interview would be analyzed for performance evaluation.

We emphasized no link with performance evaluation many times, but MRs responded negatively. It took some time to clarify the objectives and communicate the positive intention that the tools are implemented to support managers’ self-learning.

――Do you think other companies may be faced with similar problems?

Kamiya：Yes, I do. This is a common concern for many companies, not just AstraZeneca.

That is why we consider "the objective of the solution” very important. We use the concept of "Start from Love".

We define how our solution should be applied, starting from love for the person being analyzed. The project was initiated with love, from a wish to help managers in need.

Hosoda-san and Harumoto-san relayed feedback from the members in the field. After setting forth a clear “no judgement” policy never to link the analysis to performance evaluation, we carefully advanced the project, while offering psychological safety of all involved.

Analyze interview quality based on facial expressions and conversation volume

――Can you tell us how you analyzed videos at the start of the project?

Harumoto：We set up a special zoom account for the project equipped with analytics and asked participants to conduct interviews on that zoom account.

In addition, we decided to limit analysis to "facial expressions" and "conversation volume" in this project, though I’mbesideyou can provide much more datasets.

――Why was that?

Harumoto：We thought that having too much data may defocus the point of our analysis.

It defeats the purpose if the produced data are not properly used for subsequent analysis and application. We discussed how we could motivate employees into using the tool and consulted with I’mbesideyou members to determine points for analysis.

Kamiya：In the case of AstraZeneca, we further narrowed our focus down to visualizing “smile" and "anger” for analyzing facial expressions.

It is said that there are seven types of facial expressions by the global standard. Japanese people tend to lack uniform patterns when reacting with facial expressions. So, we set our focus on the two types of facial expressions.

“Anger” by the global standard, but it is not necessarily true. The facial expression can also be a measure of “seriousness” if the subject is deep in thought. To make it easy to understand, we will rephrase “anger” as “seriousness” today, as the facial expression was interpreted that way in this project.

The orange line shows when the subject is happy (smile), and the purple line shows when the subject is serious. These lines indicate the degree to which and when the subject used these facial expressions in the interview.

Kamiya：As shown in the screen from the analysis, the purple line goes up when, for example, AI determines the manager is being “serious” based on the facial expression during the interview.

The "seriousness" score tends to increase when the person is concentrated or deep in thought. So, the data measures how serious the speaker is while talking or thinking.

――It is interesting to see our facial expressions are quantified this way.What perspectives did you have when analyzing the other metric, "conversation volume"?

Harumoto：In 1-on-1 interviews, we watch for a manager monopolizing the conversation versus actively listening to staff.

However, monopoly or not can be very subjective. In general, managers are unaware of the fact that they are talking too much. By visualizing the fact in numbers, they can clearly recognize themselves.

We provided facial expressions and conversation volume data, offering materials for District Managers (DMs) to self-reflect their interviews.

In addition, we were presented with personal, team and company characteristics data as well as distribution plotting for comparison. I think this is the part that made this project more engaging. This kind of insight is one of the benefits that I’mbesideyou has to offer, for which we are grateful.

Ninety percent of the target used the service. The service is scaled to a “product briefing” role-play session, the cornerstone of MR activities.

――I'm curious about the utilization rate and user reaction.

Harumoto：To tell the truth, uptake in utilization was slow in the beginning.

In order to increase the utilization rate, we prepared detailed specification documents, added verbal instructions on how to use, and interviewed managers using the tool for practice share. Communicating the value of the solution carefully, we were able to eventually reach 90% of the target managers.

Several of them are heavy users. One of them proposed using this solution to analyze a role-play session beyond 1-on-1 interviews.

At the request of the manager, we did analyze role-play sessions.

――Role-playing?

Harumoto：Yes. A critical role that MRs play is to communicate drug information to HCPs such as physicians and pharmacists. They practice through role-playing sessions for each brand before going out to the field and meet them.

Role-plays are structured in the way that one person plays a role of an MR and the other person of a physician. Afterwards, a manager provides feedback to develop skills of the MR.

Among the skills, the selling skill of “making others emotionally engaged” is particularly important. Years of experience as a manager can tell a good call from a bad call in a role-play. But the judgement is based on a hunch.

My experience as well as Hosoda-san's was driving the concept to scale to role-playing sessions. So, we were happy to see it added and validated in the project.

――What was your reaction to the role-play analysis?

Harumoto： In the role-playing session, we mainly analyzed "speech rate" and "length of speech”. With the I'mbesideyou solution, what used to be instinctive become visible as data.

We reviewed all of the role-plays analyzed. We found a match between what our instincts and analytics determined to be a good role-play. We keep our hopes high that the solution, if implemented, can support managers when providing role-play feedback.

Hosoda：As Harumoto-san said, feedback on role-playing is extremely difficult.

For one thing, feedback is quite instinctive; for another, role-players tend to be too serious to remember the details.

It is quite difficult to give feedback on something that is not remembered. However, video analysis allows us to check the video and provide feedback based on the combination of our instinctive thoughts and data. I believe this dual approach can add sophistication to our role-plays.

Kamiya：Communication that elicits emotional engagement is a wonderful method. On the other hand, as Harumoto-san and Hosoda-san mentioned, it is difficult to make communication reproducible.

I'mbesideyou closes this gap by offering measurability and objectivity. The project was challenging, but I believe it will help us move forward.

i2.JP offers opportunities for business matching and discussions. If you are a company or organization with the commitment to having dialogue to chart a course toward patient centricity, please visit i2.JP. please contact us