AstraZeneca x Osaka Innovation Hub

I'mbesideyou x AZ
Challenge to quantify communication!

Learning Innovation through Data Conversion of Instinctive Communication in 1-on-1 Meetings

The spread of the new coronavirus infection ("COVID-19") has led to an increase in the number of companies adopting remote working. Digitization of business operations and the shift to online communication call for significant changes in every part of corporate activities.

One such area is human resource development and training. I'mbesideyou Inc. is a multimodal AI-based online communication video analysis service provider. The company kicked off a new collaborative project with AstraZeneca to support enterprise capability uplift.

We spoke with Shozo Kamiya of I’mbesideyou and Seiichi Hosoda and Yosuke Harumoto of AstraZeneca about the joint project.


  • PoC to verify how the online communication analysis service “I’mbesideyou” can improve coaching and teaching skills
  • Self-learning opportunities by letting data speak about communication in the recorded 1-on-1 meetings
  • COVID-19 drove shift to online communication. This shift unveiled hidden communication barriers.
  • Amid the shift to more virtual environment, innovation is needed for people development
  • 90% of the target population used the service. Some were heavy users, supporting value proposition of the solution.
  • Plan to scale beyond 1-on-1 meetings to product briefing role-playing after the pilot.

Remote working unveiled potential communication barriers

――First of all, can you give us an overview of the project and how AstraZeneca and I’mbesideyou worked together?

Seiichi Hosoda (Hosoda):The project created an opportunity for our employees to look back on 1-on-1 meetings with the help of the multimodal AI video analysis.

We used I'mbesideyou, a multimodal AI online communication analysis service for the video analysis, with a particular analytical focus on “facial expressions” and “conversation volume”.

The aims of the project were to create self-learning opportunities and encourage users to review their coaching and teaching skills based on data analytics, which would result in upskilling of the target population, and to provide better training opportunities for the trainees who receive guidance from the users.

(MR:Medical Representative、DM:District Manager、RSD:Director)

――What was the background behind this project?

Hosoda :Under the pandemic, majority of our work and communication processes offline were brought online, allowing us to work fully remotely.

In the meantime, our employees started voicing communication-related concerns from various parts of their work processes, such as feeling uneasy about communication approach and struggling with people management since changes that our staff are going through are less visible online.

Rather than new issues arising, latent issues have become apparent.

AstraZeneca has large salesforce, many of them Medical Representatives (MRs). Their main job is around communicating with customers. As we assign priority on capability uplift under the remote working environment, we felt that communication is the top issue we should be working on.

――How did you meet I'mbesideyou?

Hosoda :We met through Liu-san, Director of i2.JP, which is a healthcare open innovation network. He had met Kamiya-san from I’mbesideyou through an event. He matched us up, with the hope to address AZ internal pain points and introduce solutions for AZ Sales Academy.

Shozo Kamiya (Kamiya): My encounter with AstraZeneca was very exciting.

Many potential partners approach us with a casual show of interest but often without progress. However, Liu-san and Hosoda-san were completely opposite. They were very engaged, asking themselves questions and proposing possible use cases.

They were quick to action - Project was kicked off soon after. I am grateful to i2.JP for connecting us to the right partner.

――When you met Kamiya-san, did you already have a clear view about the use case of the tool?

Hosoda :To tell the truth, I didn’t. We thought it would be best if we could analyze the communication between MRs and physicians, but this goal is quite difficult to meet. It took a lot of thinking to determine how we can leverage the tool.

In the course of the discussion, we learned that Kamiya-san had a considerable passion for education with an abundance of use cases in people development.

With our employees working remotely, AZ was required to update its people development scheme. So, we decided to first test this solution in 1-on-1 interviews between managers and staff, with the goal of improving coaching and teaching skills.

Our Oncology team is highly innovation driven. Harumoto-san is a sales academy lead there. We chose Oncology team as our sounding board.

The point of the pilot was to use “no judgment”. The tool was offered not for performance evaluation but for self-learning.

――What was your impression, Harumoto-san, when you heard about this project?

Yosuke Harumoto (Harumoto):I was very excited to apply the solution in communication between managers and staff.

I used to manage a team of MRs for three and a half years in the past.

As a manager, I regularly held 1-on-1 interviews with my MRs, but to be honest, I was always unsure, wondering in the back of my mind, if the way I communicate with them based on my own instincts, is correct.

But there was no way to be sure, and no way to know how peers in my role were communicating in the interviews.

I was certain there were managers out there who have similar concerns as mine. I thought this tool may offer solutions to them and their concerns.

――You yourself had struggled with team management. Was the project kicked off smoothly?

Harumoto:Actually, that was the point we struggled with.

What used to be a closed interview was redesigned by introducing the video analysis. MRs or interviewees assumed that their response in the interview would be analyzed for performance evaluation.

We emphasized no link with performance evaluation many times, but MRs responded negatively. It took some time to clarify the objectives and communicate the positive intention that the tools are implemented to support managers’ self-learning.

――Do you think other companies may be faced with similar problems?

Kamiya:Yes, I do. This is a common concern for many companies, not just AstraZeneca.

That is why we consider "the objective of the solution” very important. We use the concept of "Start from Love".

We define how our solution should be applied, starting from love for the person being analyzed. The project was initiated with love, from a wish to help managers in need.

Hosoda-san and Harumoto-san relayed feedback from the members in the field. After setting forth a clear “no judgement” policy never to link the analysis to performance evaluation, we carefully advanced the project, while offering psychological safety of all involved.

Analyze interview quality based on facial expressions and conversation volume

――Can you tell us how you analyzed videos at the start of the project?

Harumoto:We set up a special zoom account for the project equipped with analytics and asked participants to conduct interviews on that zoom account.

In addition, we decided to limit analysis to "facial expressions" and "conversation volume" in this project, though I’mbesideyou can provide much more datasets.

――Why was that?

Harumoto:We thought that having too much data may defocus the point of our analysis.

It defeats the purpose if the produced data are not properly used for subsequent analysis and application. We discussed how we could motivate employees into using the tool and consulted with I’mbesideyou members to determine points for analysis.

Kamiya:In the case of AstraZeneca, we further narrowed our focus down to visualizing “smile" and "anger” for analyzing facial expressions.

It is said that there are seven types of facial expressions by the global standard. Japanese people tend to lack uniform patterns when reacting with facial expressions. So, we set our focus on the two types of facial expressions.

“Anger” by the global standard, but it is not necessarily true. The facial expression can also be a measure of “seriousness” if the subject is deep in thought. To make it easy to understand, we will rephrase “anger” as “seriousness” today, as the facial expression was interpreted that way in this project.

The orange line shows when the subject is happy (smile), and the purple line shows when the subject is serious. These lines indicate the degree to which and when the subject used these facial expressions in the interview.

Kamiya:As shown in the screen from the analysis, the purple line goes up when, for example, AI determines the manager is being “serious” based on the facial expression during the interview.

The "seriousness" score tends to increase when the person is concentrated or deep in thought. So, the data measures how serious the speaker is while talking or thinking.

――It is interesting to see our facial expressions are quantified this way.What perspectives did you have when analyzing the other metric, "conversation volume"?

Harumoto:In 1-on-1 interviews, we watch for a manager monopolizing the conversation versus actively listening to staff.

However, monopoly or not can be very subjective. In general, managers are unaware of the fact that they are talking too much. By visualizing the fact in numbers, they can clearly recognize themselves.

We provided facial expressions and conversation volume data, offering materials for District Managers (DMs) to self-reflect their interviews.

In addition, we were presented with personal, team and company characteristics data as well as distribution plotting for comparison. I think this is the part that made this project more engaging. This kind of insight is one of the benefits that I’mbesideyou has to offer, for which we are grateful.

Ninety percent of the target used the service. The service is scaled to a “product briefing” role-play session, the cornerstone of MR activities.

――I'm curious about the utilization rate and user reaction.

Harumoto:To tell the truth, uptake in utilization was slow in the beginning.

In order to increase the utilization rate, we prepared detailed specification documents, added verbal instructions on how to use, and interviewed managers using the tool for practice share. Communicating the value of the solution carefully, we were able to eventually reach 90% of the target managers.

Several of them are heavy users. One of them proposed using this solution to analyze a role-play session beyond 1-on-1 interviews.

At the request of the manager, we did analyze role-play sessions.


Harumoto:Yes. A critical role that MRs play is to communicate drug information to HCPs such as physicians and pharmacists. They practice through role-playing sessions for each brand before going out to the field and meet them.

Role-plays are structured in the way that one person plays a role of an MR and the other person of a physician. Afterwards, a manager provides feedback to develop skills of the MR.

Among the skills, the selling skill of “making others emotionally engaged” is particularly important. Years of experience as a manager can tell a good call from a bad call in a role-play. But the judgement is based on a hunch.

My experience as well as Hosoda-san's was driving the concept to scale to role-playing sessions. So, we were happy to see it added and validated in the project.

――What was your reaction to the role-play analysis?

Harumoto: In the role-playing session, we mainly analyzed "speech rate" and "length of speech”. With the I'mbesideyou solution, what used to be instinctive become visible as data.

We reviewed all of the role-plays analyzed. We found a match between what our instincts and analytics determined to be a good role-play. We keep our hopes high that the solution, if implemented, can support managers when providing role-play feedback.

Hosoda:As Harumoto-san said, feedback on role-playing is extremely difficult.

For one thing, feedback is quite instinctive; for another, role-players tend to be too serious to remember the details.

It is quite difficult to give feedback on something that is not remembered. However, video analysis allows us to check the video and provide feedback based on the combination of our instinctive thoughts and data. I believe this dual approach can add sophistication to our role-plays.

Kamiya:Communication that elicits emotional engagement is a wonderful method. On the other hand, as Harumoto-san and Hosoda-san mentioned, it is difficult to make communication reproducible.

I'mbesideyou closes this gap by offering measurability and objectivity. The project was challenging, but I believe it will help us move forward.

Learning innovation for MRs – First step for AstraZeneca

――The project was concluded in March 2022. How do you look back about the whole project?

Harumoto:After the project was completed, we conducted a questionnaire internally. About seventy percent of the respondents answered that they gained insights and added changes through analyzing their own interviews. After witnessing points to improve and issue resolution, about eighty percent reacted positively, sharing their intention to use the solution including under limited scenarios. I think this was a big gain.

Hosoda:Not only the managers who manage MRs, but also the one level above realized the need and potential for improving interviews as a result of this project. We are starting to hear a request to educate managers on coaching. This is evidence that the project has had positive impact on overall communication among the members.

Kamiya:We are very pleased with the positive response we have received.

What surprised me when we worked together on this project was AstraZeneca's enthusiasm.

The commitment of everyone we met was enormous. The enthusiasm did not drop off until the end of the project's six-month period. I have never seen a project progress in such a way. It was awesome.

We provide online communication video analysis with the aim of realizing a society where all people respect and learn from each other, and a world where people can be mentally healthy. From user’s point of view, our tool may be just a “means”.

But we need to deliver an “end”. This project has given us many perspectives to realize this purpose.

―― I understand it was a fruitful project for both companies.How do you see this achievement leading to the next step, and what is AstraZeneca's vision for the future?

Hosoda:This PoC revealed possibilities and issues. So, we are continuing to consult with I’mbesideyou although the project is completed.

Under the pandemic, there has been a need for change in the areas of talent development and training, but we felt that there were not enough innovative elements.

In addition, conversations that take place in 1-on-1 interviews and meetings with HCPs often reside in a “black box”. Problems are difficult to be known.

In addition to the novelty of data analysis in the field of communication (as it is hard to reproduce), this project provided us with an opportunity to think about how we can use the data to improve our capability and to rethink our approach to training.

Going back to the mission of our team, MR capability uplift is intended to ensure our products are delivered to patients who are suffering from illnesses.

In order to achieve this, we would like to continue to walk in the footsteps of I’mbesideyou, while borrowing from their technology and wealth of knowledge.

――What has your participation in i2.JP been like for I’mbesideyou?

Kamiya:We founded our business under the pandemic. We are currently working on programs in India, the U.S., and other countries around the world.

In this context, we are starting to sense overwhelming corporate value of AstraZeneca. The company is virtually recognized by all players in the global market.

Of course, the experience gained from this project will be utilized in our next chapter. But the fact that we were able to promote the project with AstraZeneca have huge impact on us as a start-up company, and I feel that it is helping us today.

Since we have been given a wonderful opportunity to collaborate through i2.JP, I hope that new collaborations will continue to be born through i2.JP.

――Lastly, what would you say to the companies and organizations that are considering joining i2.JP.

Kamiya: I think our history of collaboration with AstraZeneca, a globally established company, will be a big advantage.

Right now, we offer solutions in the mental health area. We hope to have opportunities to work with you in the future!

i2.JP offers opportunities for business matching and discussions. If you are a company or organization with the commitment to having dialogue to chart a course toward patient centricity, please visit i2.JP. please contact us

神谷 渉三(Shouzou Kamiya)
Shozo Kamiya
President I'mbesideyou Inc.

Founded I'mbesideyou in 2020 with the aim of realizing a society that embraces individuality of people. More than 120 patent applications filed for proprietary AI analysis technology specializing in online communication video analysis. Cumulatively over 300,000 hours of videos were analyzed in just two years.

細田 征一(Seiichi Hosoda)
Seiichi Hosoda
Director, Sales Academy, Commercial Excellence, AstraZeneca K.K.

Supervises four teams in Commercial Excellence. Responsible for planning, strategizing, and implementing training programs for human resource development for approximately 2,000 employees

春本 洋佑(Yosuke Harumoto)
Yosuke Harumoto
Commercial Capability Lead Manager for Oncology Business Unit, Sales Academy, Commercial Excellence, AstraZeneca K.K.

Responsible for strategic planning and implementation of product training, disease training, and skills training for the Oncology Business Unit within Commercial Excellence.

contact us

For inquiries

Interested in joining us?

Please confirm the Privacy Policy before sending your inquiry

Thank you for your interest. We will get in touch soon.