Overview of Analytics and Experiential Learning


October 28, 2021 - Sophie Thompson

Transcript


Intro

Hello, I'm Sophie Thompson and I'm the Co-founder and CEO of VirtualSpeech, a multi-award winning soft skills training platform used by 350,000 learners worldwide.

Experiential Learning

VR training means it's now possible to expose individuals to a wide variety of environments with the simple click of a button. We can now expose learners to environments, social behaviors, people and conversations that we previously weren't able to, which in turn increases the possibility of meaningful changes in behavior.

But now we know we can tap into this idea of learning through experience on-demand, how do we measure the results?

VR training allows for unique data capture that provides systematic, objective and unique insights into individual learners as well as the collective group.

The data points we receive through experiential learning can enhance the learning process by revealing insights into real world behaviors and identifying gaps in learning and understanding in a quantifiable way.

Types of Analytics & Data

Let's take a look at some of the data and insights that VR training can provide coaches. We can divide these insights into 3 key categories:

Spatial

  • Monitor how they move and the learning paths they take?
  • Where do they spend most of their time in the learning experience?
  • Which environments, objects or tools are they interacting with?

Behavioral

  • What are they focusing on in the experience?
  • How are they interacting with the scene - are they engaged?
  • What are they doing - are they making the most of the experience?

Performance

  • How do they score on assessments in VR?
  • What is their progress over time?
  • What can we infer from collective assessments about the training experience?

At VirtualSpeech, we've created a number of different tools to provide insights into learners' behaviors so that we can accurately measure how effective a VR training experience is, and what the ROI looks like.

The first of these is: speech analysis. I'm going to use the example of our VR public speaking training to illustrate each of the analytics tools below.

Speech analysis provides real-time feedback to accelerate learning and instantly identify areas that need improvement. For example, a learner can practice their presentation skills in a range of VR environments varying from 1-on-1 meetings to performing in front of an audience of hundreds of people.

When the learner has finished speaking, they are instantly provided with AI-powered feedback on their dialogue, including their use of hesitation words, pace, volume, tone, listenability, and how the audience is likely to have perceived them.

For example, the software can tell someone if their sales pitch has come across as persuasive or aggressive, or whether their speech was engaging and energetic or low-energy and boring.

Here is what some of that feedback looks like in the VirtualSpeech app.

This analysis can be saved and tracked for every learning exercise, quiz, and dialogue completed in VR.

Now we all know communication isn't just about what we say or how we say it, but also how we deliver the message with our body language too.

And VR can provide feedback on that. Here's an example of an eye contact heat map used by clients such as Vodafone and Deutsche Telekom to analyze a learner's understanding of spreading their gaze across different areas of a room.

Eye contact is critical for strong communication skills but before VR there was no way of measuring eye contact in a quantifiable way, and so we didn't have the data for tracking progress, and how this would translate into real-world behavior change and connection.

Learners' body language can also be measured in VR based on their hand gestures, which can be tracked with the VR controllers, and help make up an overall score for communication skills such as empathy, and storytelling, and persuasion.

Dashboard

So where does all this data go? Most VR providers will bring all this data together for you in a dashboard so you can see progress over time, and some of the more spatial and behavioral analytics I mentioned at the start too.

At VirtualSpeech, both the learner and coach/ manager can track individual progress to help identify areas of improvement and subsequent learning pathways. In addition to this, the admin can see the time spent in VR, which VR environments they spent the most time, and an individual's overall score each time they went into VR.

This helps to learn more about the individual's learning journey and their personal progression across various skill sets.

It's on the admin dashboard where the learner can also choose to send the coach or manager an in-app recording of their performance too. Whether that be the audio of a presentation or sales pitch, or a video of them carrying out a branched learning scenario, a coach can provide tailored, personable feedback to learners to complement the objective AI feedback they've already received in VR.

In VR, learners can also practice together in the same virtual space even if they are geographically hundreds of miles apart. By doing so, they can receive a third layer of feedback and analysis from their peers.

This provides an unparalleled feedback loop for soft skills training. Learners can receive instant feedback from AI, personalized feedback from a human coach, and even peer-reviewed feedback. Together, this provides three-factor feedback, so learners are able to recognize areas of strength and improvement, and thus accelerate their learning.

Benefits of VR Analytics

  1. The individual learner can track their progress in a quantifiable way, motivating them for their learning.
  2. Admins, managers and coaches can receive quantitative feedback on soft skills learning experiences, enabling them to measure the effectiveness of a training experience, quantify ROI and have realistic data points for real-world behavior change
  3. AI analysis can be used alongside subjective feedback provided by colleagues or coaches to provide a more reliable picture of someone's performance, skills and behaviors

Ethics

Whenever we talk about data, it's important to consider the ethical implications of what we are tracking and consider what data we actually need to enhance an experience or track ROI.

Across many VR solution providers, individual user data can be anonymized. For example, when we worked with one of the Big 4 Accountancy Firms, learners were assigned as User 1, User 2, etc. so neither us nor the client had personally identifiable information about any of the learners.

The client could still receive feedback and measure progress of the group before and after a learning experience, and quantify the effectiveness and ROI, but recognizing the journey of individual learners wasn't important to them, so we simply removed that ability.

Another consideration we are often asked about with VR analytics is whether there is bias in the AI. We build our system to avoid this as much as possible but we also recommend not to use AI feedback to pit students against each other or use the performance score data for employee performance reviews.

Everyone's journey is different and progress for one person doesn't look the same as someone else's and we should use AI as a method and guide, but not forget the humans it is analyzing and their needs.

Conclusion

One of the unique things about VR training is the wealth of data and analysis that's available compared to solely traditional methods of learning.

With the data generated from VR, insights into the effectiveness of soft skills training are made both accessible and actionable.

VR offers unprecedented opportunities for analytics for both the learner themselves and coaches. Learners can use the analytics to track their progress in a quantitative way and identify areas of strength and weaknesses, and coaches/ managers can use these data points to measure the effectiveness of soft skills training, learner understanding and progress, and track ROI of the learning experience.