SCOTTIE

SCOTTIE is an ARPA-E funded project in partnership with Raytheon Technologies. For this project, I helped design study protocol to investigate the impact of VR and other remote work technologies on collaboration.

A human-like avatar interacting with a cirtuit board inside a virtual environment.

Problem Statement

The goal of this ongoing work is to investigate how different modes of communication, such as Virtual Reality, impact collaboration outcomes. In the long term, this work will help companies choose if air travel for face-to-face collaboration is necessary, or if other technologies can meet their needs. By replacing business travel with technology solutions, companies can reduce their environmental footprint and save money.

Research Questions

When planning this research, it was important to first establish how the use of different technologies, or lack thereof, might affect a team’s ability to collaborate. The team used existing literature to identify four main categories of variables that might impact collaboration: task related elements (e.g., time, space, and symmetry), social factors, and technology affordances.

The current experiment focuses on the affordances associated with different communication technologies. The experiment explained here seeks to answer the following research questions:

  1. How does the amount of agency provided by different technologies effect collaboration?
  2. How does the amount of embodiment provided by different technologies effect collaboration?
Two avatars collaborating in a virtual environment.

Circuit World application

Task

In order to test these research questions, the team developed a collaborative task to use as a test bed. The task was for a researcher to train a participant to assemble a bread board circuit. The goal of the task was to complete the circuit using a soft potentiometer to manipulate the color of an LED light. To facilitate task collaboration online, the team’s developer created an app called Circuit World, which allowed the instructor and participant to interact via a web browser or VR to assemble the circuit.

Experiment Design

For the experiment, each participant was assigned to collaborate with a researcher in one of three conditions: in-person, online via Zoom, or using a Virtual Reality. Next, researcher met with the participant to train them on the assembly of the circuit. During the training, the participants were encouraged to ask questions and engage in dialog with the researcher to improve their understanding. Then the participant was given the opportunity to assemble the circuit without the help of the researcher. Lastly, they took a multiple-choice quiz on what they learned during the session. Participants were invited back two weeks later to attempt the circuit assembly and quiz again and see how much information they retained.

Variables

The experiment has two independent variables which changing depending on which technology is used during the experiment. The first is “agency,” which is the extent to which the participant has control over their point of view in the environment, as well as their ability to alter the state of the environment. The second is “embodiment,” which is how realistically the participant’s body is represented in the environment. The VR condition represents high agency, but low embodiment, while the Zoom condition has low agency, and high embodiment.

Since the dependent variable “collaboration” cannot be measured directly, the team used existing literature to determine several factors that can impact collaboration. These are then measured during the experiment to determine which are affected by the independent variables. A full list of the dependent variables can be found in the table below.

Table listing dependent variables used to measure collaboration, including: Co-presence, engagement, knowledge, mental workload, performance, rapport, shared situational awareness, trust, and usability.

Data Analysis Plan

Once data collection is complete, the team plans to analyze the data by conducting analysis of variance between each of the three condition groups over each of the dependent variables.

Future Work

After the experiment and data analysis are complete, the SCOTTIE team will use the results to develop thresholds to determine when business travel is really necessary for effective collaboration. The thresholds will be used to create a web tool that companies can use to make travel decisions, ultimately saving money and their impact on the environment. For additional information about the SCOTTIE project, please visit https://sites.google.com/iastate.edu/scottie/home

What's next?