UCLA Engineers Part of Avatar Team Advancing to $10M XPRIZE Semifinals

A multidisciplinary team developing an immersive avatar system that enables remote human connection has advanced to the semifinals for the $10 million ANA Avatar XPRIZE competition.

XPRIZE announced on April 5 that 38 teams from 16 countries were selected out of 77 qualified contestants, including the Human Fusions team composed of engineers from the UCLA Samueli School of Engineering and researchers from four other U.S. universities.

Led by Case Western Reserve University (CWRU) biomedical engineers, the Human Fusions avatar’s core technology is designed to allow humans to be physically in one place and experientially in another. The team entered its trademarked NeuroReality platform and the UCLA-built, health care Sensa avatar for the four-year competition launched in 2018.

Veronica Santos, a professor of mechanical and aerospace engineering at UCLA Samueli, leads the team in the development of the mechatronics systems for NeuroReality and the embedded human experience. She is joined by her graduate students Lionel Zhang, Yi Zheng, Huajing Zhao and Eric Peltola, as well as undergraduate student Cole Ten.

By directly connecting the human experience to that of an avatar, the robot becomes an extension of human intelligence.

Sensa operates on a Stretch robotic platform built by HelloRobot, which has an arm that can move in three dimensions with a mobile base. The avatar has a gripper that can open and close with a movable wrist that can roll, pitch and yaw. There are two MIT-built optical-tactile sensors located on either side of the gripper, which allow the human operator to feel the pressure exerted by the gripper when it grasps an object. The avatar also has a movable head with a built-in camera that can tilt and pan.

The human operator wears a virtual reality headset that provides audio and video feed from the avatar in real time. The operator controls the avatar’s movement and gripper by moving their own arm in space and by opening and closing their own hand, respectively. Through a NeuroReality haptic glove, the operator is able to receive tactile feedback directly from the robot. The glove utilizes non-invasive neural stimulation to produce sensation on the human thumb and index finger. The intensity of the touch on each finger corresponds with the pressure exerted on the respective sensor in the robot gripper.

“Imagine a mother being able to hold the hand of her child in quarantine with a real sense of touch,” said Santos who directs UCLA’s Biomechatronics Lab. “The dream is to transcend the use of robots as remotely operated tools and to enable people to experience robots as parts of their own bodies, even if miles away. The sense of touch is a critical part of the multisensory integration that will enable this feat.”

Sponsored by All Nippon Airways, Japan’s largest airline, the avatar XPRIZE competitors are challenged to create transformative avatar technology that can transport a human’s senses, actions and presence to a remote location in real time.

In February, the qualified teams completed their second round of technical submissions for evaluation by a panel of judges, including a detailed write-up of their avatar system’s capabilities and a video demonstration of their technology in action.

The Human Fusions team demonstrated a visit from a human operator from CWRU in Cleveland, Ohio, through the Sensa avatar, with a recipient from UCLA in Los Angeles, California. During the demo, the recipient and the operator were able to shake hands and did a celebratory champagne toast through the Sensa avatar. The Human Fusions avatar has safety constraints placed on the robot gripper, so a recipient will not be harmed by the robot’s grip force. The avatar can also allow the recipient and the operator to work collaboratively on shared tasks.

For the semifinal round in September 2021, the teams will compete in a live demonstration in Miami, Florida. Up to 20 finalists will be selected and share a $2 million prize equally. For the finals, which will begin in early summer 2022, there will be performance tests to showcase the application of avatar technologies to perform specific tasks across several use-case scenarios. An $8 million grand prize will be announced in June 2022, and awarded to the top three-ranked teams, with $5 million for the first place, $2 million for the runner-up and $1 million for the third place.

The Human Fusions team is part of the multi-university Human Fusions Institute led by Dustin Tyler, the Kent H. Smith II Professor of Biomedical Engineering at CWRU’s School of Engineering. In addition to faculty and students from CWRU and UCLA, the team also includes researchers from Carnegie Mellon University, Cleveland State University and the University of Wyoming.

Santos is the associate dean for diversity, equity and inclusion and faculty affairs at UCLA Samueli. She also holds an affiliate faculty appointment in the Bioengineering Department.