T. Ji et al.: Multi-Modal Anomaly Detection for Unstructured and Uncertain Environments

Bio Information

Tianchen Ji is currently working towards the Ph.D. degree with the Department of Electrical and Computer Engineering at the University of Illinois at Urbana-Champaign. His research interests include safe robot learning and optimization based control in uncertain real-world environments. He received the B.Eng. degree in electrical engineering from Xi’an Jiaotong University.


  • Sri Theja Vuppala is a research engineer at Field Robotics Engineering and Science Hub (FRESH) in the Department of Agriculture and Biological Engineering at the University of Illinois at Urbana-Champaign. His research interests include design, testing, validation of the robotic platforms, and autonomous systems. He is interested in developing and deploying robot swarms. He earned his Master of Science degree in Mechanical and Aerospace Engineering focusing on Control systems on Unmanned Aerial Systems. He is also an FAA-certified Remote Pilot in Command.
  • Girish Chowdhary is an associate professor and Donald Biggar Willet Faculty Fellow at the University of Illinois at Urbana-Champaign. He is the director of the Field Robotics Engineering and Science Hub (FRESH) at UIUC, the Chief Scientist on the Illinois Autonomous Farm, and associate director of research in the AIFARMS national AI institute . Girish is affiliated with Agricultural and Biological Engineering, Coordinated Science Lab, Computer Science, Electrical Engineering, and Aerospace Engineering at UIUC.
  • Katie Driggs-Campbell is currently an assistant professor at the University of Illinois at Urbana-Champaign in the Department of Electrical and Computer Engineering. She received a BSE with honors from Arizona State University and an MS/PhD from UC Berkeley in Electrical Engineering and Computer Sciences. She runs the Human-Centered Autonomy Lab, which focuses on developing safe and interactive autonomous systems, merging ideas robotics, learning, human factors, and control.

Presentation Abstract

To achieve high-levels of autonomy, modern robots require the ability to detect and recover from anomalies and failures with minimal human supervision. Multi-modal sensor signals could provide more information for such anomaly detection tasks; however, the fusion of high-dimensional and heterogeneous sensor modalities remains a challenging problem. We propose a deep learning neural network: supervised variational autoencoder (SVAE), for failure identification in unstructured and uncertain environments.

Our model leverages the representational power of VAE to extract robust features from high-dimensional inputs for supervised learning tasks. The training objective unifies the generative model and the discriminative model, thus making the learning a one-stage procedure. Our experiments on real field robot data demonstrate superior failure identification performance than baseline methods, and that our model learns interpretable representations.

Authors: Tianchen Ji, Sri Theja Vuppala, Girish Chowdhary and Katherine Rose Driggs-Campbell


Ask the Speaker

You can post your questions for the speaker of this talk here until Nov 8. Your questions are visible to all other registered participants of the conference and will be visible together with your username that consists of the beginning of your email address. Participants can “like” your question. Questions with more like are more likely to be discussed during the live Q&A session on November 10, 2020.