AASS Seminar - Bidirectional intent communication with a conversational interface

The research centre AASS arranges a seminar with Jens Rüppel, TU Chemnitz, DE.

Remote attendance: https://oru-se.zoom.us/j/65096623293

Abstract

While assistive robots are becoming an integral part of many different areas in modern day's lives, examining feasible approaches towards a natural interaction between humans and robots is crucial. In recent years, Large Language Models (LLMs) have made a huge leap in domains like Natural Language Processing or Reasoning and may enhance collaboration in a variety of different use cases with promising possibilities and chances using multimodal inputs. In this presentation, I will give an overview of the multi modal human-robot interface I am currently working on for my Master's Thesis. The interface enhances the functionality of current approaches and seamlessly integrate a comprehensive LLM reasoning module into a responsive human-robot interface using video and audio inputs, as well as gaze inputs from an eye tracker. As an addition, I will address possible validation approaches using the example of a conducted experiment in September.