State-Based Dynamic Graph with Breadth First Progression For Autonomous Robots

• Published in: 2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC) pp. 0365 - 0369
• Main Authors: Chugh, Tushar, Tyagi, Kanishka, Srinivasan, Pranesh, Challagundla, Jeshwanth
• Format: Conference Proceeding
• Language: English
• Published: IEEE 08.01.2024
• Subjects: Adaptation models; agent-based approach; autonomous operations; Decision making; graph framework; large Language Models; mixture of experts; Pattern recognition; Robots; Task analysis; Technological innovation; Vehicle dynamics
• DOI: 10.1109/CCWC60891.2024.10427646

This paper introduces a novel method for enhancing robotic systems using Large Language Models (LLMs). We focus on leveraging LLMs to substantially improve robots' decision-making and interaction with their environment. Our proposed framework employs an agent-based approach, where robots utilize LLMs for sophisticated pattern recognition, environmental understanding, and autonomous decision-making. The main innovation of this research is the integration of LLMs into a robotic system, enabling robots to process large volumes of unstructured data, recognize complex patterns, and make informed decisions with increased precision and efficiency. This integration marks a significant leap in robotic cognitive abilities, surpassing the constraints of traditional programming. Our methodology transforms LLMs into dynamic elements within robotic systems, fostering enhanced and responsive interactions with the environment. Robots equipped with LLMs are thus capable of advanced autonomous operations, including navigating complex environments, solving intricate problems, and interacting more naturally with humans. The primary contribution of this work is the creation of an agent-based graph framework, designed to facilitate collaborative problem-solving in robotic systems. This framework consists of multiple agents working collaboratively, spanning from data ingestion to comprehensive world understanding and decision-making. These agents include modules responsible for various operational aspects, such as environmental analysis, data processing, and specialized LLMs for data interpretation and summarization. Positioning LLMs as proactive, inquisitive agents, our approach enables them to actively seek information and efficiently collaborate with other agents to complete tasks. The dynamic nature of this graph search and inter-agent communication model is a considerable innovation in robotics, offering a more integrated and effective approach for robots to tackle complex tasks, thereby enhancing their ability to operate autonomously and intelligently in diverse environments.