PlanCollabNL: Leveraging Large Language Models for Adaptive Plan Generation in Human-Robot Collaboration

Silvia Izquierdo Badiola, Gerard Canal, Carlos Rizzo, Guillem Alenyà

Abstract: “Hey, robot. Let's tidy up the kitchen. By the way, I have back pain today". How can a robotic system devise a shared plan with an appropriate task allocation from this abstract goal and agent condition? Classical AI task planning has been explored for this purpose, but it involves a tedious definition of an inflexible planning problem. Large Language Models (LLMs) have shown promising generalisation capabilities in robotics decision-making through knowledge extraction from Natural Language (NL). However, the translation of NL information into constrained robotics domains remains a challenge. In this paper, we use LLMs as translators between NL information and a structured AI task planning problem, targeting human-robot collaborative plans. The LLM generates information that is encoded in the planning problem, including specific subgoals derived from an NL abstract goal, as well as recommendations for subgoal allocation based on NL agent conditions. The framework, PlanCollabNL, is evaluated for a number of goals and agent conditions, and the results show that correct and executable plans are found in most cases. With this framework, we intend to add flexibility and generalisation to HRC plan generation, eliminating the need for a manual and laborious definition of restricted planning problems and agent models.
Paper: To be added.

PlanCollabNL: an LLM is used to generate and filter subgoals from an abstract goal and the current environment (1), and later to reason about the subgoal allocation among the agents, favouring or disfavouring subgoals for each agent based on their conditions (2). This information is encoded into the PDDL planning problem, translating the LLM output subgoals into PDDL goals, and modifying the action costs based on the LLM recommendations. The complete planning problem is given to a planner to produce grounded and executable collaborative plans that consider the agent states (3).


Planning subgoals generation from an abstract NL goal

The dataset used to evaluate the generation of planning subgoals from an abstract NL goal can be found here: Open csv file.

Subgoal allocation reasoning based on NL agent conditions

The dataset used to evaluate the plan generation with appropriate subgoal allocation in the plan based on NL agent conditions can be found here: Open csv file.

Planning Domain and Problem

The PDDL planning domain and problems defined for the system can be found here: Open domain file. Open problem file.

Note that the domain is fixed for all scenarios in the application. The problem goal and the functions related to the action costs are determined by the LLM for each specific scenario.

LLM Prompts

The prompts designed for the system can be found here. We utilise n-shot learning, where the prompt includes n previous examples. For each new scenario, the templated query (found after the examples) is completed by the system with the current situation.

Planning subgoals generation from an abstract NL goal

Subgoal generation prompt
Goal: {goal}
Predicates: {predicates}
Objects: {objects}
Locations: {locations}

Subgoal filter prompt - common-sense
A {object} is {predicate} in {location}. Does it make sense?

Subgoal filter prompt - goal contribution
The goal is: {goal}. Would the following subgoal contribute to the goal? Answer yes or no.

Subgoal allocation reasoning based on NL agent conditions

Subgoal favour/disfavour prompt
I have the following tasks:
The agent {condition}.
The agent should {favour/disfavour} these tasks:


Coming soon.