CIC’s work on STARS, a neurosymbolic approach that mines world knowledge from an LLM when a Soar agent is learning new tasks, has been published in Frontiers in Computer Science.
The paper describes STARS, the system we introduced in a AAAI24 paper, and our latest extensions. STARS allows an agent to use its understanding of its current situation (e.g., a visual scene) to pose questions to an LLM to identify potential goals, tasks, and object affordances. The agent tests the responses it gains from the LLM via grounding to the scene, integration with its existing knowledge, and mental simulation (aka “world modeling”).
If/when LLM-derived information passes those tests, the agent then uses it to inform its task learning, acting in the world to perform tasks. It also incorporates that new information in its long-term memories so that, e.g., once the agent has learned where a cleaned mug should be stored in this kitchen, it never asks the LLM again for that information (a kind of one-shot learning from experience).
If the information from the LLM doesn’t pass the tests, the agent does not attempt to use it. This approach then mitigates the challenges of hallucination and ignoring guardrails that direct use of an LLM imposes on agentic applications.