TBNAI
What Does an Ontology Catch? Curated, LLM-Generated and Data-Derived Ontologies as Checkers of LLM Agents over Knowledge Graphs
Publication: cs.AI updates on arXiv.org
Author: Michal Orzechowski, Rund Tawfiq, Tomasz Kosciolek
Source published: 30/10/2026, 04:00:00 UTC
Briefing created: 30/10/2026, 12:03:32 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 30/10/2026, 06:48:24 UTC.
Briefings
A study of LLM agents using knowledge graphs reveals that curated ontologies as checkers often miss errors and falsely flag correct answers. On UniProt, the curated ontology caught more correct answers than data-derived or LLM-derived ontologies, but on real agent answers, its flags were only marginally more precise than random guesses. The research compares three ontology sources: curated schemas, LLM-generated ontologies, and data-derived ontologies across public and private knowledge graphs.
Key points
- On UniProt, the curated ontology caught more correct answers than either an ontology derived from the data or one an LLM derives from a summary of the data with masked identifiers.
- On real agent answers, the curated ontology's flags were only marginally more precise than random flags.
Context
This research investigates whether formal ontologies serve as reliable judges for LLM agents answering questions over Web knowledge graphs. The study compares three ontology sources: curated schemas, LLM-generated ontologies, and data-derived ontologies across public and private knowledge graphs.
Limitations
The study covers public graphs and a private graph of nearly 10^9 triples. It accounts for what ontologies catch versus what they wrongly flag, but does not provide causal claims or forecasts on whether combining sources or giving the agent the checker's specific feedback pays off. Results are limited to the specific ontology types and graph types tested.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 30/10/2026, 12:03:32 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeiedmbvxhfws3a5xj36oicvlwaerp4pcs3qyfxsujmnxgnbll4f2qa