AI
Build2SPARQL: A Large-Scale Text-to-SPARQL Benchmark Dataset for Building Knowledge Graph Querying
Publication: cs.AI updates on arXiv.org
Author: Wooyoung Jung
Source published: 02/10/2026, 04:00:00 UTC
Briefing created: 02/10/2026, 04:04:45 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 02/10/2026, 04:03:25 UTC.
Briefings
This article introduces Build2SPARQL, a large-scale benchmark dataset for building knowledge graph query generation. The dataset was created using a KG-grounded pipeline where SPARQL queries are generated by graph-traversal code, while natural language questions are generated by large language models. The pipeline mines six query-pattern families across five vocabulary registers, resulting in 6,136 executable SPARQL queries and 30,680 questions. A two-rater human validation confirmed high semantic fidelity and naturalness, while a retrieval-augmented evaluation showed significant improvement in exact-match accuracy when using retrieved context compared to zero-shot prompting.
Key points
- The Build2SPARQL benchmark consists of 6,136 executable SPARQL queries and 30,680 natural language questions derived from 201 building knowledge graphs.
- Query generation relies on graph-traversal code for SPARQL queries and large language models for natural language questions, ensuring query correctness is independent of model behavior.
Context and limitations
The dataset covers only 201 building knowledge graphs from two specific ontologies (Brick and ASHRAE 223P). Validation results are limited to a two-rater human study and three open-weight language models, with no mention of performance on other models or real-world operational scenarios.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 02/10/2026, 04:04:45 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeigceqas3otug7dvdsnfq2axmohq2uvg4ocuzowrwsxvfgjyfcie5y