AI

MARQ: A Modular, LLM-Agnostic Requirement Quality Assessor Infused with Systems Engineering Knowledge

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 28/10/2026, 04:03:25 UTC.

Briefings

A new study introduces MARQ, a modular LLM-agnostic system that integrates Systems Engineering (SE) knowledge into requirement quality assessment. Using an expert-labeled NASA requirement set, MARQ outperformed off-the-shelf baselines on median F1 scores, with Claude Opus 5 showing the most significant improvement in defect detection and false-alarm control. The study highlights that performance gains are highly model-dependent, as GPT-5.6 Sol improved detection and consistency but increased false alarms, while Gemma 3 12B matched proprietary models but remained highly variable. These findings suggest SE knowledge infusion is a promising pathway for trustworthy LLM-based RE support, emphasizing the need for criterion-specific interventions and human-in-the-loop review.

Key points

• MARQ is a modular, LLM-agnostic prototype that supplements LLMs with a criterion-specific knowledge stack operationalizing INCOSE requirement quality criteria.
• On the NASA requirement set, MARQ achieved median F1 scores of 0.80 for Claude Opus 5, 0.62 for GPT-5.6 Sol, and 0.53 for Gemma 3 12B, significantly outperforming off-the-shelf baselines of 0.55, 0.52, and 0.18 respectively.

Context

Requirements engineering is foundational to systems engineering, as requirement defects can propagate downstream and adversely affect development outcomes. This study investigates whether infusing expert-derived SE knowledge can improve LLM-based requirement quality evaluation beyond off-the-shelf capability.

Limitations

Performance gains are model-dependent, with specific outcomes varying across different LLMs. The study utilized an expert-labeled NASA requirement set, and the nature of gains is not uniform across all criteria or models. The evidence is from a preprint (arXiv:2610.36234v1) and represents a single snapshot of execution results with 100 executions per model.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 28/10/2026, 04:59:24 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeic7mpa3h5bapozqozhbs22vlzgtuwhmo33stfmrgdfrkessb5xfou