The International Civil Aviation Organization (ICAO) monitors whether Contracting States conform to its Standards and Recommended Practices (SARPs) through the Universal Safety Oversight Audit Programme Continuous Monitoring Approach (USOAP-CMA), and each State must report its differences through the Electronic Filing of Differences (EFOD) system. The gap analysis behind these reports has depended on experts working by hand through a ten-step process that includes Korean translation, searching for corresponding provisions, comparison and mapping, and deciding on differences with written justification. Searching is the main bottleneck, because the Korean basis for a single ICAO provision is often spread across acts, administrative rules and annexed tables. This places structural limits on keeping the analysis current as more legislation comes into scope and amendments call for continuous review.
Large language models (LLMs) are increasingly used for regulatory analysis, but in aviation regulation, hallucination rates of 15 to 25 percent and a lack of source traceability are cited as major obstacles to audit readiness [1]. This study automates the entire gap analysis between ICAO reference documents and Korean legislation with a multi-agent system. It also proposes a multi-layer reliability management framework that works without a prepared expert gold set, along with uncertainty-based human-in-the-loop gates. A pilot validation showed that the results the system confirmed automatically were in close agreement with expert judgments.
- Regulatory gap analysis
- Multi-agent system
- Expert review
