Accelerating Software Acquisition Using Generative AI for Regulatory Compliance

Abstract

"Detecting document incompleteness, inconsistencies, and discrepancies between regulatory documents and software artifacts is a common and people-intensive task for acquisition teams. Department of Defense (DoD) Acquisition environments have extensive documentation describing policies, guidance, and standards that must be repeatedly compared to delivered software artifacts for a DoD program to ensure regulatory conformance throughout a project’s lifecycle. Acquisition professionals in these environments must learn the extensive and complex regulatory information, apply the knowledge to multiple projects, and identify document incompleteness, inconsistencies, and discrepancies (DIID) that could indicate non-compliance or high-risk areas. Currently, teams of people review multitudes of documents and data, reading and using general search on keywords to find relevant text to review and compare to regulatory documents. As the DoD continues moves toward DevSecOps with continuous integration and rapid capability deployment approaches, people-intensive approaches to ensure regulatory compliance are slow, do not scale, and delay mission capability. This paper investigates the use of large language models (LLMs) to improve the efficiency and accuracy of DIID detection while enabling customization through prompt engineering. The proposed approach leverages LLMs to augment acquisition professionals by providing semi-automated and meaningful connections of software artifacts to regulatory documents. Testing approaches are proposed to assess the effectiveness of LLMs for DIID detection, and preliminary results are provided for detecting DIID with augmented LLMs. This paper also proposes prompt engineering approaches for DIID detection and suggests benefits for DIID detection in software acquisition activities. "