Evaluating SBIR Proposals: A Comparative Analysis using Artificial Intelligence and Statistical Programming in the DoD Acquisitions Process

Abstract

The Small Business Innovation Research (SBIR) program is a tool that the Department of Defense (DOD) uses to encourage industry development in technology that the market is otherwise not demanding. This helps to drive innovation and facilitate competition for government contracts. However, the source selection process within the SBIR program could be improved. It currently takes too long and is riddled with inconsistencies. Given this application and the rising interest in artificial intelligence (AI), it is worth exploring ways to augment the source selection process with AI. This study assesses the effectiveness of using large language models (LLMs) to automate classification of acquisition proposals as either competitive or noncompetitive. This study used R to extract text from the proposals, interact with OpenAI’s models, and then iteratively loop through all of the proposals until completion. The intent was to establish a faster, more consistent, and objective evaluation system when compared to subjective human assessments. The final analysis indicated an emerging capability with vast potential, but one that is not reliable enough for immediate application into the SBIR program. This study emphasizes the importance of accuracy and reliability in DOD’s initiatives and highlights the potential roles of AI in optimizing DOD acquisitions.