news | 15 November 2021

DIU Operationalizes Responsible AI Guidelines in Practice

Lessons Learned Applying Guidelines to Commercial Prototyping and Acquisition Programs.

MOUNTAIN VIEW, CA--The Defense Innovation Unit (DIU) today released initial Responsible AI Guidelines for integrating  DoD’s Ethical Principles for AI into DIU’s commercial prototyping and acquisition programs. This effort was  launched in March 2020 to draw upon the best practices from government, non-profits, academia and industry partners to make actionable guidelines for each phase of AI software development.  

"DIU’s RAI Guidelines provide a step-by-step framework for AI companies, DoD stakeholders, and program managers that can help to ensure that AI programs align with the DoD’s Ethical Principles for AI and that fairness, accountability and transparency are considered at each step in the development cycle of an AI system," said Dr. Jared Dunnmon, technical director of the AI/ML portfolio at DIU

DIU has been working to implement these ideas in several of its active AI prototype projects as a way to operationalize the DoD AI Ethical Principles. 

This work has:

  • accelerated programs by clarifying end goals and roles, aligning expectations, and acknowledging risks and trade-offs from the outset​;

  • increased confidence that AI systems are developed, tested, and vetted with highest standards of fairness, accountability, and transparency in mind​;

  • supported changes in the way AI technologies are evaluated, selected, prototyped and adopted​ and helped avoid potential bad outcomes; and

  • provoked and surfaced questions which have spurred conversations that are crucial for AI project success​.

Whether one is focused on developing AI solutions for national security applications or is eager to introduce AI based products within an organization, this report provides a perspective on and examples of how responsible AI considerations can be put into practice on real-world programs. 

"Users want to know that they can trust and verify that their tools protect American interests without compromising our collective values," said Quantifind co-founder John Stockton. Quantifind was one of the companies that provided DIU feedback on the guidelines during the prototype project. "These guidelines show promise for actually accelerating technology adoption, as it helps identify and get ahead of potentially show-stopping issues. We’ve found that leaning into this effort has also served us well outside of government, by strengthening internal controls and producing transparency and patterns of trust that can also be leveraged with all users, both public and private."

The RAI Guidelines were drafted by members of DIU’s AI/ML Portfolio in collaboration with researchers at the Carnegie Mellon University Software Engineering Institute, and incorporate material, insight, and feedback from partners in government, industry, and academia.