Pentagon rewards for detecting bias in artificial intelligence (AI) models – key information

• The U.S. Department of Defense has announced a reward program to detect legal bias in artificial intelligence (AI)
• The competition, open to the public until February 27, focuses on finding realistic examples of bias in Meta’s LLama-2 70B large language model.
• The Pentagon is looking for examples of legal bias against protected groups of people in an effort to eliminate bias in AI language models.
• The bias award is a competition where the top three submissions will split $24,000 and each participant will receive $250, judged on five categories.
• The Pentagon stresses that this is the first of two planned „bias awards” demonstrating their commitment to eliminating bias in AI.

Pentagon invites cooperation against AI!

The U.S. Department of Defense (DoD) has announced a new awards program aimed at detecting legal bias in artificial intelligence (AI) models. The competition, open to the public until February 27, focuses on trying to find real-world examples of bias in Meta’s large LLama-2 70B language model. The DoD is looking for situations in which language models may exhibit bias or erroneous results in the context of the Department of Defense, particularly against protected groups of people.

Finding realistic examples of bias

The goal of the competition is to identify realistic situations with potential real-world applications where language models may exhibit bias. Although the Pentagon does not specify what groups are protected, the explanation indicates that examples of legal bias against specific groups of people are sought. The example presented in the video illustrates an AI model responding with bias to medical questions related to different groups, highlighting the need to eliminate bias in such systems.

Competition and rewards

The bias award will not be paid on a case-by-case basis, but will be awarded on a competitive basis. The top three submissions will share a prize pool of $24,000, with each participant receiving $250. A rubric will evaluate the submissions, including five categories such as realism of the scenario, relevance to the protected group, evidence provided, concise description and number of clues needed for replication. The Pentagon stresses that this is the first of two planned „bias awards” as part of their efforts in this field.