Kaggle Launches $200K Hackathon to Measure Progress Towards AGI

Kaggle, a subsidiary of Google, has launched a new hackathon – “Measuring progress toward AGI: Cognitive abilities” – with a total prize pool of $200,000. The competition aims to benchmark artificial intelligence (AI) system performance against human capabilities across key cognitive abilities.

The Challenge: Evaluating AI Cognitive Abilities

The hackathon focuses on five cognitive abilities where evaluation gaps are currently the largest: learning, metacognition, attention, executive functions, and social cognition. Participants are encouraged to design evaluations for these areas, utilizing Kaggle’s newly launched Community Benchmarks platform to test their assessments against a range of advanced AI models.

Three-Stage Evaluation Protocol

The initiative is built around a three-stage evaluation protocol designed to assess AI systems in relation to human cognitive performance. This protocol involves:

• Evaluating AI systems across a broad suite of cognitive tasks for each ability, using held-out test sets to ensure data integrity.
• Collecting human baseline performance data for the same tasks from a demographically representative sample of adults.
• Mapping each AI system’s performance relative to the distribution of human performance in each cognitive ability.

Prize Distribution

The $200,000 prize pool will be distributed as follows:

• $10,000 awards for the top two submissions in each of the five tracks (learning, metacognition, attention, executive functions, and social cognition).
• $25,000 grand prizes for the four best overall submissions.

Key Dates

Submissions are open from March 17 through April 16, 2026, with results announced on June 1, 2026. Interested participants can find more information and begin building their evaluations on the Kaggle website.

The Importance of Human-AI Collaboration Evaluation

Evaluating Human-AI Collaboration (HAIC) is a complex undertaking due to the dynamic interactions between humans and AI systems. Effective HAIC depends on system performance, the quality of human-AI interaction, trust, and adaptability. Recent research highlights the need for tailored evaluation approaches, particularly for creative and linguistic AI applications like large language models and generative AI Evaluating Human-AI Collaboration: A Review and Methodological Framework.

assessing human-AI systems requires evaluating both their intrinsic capabilities and performance in real-world scenarios SPHERE: An Evaluation Card for Human-AI Systems. Understanding AI’s impact on workers also necessitates careful human-centered evaluations Nurturing Capabilities: Unpacking the Gap in Human-Centered Evaluations.

This Kaggle hackathon represents a significant step towards developing more robust and meaningful benchmarks for AI progress, ultimately fostering more effective and trustworthy human-AI collaboration.