# AI visual reasoning: how AI solves visual puzzles, and where it still fails

> Visual abstract reasoning is inferring a rule from a few pictures. How AI is tested on it, the methods that work, and what studies of multimodal models found.

- URL: https://computese.com/ai-new-frontier-solving-visual-puzzles/
- Author: Duong Quan Nguyen, CEO, Computese
- Published: 2024-10-14
- Updated: 2026-09-25
- Topics: AI & automation

## In short
- Visual abstract reasoning is inferring a rule from a few pictures and applying it to a new one. Raven's matrices, Bongard problems and ARC-AGI grids test it, and each needs exact perception as well as the right rule.
- In the USC study behind the October 2024 headlines (COLM 2024), no multimodal model beat chance consistently across three sets of Raven-style puzzles, and GPT-4V was right, with the right reasoning, on 26% of one set.
- Models fail at two separate steps: reading exact detail such as rotation, overlap or position, and reasoning from it. Some still failed when the puzzle was described to them in text.
- What works best turns pictures into symbols: program search checked against the examples, small networks trained at test time, and reasoning models that read grids as numbers.
- As of September 2026, ARC Prize verified 95.0% on ARC-AGI-2 for OpenAI's GPT-6 Astra. Scores still depend on how the puzzle is presented and on the harness, so test models on your own images.

Visual abstract reasoning is working out a rule from a few pictures and applying it to a new one, as Raven's Progressive Matrices, Bongard problems and ARC-AGI grids test. AI now solves a growing share of these puzzles, mostly after turning pictures into symbols, but published tests keep finding errors in how models read the pictures themselves.

The subject made headlines in October 2024, when researchers at the University of Southern California's Information Sciences Institute (ISI) asked whether multimodal AI could solve the visual puzzles of human IQ tests. This explainer covers what visual abstract reasoning is, how it is tested, the three ways machines approach it, what that study and later ones found, and where things stand in 2026 according to the ARC Prize Foundation's published results. For ARC-AGI as a test of reasoning in general, and for reasoning models, see our guide to [logical AI](https://computese.com/exploring-the-new-frontier-in-ai-logical/).

## What visual abstract reasoning is

A visual abstract reasoning puzzle shows a handful of images that share a hidden rule and asks you to apply that rule to a case you have not seen. The rule is abstract, such as "each row adds one line", "the shape slides until it touches a wall" or "everything on the left is convex". Nothing in the puzzle names it, and the puzzle is new, so a memorized answer does not help.

Psychologists use nonverbal puzzles like these to measure fluid intelligence, the ability to reason about novel problems, as opposed to crystallized intelligence, which draws on stored knowledge. AI researchers use them for a related reason. In [On the Measure of Intelligence](https://arxiv.org/abs/1911.01547) (2019), François Chollet argued that skill at any one task can be bought with unlimited training data or built-in knowledge, and that intelligence is better measured as skill-acquisition efficiency: how well a system learns something new from little experience.

Solving a visual puzzle takes two different abilities, and a system needs both:

- **Perception:** reading the images exactly. Which shapes are there, in what colour, how many, where, rotated how far, touching or not.
- **Abstraction and induction:** finding the one rule that fits every example, then applying it without a slip.

That is different from the image understanding most AI benchmarks test. Recognizing a dog, reading a chart or answering a college exam question about a diagram, as in the [MMMU benchmark](https://computese.com/google-unveils-gemini-the-most-advanced-and-versatile-ai-model-yet/), relies on knowledge learned in training. An abstract puzzle is built so that such knowledge does not help. Chollet designed ARC around the priors that the developmental theory of Core Knowledge says humans are born with: objectness, goal-directedness, numbers and counting, and basic geometry and topology.

## How visual reasoning is tested: matrices, Bongard problems and ARC grids

Three families of puzzles dominate the research. They differ in what the solver sees, what it must produce and how it can cheat.

### Raven's Progressive Matrices

The psychologist John Raven introduced his Progressive Matrices in the 1930s. A problem is a three-by-three grid of panels with the bottom-right panel missing. Shapes change along the rows and columns according to rules (size grows, a line is added, one shape is the combination of two others), and the solver picks the panel that completes the pattern from a set of candidates, usually eight.

![A three-by-three grid of panels in which shapes grow along each row, with the bottom-right panel empty, and a row of eight answer panels below; an orange square is carried up into the gap.](https://computese.com/images/blog/ai-new-frontier-solving-visual-puzzles/matrix.f5937eb662-1536.webp)

*The rule is in the matrix, so a model that picks right by studying only the options is not reasoning at all.*

AI research generates such matrices by program, which gives large training sets and precise control over the rules:

- **PGM** (Procedurally Generated Matrices), from DeepMind researchers at ICML 2018 ([Barrett et al.](https://arxiv.org/abs/1807.04225)), was built to test generalization, with training and test sets that differ in controlled ways. Their Wild Relation Network (WReN) scored 62.6% when the test used the same kinds of rules as training, but 17.2% in the extrapolation regime, where colour and size values in the test lay above the range seen in training. Guessing among eight options gives 12.5%.
- **RAVEN** ([Zhang et al.](https://arxiv.org/abs/1903.02741), CVPR 2019) has 70,000 problems in seven figure configurations, with structural annotations for each.
- **I-RAVEN** ([Hu et al.](https://arxiv.org/abs/2002.06838), AAAI 2021) fixed a flaw in RAVEN: the correct panel tended to share the most common attributes of the eight options, so it could be found without looking at the matrix. A "context-blind" CoPINet model that saw only the eight options scored 94.2% on RAVEN. On the rebuilt I-RAVEN it scored 14.2%.

The I-RAVEN result applies well beyond RAVEN: a high score on a visual puzzle proves little until you know the model could not reach it by a shortcut.

### Bongard problems

More than fifty years ago, the Russian computer scientist Mikhail Bongard designed a collection of 100 visual puzzles. Each shows six drawings on one side that share a concept and six on the other side that violate it, and the solver has to state the concept: convex against concave, say, or figures on the left of the frame against figures on the right. There are no options to choose from; the answer is a rule.

[Bongard-LOGO](https://arxiv.org/abs/2010.00763) (NeurIPS 2020) generated 12,000 problems in this style from programs in the LOGO drawing language, and turned each into a yes-or-no question about new drawings, where chance is 50%. Human experts scored above 99% on its basic-shape problems and above 90% on the harder sets, while the best models of the time reached around 70% on basic shapes and at most about 65% elsewhere.

### ARC-AGI grids

ARC, the Abstraction and Reasoning Corpus that Chollet published in 2019 and now called ARC-AGI-1, uses coloured grids instead of drawings. Each task gives a few demonstration pairs, 3.3 on average, each an input grid and an output grid between 1×1 and 30×30 cells in up to 10 colours. The solver must build the output grid for a new input from scratch, choosing its size and every cell, and only an exact match counts. With no options to choose from, RAVEN's kind of answer leak cannot happen.

[ARC-AGI-2](https://arcprize.org/arc-agi/2) (2025) kept the format with harder tasks, and every task in its evaluation sets was solved by at least two people within two attempts, in testing with more than 400 members of the public. ARC-AGI-3 (2026) turns the grid into a game: according to its [game schema](https://docs.arcprize.org/game-schema), an agent receives frames of up to 64×64 cells with 16 possible values, takes actions and has to discover the goal by itself.

| Test                    | What the solver sees                                    | What it must produce                       | Known pitfall                                      |
| ----------------------- | ------------------------------------------------------- | ------------------------------------------ | -------------------------------------------------- |
| Raven-style matrices    | A 3×3 grid with one panel missing, plus about 8 options | The right option                           | The options can leak the answer (RAVEN)            |
| Bongard problems        | Six drawings that fit a concept, six that do not        | The concept, or a yes or no for new images | Open answers need a human or a model to judge them |
| ARC-AGI-1 and ARC-AGI-2 | A few pairs of input and output grids                   | The exact output grid                      | Brute-force search solves part of ARC-AGI-1        |
| ARC-AGI-3               | Game frames up to 64×64, with no instructions           | Actions that finish each level             | Scores depend heavily on the harness               |

## Three ways machines approach visual puzzles

Research systems fall into three families, and the strongest ARC entries combine them.

### Program synthesis and search

Program synthesis treats a puzzle as a search for a short program that turns every example input into its output. If a candidate reproduces all the demonstration pairs, it is run on the test input. The examples double as a free checker, which is the approach's great strength, and it needs no understanding of pictures beyond the grid itself.

It was the first approach to work on ARC. According to the [ARC Prize 2024 technical report](https://arxiv.org/abs/2412.04604), the winner of the first ARC competition on Kaggle in 2020, an entry by a participant known as icecuber, scored 20% on the private evaluation set with brute-force program search, and 49% of that set was solved by at least one 2020 team, all using some form of brute-force search. The classic form searches exhaustively through a domain-specific language (DSL) of grid operations, which runs into combinatorial explosion as programs grow, so later systems use deep learning to guide the search. In 2024, Ryan Greenblatt reached 42% on the public ARC-AGI leaderboard by having GPT-4o write thousands of candidate Python programs per task and debug the most promising ones.

![Pairs of small input and output grids feed a gear that produces many candidate programs. Every program fails the check except one orange program, which is applied to a new grid.](https://computese.com/images/blog/ai-new-frontier-solving-visual-puzzles/search.1dd758ce9b-1536.webp)

*Program search needs no eyes, only a fast way to propose candidates and a few examples to reject the wrong ones.*

In its [2025 results](https://arcprize.org/blog/arc-prize-2025-results-analysis), the ARC Prize Foundation named this pattern, which it calls a refinement loop, as the central theme of the year: generate many candidate solutions, check them against the examples, and keep refining until one fits every pair.

### Neural networks trained on puzzles

The second family learns to produce the answer directly. Early work trained networks on large generated sets, such as WReN on PGM and many later architectures on RAVEN. Their weakness, as PGM and I-RAVEN showed, is that they learn the statistics of their training data, shortcuts included, and generalize poorly beyond it.

ARC offers only a few hundred training tasks, so the current version of this idea trains at test time. In test-time training (TTT), a model is fine-tuned on the demonstration pairs of each task and then predicts the output grid directly, which the ARC Prize reports call transduction. Together with deep learning-guided program synthesis, it lifted the state of the art on the ARC-AGI-1 private set from 33% to 55.5% during ARC Prize 2024. Effective TTT systems often used transformers adapted to two-dimensional grids, and the top scores combined induction (a program) with transduction, because each solved different tasks.

2025 brought small, specialized networks. ARC Prize's first paper award went to the Tiny Recursive Model (TRM), a network of about 7 million parameters that repeatedly refines its own answer and scores about 45% on ARC-AGI-1 and 8% on ARC-AGI-2. The top Kaggle entry, NVARC, combined a test-time-trained model with TRM components and reached 24.03% on the ARC-AGI-2 private set at $0.20 per task.

One paper ARC Prize gave an honourable mention is directly about vision. [ARC Is a Vision Problem!](https://arxiv.org/abs/2511.14761) (November 2025), from MIT researchers including Kaiming He, notes that most ARC methods treat the grids as sequences of tokens, like language. It instead places each grid on a larger canvas and trains a standard Vision Transformer from scratch, on ARC data only, as image-to-image translation, with the random shifts and changes of scale that vision models usually train with. The 18-million-parameter model scored 54.5% on ARC-AGI-1, and an ensemble 60.4%, which the authors say matches reported average human performance.

### Multimodal language models

The third family uses general-purpose multimodal large language models (MLLMs), also called vision-language models: a vision encoder turns the image into features, and a language model reasons over them and writes the answer. They need no puzzle-specific training and can explain their answers, which makes them the obvious thing to try.

A useful baseline comes from text. When a matrix puzzle is written out as text, language models do well: in a study published in [Nature Human Behaviour](https://www.nature.com/articles/s41562-023-01659-w) in July 2023, Webb, Holyoak and Lu found that GPT-3 matched or surpassed human participants in most settings of a non-visual matrix task built on the rule structure of Raven's Standard Progressive Matrices. The open question was whether multimodal models could do the same from pixels.

## The October 2024 study: can multimodal AI solve Raven's matrices?

The study behind the October 2024 headlines is [The Curious Case of Nonverbal Abstract Reasoning with Multi-Modal Large Language Models](https://arxiv.org/abs/2401.12117), by Kian Ahrabian, Zhivar Sourati and colleagues at USC's Information Sciences Institute. It was presented at the first Conference on Language Modeling (COLM 2024) in Philadelphia on October 7 to 9, 2024, and [announced by ISI](https://www.isi.edu/news/72195/does-chatgpt-belong-in-mensa/) under the headline "Does ChatGPT Belong in Mensa?"

The team tested 24 open-source and closed-source MLLMs on three sets of Raven-style puzzles:

- **IQ50:** 50 puzzles collected from the internet, with six options each.
- **RAVEN-S:** 3,500 problems sampled from the 70,000 in RAVEN, with eight options each.
- **CCSE:** 175 problems from China's civil service examination, with four options each.

What they found:

1. **Near chance, at every size.** Scored automatically, no model consistently beat random guessing across the three sets, the best scores were within about 2 percentage points of always picking the most common answer, and larger models did no better than smaller ones.
2. **A wide gap between closed and open models.** Graded by hand, with the model asked to think step by step, GPT-4V gave the correct answer with correct reasoning on 26% of IQ50, Gemini Pro Vision on 10% and the best open-source model on 4%. A simple heuristic that compares image embeddings from CLIP scored 48% on the same set.
3. **Perception fails, and so does reasoning.** Models generally recognized the shapes but hallucinated details such as rotation, shadows and orientation, and they tended to describe the puzzle instead of reasoning about it. Given expert-written text descriptions in place of the images, most models still could not reason to the answer; only GPT-4V reached a high level.
4. **Guidance helps the strongest models.** General hints and puzzle-specific hints made results worse. A corrective hint aimed at the model's main mistake, or worked examples with step-by-step reasoning (chain-of-thought prompting), raised GPT-4V's accuracy by as much as 100%, which means doubling it in the best case. Open-source models could not make consistent use of examples.

The paper's conclusion is that the models fail at two separate steps, gathering precise visual details and reasoning correctly and faithfully from them, and that small puzzles a person finds easy are enough to expose both. The models tested were those of late 2023; the next two sections show what changed afterwards.

## Other studies found the same two weak spots

Studies from 2023 to 2025 tested other puzzle types and newer models, and they point the same way.

| Study                              | Published                         | Puzzles                           | Main result                                                          |
| ---------------------------------- | --------------------------------- | --------------------------------- | -------------------------------------------------------------------- |
| Mitchell, Palmarini and Moskvichev | November 2023, AAAI 2024 workshop | 48 very simple ARC-style tasks    | Humans 95%, GPT-4 given text 69%, GPT-4V given images 23% to 25%     |
| Ahrabian et al. (USC ISI)          | COLM, October 2024                | Raven-style matrices              | GPT-4V right, with right reasoning, on 26% of IQ50                   |
| Rahmanzadehgervi et al.            | arXiv, July 2024 (revised 2025)   | Seven simple geometry tasks       | Four vision-language models averaged 58.07%; the best scored 77.84%  |
| Wüst et al.                        | ICML 2025                         | The 100 original Bongard problems | OpenAI o1 solved 43 on average; the best human participant solved 68 |

**Images made GPT-4V worse, not better.** [Mitchell and colleagues](https://arxiv.org/abs/2311.09247) gave the same simple ConceptARC tasks to GPT-4 as text and to GPT-4V as images. Besides scoring far lower, GPT-4V could not consistently translate a visual grid into text, whether as colour names or numbers.

**The detail survives the encoder and is lost in words.** [Vision language models are blind](https://arxiv.org/abs/2407.06581) used tasks a person does without effort, such as whether two circles overlap or how many times two lines cross. The models reached near 100% only when the shapes were spaced further apart. Linear probes showed that the vision encoders held enough information to answer, and that the language models failed to decode it into correct answers.

![A drawing of overlapping circles and crossing lines enters a vision encoder and becomes a column of feature blocks; an orange funnel passes only a few to the language model, whose answer is marked wrong.](https://computese.com/images/blog/ai-new-frontier-solving-visual-puzzles/perception.04f02a2c12-1536.webp)

*The detail is often still in the image features; it gets lost when the language model has to put it into words.*

**A reasoning model did best on Bongard problems, and still fell well short of people.** In [Bongard in Wonderland](https://arxiv.org/abs/2410.19546), OpenAI's o1 solved the most problems, ahead of Claude 3.5 Sonnet with 31 and GPT-4o and Gemini 2.0 Flash with 25 each. On problems about spatial relations, every model solved under 25% and the top humans more than 60%. Nearly every problem was solved by at least one of the 20 participants, against 66 by any model, although o1 solved some that most people missed. Even simple concepts such as spirals caused trouble.

## Where AI visual reasoning stands in 2026

The ARC Prize Foundation's published results give the clearest recent picture, with one caution that matters for vision: most frontier scores on ARC-AGI-1 and ARC-AGI-2 are not produced from pictures. According to its [December 2025 analysis](https://arcprize.org/blog/arc-prize-2025-results-analysis), ARC Prize's verification harness gives language models the grids as arrays of integers without naming ARC or its colours, yet Gemini 3 Deep Think used ARC's correct colour mapping in its reasoning. The foundation concluded that ARC data is well represented in the models, and that ARC-AGI-1 and ARC-AGI-2 are being overfit through knowledge, incidentally or intentionally; it could not say by how much.

| Date             | Published result (ARC Prize Foundation)                                                                                                                                | Benchmark               |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
| December 5, 2025 | Top Kaggle entry NVARC 24.03%; top verified commercial model Claude Opus 4.5 (Thinking, 64k) 37.6%; Poetiq's refinement of Gemini 3 Pro 54%                            | ARC-AGI-2               |
| March 25, 2026   | ARC-AGI-3 launched: humans 100%, frontier AI 0.51%                                                                                                                     | ARC-AGI-3               |
| May 1, 2026      | GPT-5.5 0.43%, Claude Opus 4.7 0.18%                                                                                                                                   | ARC-AGI-3               |
| September 2026   | [GPT-6 Astra](https://arcprize.org/results/openai-gpt-6-astra) 95.0% on ARC-AGI-2; 62.7% on ARC-AGI-3 with the standard harness, 99.9% with a provider adapter harness | ARC-AGI-2 and ARC-AGI-3 |

ARC-AGI-3 is where perception came back into play. [Launched on March 25, 2026](https://arcprize.org/blog/arc-agi-3-launch), it consists of hundreds of turn-based environments with no instructions, rules or stated goals. ARC Prize's [May 2026 analysis](https://arcprize.org/blog/arc-agi-3-gpt-5-5-opus-4-7-analysis) of GPT-5.5 and Claude Opus 4.7 found that the models often saw the local effect of an action correctly but failed to turn it into a rule for the whole game. A surface resemblance also led them to treat an environment as a game they knew, such as Tetris, Sokoban or Breakout, and to waste actions testing mechanics that were not there.

The [first ARC-AGI-3 milestone prize](https://arcprize.org/blog/arc-prize-2026-milestone-1), awarded in July 2026, went to entries that look at the board. The winner, Tufa Labs' "The Duck", ran Qwen 3.6 27B locally and switched between a rendered image of the board, the raw grid as text and a segmentation tool for zooming in; the team credited its gains to multimodality and better base models. The second and third entries showed rendered frames to Gemma-4-31B and asked it for the next few actions as JSON.

Then, in September 2026, [OpenAI's GPT-6 Astra](https://arcprize.org/blog/astra) reached the scores in the table, at a cost of about $26,000 for the standard-harness run and $19,000 for the run with a harness that keeps the model's own reasoning state between requests. Its replays show it turning each environment into a compact symbolic model, with its own shorthand for objects, rules and plans: the picture-to-symbols step again. ARC Prize calls this meaningful progress, says it is not AGI, and notes that ARC-AGI-3's closed, deterministic games do not represent the open-endedness of the real world. The [ARC Prize 2026](https://arcprize.org/competitions/2026) competition, which requires open-source solutions that run without internet access, announces its results on December 4, 2026.

> [!IMPORTANT]
> Before comparing two visual reasoning scores, check how the puzzle reached the model (pixels, text or numbers), how many attempts it had, and which harness ran it. On ARC-AGI-3, switching harness raised GPT-6 Astra's best score from 62.7% to 99.9%.

## What this means if you use AI to read images

Few businesses need a model that solves IQ puzzles. Many want one that reads what is in a picture: a scanned form, a chart in a report, a floor plan, a screenshot of an error. The research above carries three practical lessons.

1. **Fluent is not accurate.** A model can describe an image confidently and still misread a rotation, an overlap or a count. Test it on your own images, including the awkward ones, before trusting what it says it sees.
2. **Turn pictures into data, then check the data.** The methods that work best turn the image into symbols that something else can verify. In a business system, that means extracting values into a structured record and validating them with rules, not acting on a free-text description.
3. **Compare models under your own conditions.** Hints, worked examples, the way a puzzle was presented and the harness all changed scores in the studies above. Evaluate candidate models with the prompts and setup you will actually run.

Our [AI and automation service](https://computese.com/services/ai-automation/) covers this kind of work: emails, forms, PDFs and web pages read by a model, turned into structured records and checked by rules before anything is written, with an evaluation set built from your real cases that every model or prompt change must pass before it goes live. For the wider question of what models do and do not understand, see [AI limitations in understanding](https://computese.com/ai-limitations-in-understanding/).

## Key terms
- **Visual abstract reasoning**: Working out a rule from a few images and applying it to a new case. It needs exact perception and correct induction, and psychologists use it to measure fluid intelligence.
- **Raven's Progressive Matrices**: A nonverbal intelligence test from the 1930s: a three-by-three grid of panels with one missing, completed by choosing among candidates. PGM, RAVEN and I-RAVEN are machine-generated versions.
- **Bongard problem**: A puzzle of twelve drawings, six that share a concept and six that do not. The solver has to state the concept. Mikhail Bongard published the original set of 100.
- **ARC-AGI**: The Abstraction and Reasoning Corpus: tasks on coloured grids, each learned from about three examples, where only an exact output grid counts. ARC-AGI-3 (2026) is an interactive, game-style version.
- **Multimodal large language model (MLLM)**: A language model that also accepts images, through a vision encoder whose output the language model reasons over. Also called a vision-language model (VLM).
- **Program synthesis**: Searching for a program that turns every example input into its output, then running it on the test input. The search can be brute force over a DSL or guided by a language model.
- **Test-time training**: Fine-tuning a model on a task's own demonstration examples just before it answers, so each task gets its own variant of the model.
- **Transduction**: Predicting the output directly from the examples and the test input, without writing an explicit program. Writing the program first is called induction.
- **Shortcut**: A pattern in a test that lets a model score well without doing the reasoning the test is meant to measure, such as guessing a matrix answer from the options alone.
- **Chain-of-thought prompting**: Asking a model to reason step by step, or showing it worked examples, before it gives its final answer.

## Common questions

### Can AI solve visual puzzles?

Many of them, with conditions. As of September 2026, ARC Prize verified 95.0% on ARC-AGI-2 for OpenAI's GPT-6 Astra, which reads the grids as numbers. From drawings the record is weaker: in an ICML 2025 study, the best model tested, OpenAI o1, solved 43 of 100 classic Bongard problems on average, while the best human participant solved 68.

### Can ChatGPT solve Raven's Progressive Matrices and other IQ test puzzles?

Partly. In the USC study presented at COLM 2024, GPT-4V gave the right answer with the right reasoning on 26% of a 50-puzzle set, far ahead of open-source models but below a simple image-similarity heuristic. In a 2023 comparison on simple ARC-style tasks, GPT-4 scored 69% when the grids were given as text and GPT-4V 23% to 25% when they were given as images.

### What is the difference between visual reasoning and image recognition?

Image recognition names what is in a picture, using knowledge learned in training. Visual abstract reasoning asks for a new rule inferred from a few examples, so stored knowledge helps little, and a single misread detail, such as a rotation or a count, makes the answer wrong.

### What are Bongard problems?

Visual puzzles that the Russian computer scientist Mikhail Bongard published more than fifty years ago: 100 problems, each with six drawings that share a concept and six that do not. The solver has to name the concept, such as convex against concave. Unlike Raven's matrices, there are no options to pick from.

### Why do multimodal models fail at simple visual puzzles?

Studies point to two separate failures. The model may misread the picture: four vision-language models averaged 58.07% on tasks such as counting line intersections, although probes showed their vision encoders held the needed information. And it may reason badly even from a correct description, as the USC study found when it replaced the images with expert-written text.

### Is ARC-AGI a visual reasoning test?

To people, yes: its tasks are coloured grids. Most AI systems receive them as arrays of numbers, and ARC Prize's own verification harness does the same. A 2025 MIT paper treated ARC as an image problem instead and trained a Vision Transformer from scratch that scored 60.4% on ARC-AGI-1 as an ensemble.

## Sources
1. [Does ChatGPT Belong in Mensa?](https://www.isi.edu/news/72195/does-chatgpt-belong-in-mensa/), USC Information Sciences Institute
2. [The Curious Case of Nonverbal Abstract Reasoning with Multi-Modal Large Language Models](https://arxiv.org/abs/2401.12117), COLM 2024 (arXiv, Ahrabian et al.)
3. [On the Measure of Intelligence](https://arxiv.org/abs/1911.01547), arXiv (François Chollet)
4. [Measuring abstract reasoning in neural networks](https://arxiv.org/abs/1807.04225), ICML 2018 (arXiv, Barrett et al., DeepMind)
5. [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://arxiv.org/abs/1903.02741), CVPR 2019 (arXiv, Zhang et al.)
6. [Stratified Rule-Aware Network for Abstract Visual Reasoning](https://arxiv.org/abs/2002.06838), AAAI 2021 (arXiv, Hu et al.)
7. [Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning](https://arxiv.org/abs/2010.00763), NeurIPS 2020 (arXiv, Nie et al.)
8. [ARC-AGI-2](https://arcprize.org/arc-agi/2), ARC Prize Foundation
9. [ARC-AGI-3 game schema](https://docs.arcprize.org/game-schema), ARC Prize Foundation
10. [ARC Prize 2024: Technical Report](https://arxiv.org/abs/2412.04604), arXiv (Chollet, Knoop, Kamradt, Landers)
11. [ARC Prize 2025 Results and Analysis](https://arcprize.org/blog/arc-prize-2025-results-analysis), ARC Prize Foundation
12. [ARC Is a Vision Problem!](https://arxiv.org/abs/2511.14761), arXiv (Hu et al., MIT)
13. [Emergent analogical reasoning in large language models](https://www.nature.com/articles/s41562-023-01659-w), Nature Human Behaviour (Webb, Holyoak and Lu)
14. [Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks](https://arxiv.org/abs/2311.09247), AAAI 2024 LLM-CP Workshop (arXiv, Mitchell et al.)
15. [Vision language models are blind: Failing to translate detailed visual features into words](https://arxiv.org/abs/2407.06581), arXiv (Rahmanzadehgervi et al.)
16. [Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad?](https://arxiv.org/abs/2410.19546), ICML 2025 (arXiv, Wüst et al.)
17. [Announcing ARC-AGI-3](https://arcprize.org/blog/arc-agi-3-launch), ARC Prize Foundation
18. [Analyzing GPT-5.5 & Opus 4.7 with ARC-AGI-3](https://arcprize.org/blog/arc-agi-3-gpt-5-5-opus-4-7-analysis), ARC Prize Foundation
19. [ARC Prize 2026: ARC-AGI-3 Milestone Prize #1](https://arcprize.org/blog/arc-prize-2026-milestone-1), ARC Prize Foundation
20. [OpenAI's GPT-6 Astra on ARC-AGI-3](https://arcprize.org/blog/astra), ARC Prize Foundation
21. [GPT-6 Astra: verified scores](https://arcprize.org/results/openai-gpt-6-astra), ARC Prize Foundation
22. [ARC Prize 2026](https://arcprize.org/competitions/2026), ARC Prize Foundation