The path to AGI: AI supercomputers, scaling laws and the open questions

What AGI means, and why definitions differ

Narrow AI does the jobs it was built or trained for, and even a large language model that writes code can fail at tasks a person finds easy. AGI names the other end: a system general enough to do most of what a skilled person can do with a computer, including learning skills it was never trained for. Beyond that, definitions part ways, and the choice decides whether anyone can say AGI has arrived.

OpenAI's charter defines AGI as "highly autonomous systems that outperform humans at most economically valuable work". In February 2025, OpenAI's chief executive Sam Altman called AGI "a weakly defined term" and described it loosely as a system that can tackle increasingly complex problems, at human level, in many fields.

The most careful attempt to pin it down is Google DeepMind's Levels of AGI paper, published in November 2023. Its authors reviewed nine earlier definitions and proposed six principles, among them: measure what a system can do, not how it works or whether it is conscious; count both how well it performs and how general it is; cover cognitive and metacognitive tasks, not physical ones; and treat AGI as a path with stages, not a single finish line. The result is a grid:

The same paper rates autonomy separately, from AI as a tool through consultant, collaborator and expert to AI as an agent, because how much control people hand over changes the risks as much as raw capability does. It also notes that placing a system on the grid needs a standard benchmark of tasks, which does not exist yet.

Two other definitions come up later. François Chollet's On the Measure of Intelligence (2019) argues that skill at any given task is the wrong yardstick, because unlimited training data can buy skill and hide how little a system generalizes. He defines intelligence as skill-acquisition efficiency: how quickly a system learns new tasks from little experience. And the largest survey of AI researchers, run in October 2023, used "high-level machine intelligence": unaided machines that can accomplish every task better and more cheaply than human workers.

Keep the choice of definition in mind whenever someone announces a date. A system can reach Competent on a benchmark grid long before it outperforms people at most paid work, and neither says much about Chollet's efficiency test.

The 2024 AI supercomputer network behind the headlines

The news this page first reported came from SingularityNET, the company led by Ben Goertzel that develops the OpenCog Hyperon framework for AGI. According to its own July 2024 update, on July 23, 2024 it announced an initial $53 million investment to build what it called the world's first modular supercomputer dedicated to decentralized AGI research, along with high-performance computing and AI data centers.

The plan, as SingularityNET described it:

• Hardware: a modular data center with Nvidia L40S GPUs, AMD Instinct and Genoa processors, Tenstorrent Wormhole server racks, servers with Nvidia H200 GPUs, and Nvidia GB200 (Blackwell) systems, housed in Ecoblox ExaContainer modular data centers with ASUS and GIGABYTE servers.
• Workloads: training deep neural networks and large language models, and running hybrid neural-symbolic architectures such as OpenCog Hyperon.
• Network: the first in a series of compute containers that could be placed around the world and moved as needed, acting as hubs for a wider network of mixed devices with partners in the Artificial Superintelligence (ASI) Alliance.
• Timeline: in its August 2024 update, the company said the first phase would use half of the $53 million, that the supercomputer would start to come online in September 2024, and that the first phase would be complete by the end of 2024 or early 2025, depending on supplier deliveries.

That was a plan with a budget and a hardware list, announced by the company that would build it. It did not claim AGI, and a computer cannot show that an approach to AGI works; only the systems trained on it can. It is also modest next to the clusters that train frontier models: Meta's Llama 3.1 405B, released the same month, was trained on up to 16K Nvidia H100 GPUs in one cluster. The reason AGI projects start by buying compute at all is the subject of the next section.

How compute scale relates to AI capability

Training compute is counted in floating-point operations (FLOPs): every multiplication and addition a model needs while it learns. Two papers turned "bigger works better" from folklore into something engineers can plan with.

• Scaling laws (January 2020). Kaplan and colleagues at OpenAI found that a language model's loss, its error at predicting the next token, falls as a power law with model size, dataset size and training compute, with some trends holding over more than seven orders of magnitude. Architecture details such as width or depth mattered little within a wide range.
• Compute-optimal training (March 2022). Hoffmann and colleagues at DeepMind showed that most large models were undertrained: for a fixed compute budget, parameters and training tokens should grow together. Their 70-billion-parameter Chinchilla, trained on 1.4 trillion tokens with the same budget as the 280-billion-parameter Gopher (5.76 × 10²³ FLOPs), outperformed Gopher and GPT-3.

The budgets have grown fast since. GPT-3 (2020) used about 3.14 × 10²³ FLOPs. Llama 3.1 405B (July 2024) used 3.8 × 10²⁵ FLOPs on 15.6 trillion tokens, roughly 120 times GPT-3, and Meta used its own scaling laws to choose the model's size for that budget and to predict its score on a reasoning benchmark before training started. Across the field, Epoch AI found that the compute used to train the models in its database grew 4-5x a year from 2010 to May 2024, with similar growth for frontier models.

A second lever has grown in importance since. The 2026 International AI Safety Report describes "inference-time scaling": letting a model use more computing power to produce intermediate steps before its final answer, which brought especially large gains in mathematics, software engineering and science. Compute now buys capability twice, once in training and again every time the model answers.

What an AI supercomputer is

An AI supercomputer is not one giant chip. It is thousands of accelerators (Nvidia or AMD GPUs, Google TPUs, or chips from companies such as Tenstorrent) in one data center, wired together by an interconnect fast enough that they behave like one machine for a single training job. The Llama 3 paper describes one such system in unusual detail:

No single GPU can hold a model of this size and its training state, so the work is split several ways at once. Meta calls its combination 4D parallelism. Tensor parallelism splits each layer's matrices across the GPUs inside a server, where NVLink is fastest. Pipeline parallelism gives different layers to different servers, because links between servers are slower. Context parallelism splits very long input sequences. Data parallelism runs copies of the whole arrangement on different batches of data, and after every step the copies average their gradients (a collective operation called all-reduce) so that all of them apply the same update.

That lockstep is what makes a supercomputer different from a pile of servers, and also what makes it fragile. Meta reports that a single GPU failure may force the whole job to restart. During one 54-day stretch of Llama 3 pre-training there were 466 job interruptions: 47 planned for maintenance and 419 unexpected, about 78% of them confirmed or suspected hardware faults. The team still kept effective training time above 90% by saving checkpoints (1 MB to 4 GB of state per GPU) often and restarting from the latest one.

The physical limits show up too. Meta measured a 1-2% daily swing in throughput from mid-day heat, and when tens of thousands of GPUs pause or start together, the data center's power draw can jump by tens of megawatts, which the paper says stretches the limits of the power grid. Building an AI supercomputer is as much a power, cooling and networking problem as a chip problem.

Can a network of smaller computers train frontier AI?

SingularityNET's plan was a network of relocatable compute nodes, not one cluster. The idea has a real research basis. Standard training needs many tightly connected accelerators exchanging gradients at every step, and as Google DeepMind's DiLoCo paper (November 2023) puts the problem, a single large cluster is hard to build, while several smaller ones are easier to find. DiLoCo lets each "island" of devices train on its own for many steps and share a combined update only occasionally. On the C4 dataset, eight workers matched fully synchronous training while communicating 500 times less.

A larger test came a year later. Prime Intellect's INTELLECT-1 (December 2024) trained a 10-billion-parameter model on 1 trillion tokens using up to 14 nodes on three continents, with 30 independent compute providers joining and leaving during the run. It kept compute utilization between 83% and 96%. Each node ran 100 local steps, about 38 minutes on a node of eight H100s, before a synchronization that took about 2 minutes across the United States and 10 minutes worldwide. With DiLoCo and updates compressed to 8-bit integers, it needed about 400 times less communication bandwidth than ordinary data-parallel training.

The limits are as instructive as the results:

• Scale. INTELLECT-1 had 10 billion parameters and 1 trillion tokens. Llama 3.1 405B, trained the same year in one cluster, had 405 billion and 15.6 trillion. The authors present their result as evidence that decentralized frontier training is feasible and promising, not as frontier training.
• Each node is still a small supercomputer. Decentralization happened between nodes of eight H100 data-center GPUs each, not between office PCs.
• Unreliable links. The report notes that bandwidth between nodes could drop suddenly and for no clear reason.
• Trust. A network open to outside providers also has to confirm that each contributor ran the training it claims to have run, a problem a single operator never faces.

Decentralized compute can widen who trains mid-sized models and use hardware that would otherwise sit idle. It does not remove the need for large amounts of compute, and the published results described here are well below the scale of a frontier cluster.

Reasoning: where more compute is not the whole answer

Logical reasoning is where the two main traditions in AI meet. Neural networks learn statistical patterns from data. Symbolic AI, the older tradition, represents knowledge explicitly and applies logical rules, which makes its reasoning checkable but brittle when the world does not fit the rules. Neural-symbolic computing tries to combine robust learning in neural networks with reasoning and explanation through symbolic representations, and its proponents link it to trust, safety and accountability.

SingularityNET's Hyperon is a hybrid of this kind. Its August 2024 update argued that such approaches need somewhat less data, processing and energy than standard deep neural networks, and in the same statement that the company still needed significant supercomputing facilities. Even the alternative to pure scaling was buying compute.

Large language models have moved from the other direction. Reasoning models spend inference-time compute on intermediate steps, with the largest gains on complex reasoning tasks in mathematics, software engineering and science. The 2026 International AI Safety Report still describes capabilities as "jagged": leading systems excel at some difficult tasks and struggle with simpler ones, such as counting objects in an image, reasoning about physical space and recovering from basic errors in longer workflows.

For anyone building with AI today, the practical rule follows from that jaggedness: when a result must be correct every time, such as a tax calculation or an eligibility rule, put the logic in ordinary code or a rules engine, and use a model for the steps that genuinely need judgment.

The case for near-term AGI

The optimistic case rests on trends that have held so far:

• Scaling keeps paying. The power laws measured in 2020 held across many orders of magnitude, compute grew 4-5x a year, and inference-time scaling added a second lever.
• Resources convert to capability predictably. In Three Observations (February 2025), Altman wrote that a model's intelligence roughly equals the log of the resources used to train and run it (chiefly training compute, data and inference compute), and that the cost of a given level of AI falls about 10x every 12 months.
• Some lab leaders expect very capable systems soon. Anthropic's chief executive Dario Amodei, in Machines of Loving Grace (October 2024), described "powerful AI" as a model smarter than a Nobel Prize winner across most relevant fields, able to use every interface a remote worker has, run as millions of copies: a "country of geniuses in a datacenter". He wrote that it could come as early as 2026, while noting ways it could take much longer.
• Money follows the bet. The International AI Safety Report notes that developers, betting that computing power will remain important, have announced hundreds of billions of dollars in data center investments.

Both Altman and Amodei run companies that train frontier models. That does not make them wrong, but their forecasts are the views of participants, and they are not neutral estimates.

The case against, or at least for patience

The skeptical case does not deny progress. It questions whether the current recipe leads all the way:

• Skill is not intelligence. Chollet's argument is that benchmark skill bought with vast training data says little about the ability to learn something new. The ARC Prize Foundation reports that ARC-AGI-1, created in 2019, saw little progress through a 50,000x scale-up of base language models until late 2024, when test-time adaptation methods appeared. Its successor, ARC-AGI-2 (2025), was designed so that log-linear scaling is insufficient; every task in its calibrated evaluation sets was solved by at least two people in two attempts or fewer.
• Data may run short. Villalobos and colleagues estimated (revised June 2024) that, if trends continue, models will be trained on datasets about as large as the entire stock of public human-written text sometime between 2026 and 2032. They list synthetic data, learning from data-rich domains and better data efficiency as possible ways around it.
• Jagged capabilities. Failures on simple tasks suggest the systems are not general in the way people are, whatever their scores on hard ones.
• Physical limits. Power swings of tens of megawatts in a single training cluster, and the report's own list of data and energy bottlenecks, are reminders that exponential compute growth runs on real grids and supply chains.

Researchers themselves disagree widely. In the October 2023 survey of 2,778 researchers who had published at top AI venues, the aggregate forecast gave a 10% chance that machines would outperform humans at every possible task by 2027 and 50% by 2047, 13 years earlier than a similar survey found a year before. Asked instead when all human occupations would be fully automatable, they put the 50% mark at 2116. The 2026 International AI Safety Report sums up the state of evidence: between now and 2030, progress could plausibly slow or plateau, continue at current rates, or accelerate dramatically if AI systems begin to speed up AI research itself.

Safety and governance: who oversees frontier AI

At the UK's AI Safety Summit in November 2023, attending countries agreed the Bletchley Declaration, which recognized the potential for serious, even catastrophic, harm from frontier AI, including unintended issues of control relating to alignment with human intent.

Several governments set up institutes to test those models:

• United Kingdom. The AI Safety Institute was renamed the AI Security Institute on February 14, 2025, with a sharper focus on national security and crime.
• United States. The counterpart is NIST's Center for AI Standards and Innovation (CAISI), which makes voluntary agreements with AI developers and leads unclassified evaluations of capabilities that may pose national-security risks, such as cybersecurity, biosecurity and chemical weapons.
• International. The International AI Safety Report, initiated by the governments at the AI Safety Summit, is written by more than 100 experts led by Turing Award winner Yoshua Bengio and backed by more than 30 countries and international organizations. Its second edition was published on February 3, 2026.

The European Union went further and wrote rules into law. The AI Act, Regulation (EU) 2024/1689, defines a general-purpose AI model as one that shows significant generality, can competently perform a wide range of distinct tasks and can be integrated into many downstream systems. Every provider of such a model must keep technical documentation, give downstream providers the information they need, have a copyright policy and publish a summary of the content used for training. Providers of models with systemic risk must also evaluate them, including adversarial testing, assess and mitigate systemic risks, report serious incidents and ensure adequate cybersecurity.

The link to compute is explicit. Under Article 51(2), a model is presumed to have high-impact capabilities, and therefore systemic risk, when the cumulative compute used to train it exceeds 10²⁵ FLOPs. A model trained with Llama 3.1 405B's budget would be above that line; one trained with GPT-3's would be far below it.

What this means for your organization today

Whatever you believe about AGI timelines, the systems available today are powerful, uneven and priced by the token. That points to a few steady practices:

1. Judge models on your own work. Capabilities are jagged, so a benchmark score or a lab's forecast tells you little about your documents, customers and edge cases. Our explainer on Gemini's MMMU score shows how much one headline number can hide.
2. Keep hard rules in code. Use a model where a step needs judgment, such as reading a document or drafting a reply, and ordinary software everywhere else.
3. Keep a person on consequential actions. Anything that changes records, moves money or affects people should wait for a rule or a named person to approve it.
4. Know which rules touch you. The duties above fall on providers of general-purpose AI models. If you build AI into a product or service used in the EU, the Act's transparency and high-risk rules for AI systems may apply instead. Take legal advice for your case.

If you want to put today's models to work on a real process, our AI automation service starts with one real workflow, reviewed with the people who run it, and uses a model only where a step needs judgment, with nothing changing in your systems until a rule or a person approves it. More posts on the topic are under AI.