Alex Chaloner

The Future of AI: Scaling Laws Cannot Beat The Fractal Problem

(I am writing this, as with everything, in a personal capacity).

We are here in September 2026, with the new release of GPT-6 Astra, and I feel that AI progress has hit a wall.

Let me hedge by saying: AI is making some amazing progress. We’ve seen multiple difficult Maths Problems being solved. GPT-6 Astra can now animate a Pelican riding a Bicycle exceedingly well. I do not for a moment think these are trivial things. But I feel AI is excelling in breadth, not depth. I have seen very little progress in depth.

Back in 1950, Alan Turing noted that Machine Intelligence would be well underway by the time it could play Chess or learn English. I feel we are here again.

Back in ‘25, we were excited at these new AIs which we thought were like Interns! Yes, smart interns! The inflexion point was Q4’25 with the release of Opus 4.5. Back in the beginning of ‘26, I recall everybody being very excited about OpenClaw. Agents! Personal Assistants! We made the Mac Minis sell out. That excitement has petered out. What happened? Well, we all used it, and found that it wasn’t good enough to be exciting. It made the same old stupid mistakes again and again. It was not the ‘second brain’ we so dearly crave - it filled itself up with sloppy markdown files and forgot about them.

We have 1-day interns, again and again and again.

Any student who thought they were very smart and very prepared by University runs out into the world, straight into a job, where they suddenly find they learned nothing particularly useful. The most important thing at a job is Domain Knowledge. Domain Knowledge turns out to be like a Fractal. The harder you look, the more there is to see. People are like this too. Everything in the world is like this. Away from the nicely-confined box of the exam room, things get very complicated very quickly.

The way we pre-train LLMs mean that we put a lot of effort into making the LLM understand the Fractal.

The fractal problem: GPT-4, GPT-6, and reality shown as increasingly detailed Mandelbrot sets.

Scaling Laws mean we can always spend exponentially more to get to the next iteration of the Fractal. But data inefficiency, and lack of memory, mean the LLM only gets us a surface level, even as we scale by orders of magnitude.

Your Scaling Laws cannot beat The Fractal Problem.

GPT-6 Astra can do more than ever! It is exceedingly more advanced at spatial reasoning, it is delightfully good at animation. But coders are noticing it’s not the big leap in their domain.

The cure here is In-Context Learning, and Continual Learning, and research along the axis of depth. LLMs are already great at In-Context Learning - after all, Language Models are Few-Shot Learners. But these are far from solved. Context lengths are still extremely limited, often caused by that dastardly O(N^2) self-attention. And even with larger context lengths, not much comes out of it - recall OpenClaw’s messy Markdowns. We are getting some interesting research around utilising the latent space - but training to use these is difficult, and I will take a guess that it won’t be so easy as self-supervised pre-training. Reinforcement Learning methods are - fine. They are doing exceedingly well in verifiable spaces. But we are running out of verifiable spaces, and you cannot RL again every deep fractal problem.

Scaling Laws cannot beat the Fractal Problem.

Short-term solutions seem to be in Specialized models - but an individual can surely never have an LLM with customized weights just for them. The data learning is too inefficient for an individual.

So, in the short-term (say 1-2 years), I worry that AI has hit a wall. I don’t feel we are materially different to a year ago, despite reportedly Fable and GPT-6 Astra being in the 3-10T parameter size range. We need a big research breakthrough to continue the pace, and it’s difficult to bet on a breakthrough that hasn’t happened yet.

This worries me because I feel that their may have been overinvestment, and this will cause a lot of short-term pain. There are trillions of dollars being spent on data center buildout. It is not an original thought to remember the dot-com bubble - but it’s easy to see the resemblance. I don’t like the feeling, so I will sell the majority of my tech stocks. And above all of that, the question still becomes: can AI be profitable? It is certainly not useless and it is certainly generating a lot of revenue. But the funding of the next, bigger model is exponentially more expensive. It must be trained since the competition is tight.

Nonetheless I remain positive about the long-term future of AI. A rising tide lifts all boats, and these models seem to rise the tide considerably. I have, and will continue to, dedicate a significant portion of my time towards AI research. I believe all powerful tools can be used to do great and good things.

To beat the Fractal Problem, I would put bets on:

I would not put bets on:

Personally, I’ve always thought Reinforcement Learning is one of the few ways where we see real intelligence emerge - Move 37 of Alphago is a famous example - it’s been a challenge to transfer this to LLMs, but I’m sure we’ll get better. I also see a lot of interesting research in Latent Spaces, where they seem to be much easier to manipulate than previously assumed. And I have never been convinced that O(N^2) attention is optimal – I believe we will see something akin to O(NlogN), or more likely O(N f(N)) where f(N) is the real information content of the N-size context length, for which N is the upper bound.

Finally, this is why I believe you should continue to learn and be passionate about learning in-depth. The current state of things is that AI cannot outlearn you. You are simply too data-efficient. You will see parts of the fractal that GPT-8 will never be able to fill. Continue learning, continue working, you are still far better than AI.