Right, but also wrong. The real reason why LLMs will NEVER be AGI is because the weights are fixed, once the model is trained the model cannot change on its own neural nets. A human brain, like the one of a toddler has way less knowledge than any LLM but is flexible, malleable and is making new connections as the kid learn more, LLMs can´t do that, they have to rely on a fixed neural net to solve a problem. They know python because they saw a million python examples but as soon they face a new python library that was not in their training dataset, is over. an AGI could be able to take a few examples of the new python library and "learn" by adjusting the configuration of its own weights like a normal brain would do. You don´t need a giant context window, because no human that is not in the spectrum can memorize so much. Most human memorize snippets or resume of the things they learn.
