Generative artificial intelligences like ChatGPT or Le_Chat, which are now found almost everywhere, are based on models that analyze and exploit mountains of existing data. This strategy makes it possible to reproduce—or rather, to imitate as realistically as possible—a wide variety of human abilities, ranging from solving scientific problems to writing poetry, creating images and photos, or synthesizing documents. Some AIs have even successfully passed benchmark tests like the Turing test, although some controversy remains regarding the real ability of these models, currently incapable of planning or reasoning, to succeed in this type of exercise.
This reliance on human data, however, shows its limits: while progress has been very significant so far, high-quality human sources that can satisfactorily train AI models in scientific fields have already been used up or are about to be. Progress fueled exclusively by supervised learning from these types of sources and information therefore tends to slow down. For generative artificial intelligence to continue to progress, a new source of data is essential, especially if we expect a degree of originality and innovation, in other words, true “intelligence,” from these AIs.
From the era of human data to the era of experience
Researchers David Silver and Richard Sutton, two legendary figures in the field of artificial intelligence (David Silver led the research behind AlphaGo and AlphaZero, the DeepMind AI that beat humans at chess, while Richard Sutton is considered one of the founders of reinforcement learning), posit that it must now be allowed to interact with the real world. This is the essence of the new stage they propose: the era of experience. According to them, new capabilities should emerge once the full potential of experiential learning is exploited.
The beginnings of this transition from the “age of human data” to the “age of experience” are already visible, even with current models. Last year, for example, AlphaProof and AlphaGeometry 2 – two AIs developed once again by DeepMind – tackled problems posed during the International Mathematical Olympiad, a global mathematics competition for high school and middle school students. With four out of six problems solved using their reinforcement learning algorithm, these two artificial intelligences would have won a silver medal if they had actually participated in this event.
The approach that the researchers advocate, which they call “experiment flow”, is precisely based on reinforcement learning; it aims to address the shortcomings of current large language models (or LLMs), which are primarily designed to answer individual human questions on an ad hoc basis, by giving AIs the ability to self-discover their own knowledge. In other words, generative artificial intelligences must no longer be content with short interactions via a simple exchange of questions and answers, but rather have their own stream of experience that progresses over a long time scale... like humans. Here it is, the essential new source of data.
More autonomous AIs: benefits or risks for humanity?
In this new era, agents will have to interact with the real world in a much more autonomous manner. Rather than focusing on the necessarily restrictive actions and observations favored by humans, AI will have to interact with its environment on its own to achieve future goals and continually adapt to new behavior patterns. Ultimately, these new artificial intelligences will even have to be able to actively explore the world to discover new strategies that would never have occurred to us.
By continuously learning from the results of their own experiments, these artificial intelligences could quickly acquire new knowledge. Autonomously conducting their own tests and measurements would lead to the development of new materials, drugs, and technologies at an unprecedented pace. They may even prove capable of displaying capabilities previously considered the exclusive domain of humanity: long-term problem-solving, innovation, or a deep understanding of real-world consequences.
But interacting autonomously with the world over extended periods to achieve long-term goals will, on the other hand, offer fewer opportunities for humans to intervene, and therefore require a high level of trust. Moving away from purely human data and ways of thinking may also make future artificial intelligences more difficult to interpret and understand. Further research is therefore needed to ensure a safe transition to the age of experience; in the meantime, keep saying thanks to AI, just in case.
Source: Welcome to the Era of Experience (David Silver, Richard S. Sutton)
0 Comments