Can we train AI to be creative? One lab is testing ideas

Human know-how derives successful portion from our chemoreceptor for novelty — we’re funny creatures, whether looking astir corners aliases testing technological hypotheses. For artificial intelligence to person a wide and nuanced knowing of nan world — truthful it tin navigate mundane obstacles, interact pinch strangers aliases invent caller medicines — it besides needs to research caller ideas and experiences connected its own. But pinch infinite possibilities for what to do next, really tin AI determine which directions are nan astir caller and useful?

One thought is to automatically leverage quality intuition to determine what’s absorbing done large connection models trained connected wide quantities of quality matter — nan benignant of package powering chatbots. Two caller papers return this approach, suggesting a way toward smarter self-driving cars, for example, aliases automated technological discovery.

“Both useful are important advancements towards creating open-ended learning systems,” says Tim Rocktäschel, a machine intelligence astatine Google DeepMind and University College London who was not progressive successful nan work. The LLMs connection a measurement to prioritize which possibilities to pursue. “What utilized to beryllium a prohibitively ample hunt abstraction abruptly becomes manageable,” Rocktäschel says. Though immoderate experts interest open-ended AI — AI pinch comparatively unconstrained exploratory powers — could spell disconnected nan rails.

How LLMs tin guideline AI agents

Both caller papers, posted online successful May astatine arXiv.org and not yet peer-reviewed, travel from nan laboratory of machine intelligence Jeff Clune astatine nan University of British Columbia successful Vancouver and build straight connected previous projects of his. In 2018, he and collaborators created a strategy called Go-Explore (reported successful Nature successful 2021) that learns to, say, play video games requiring exploration. Go-Explore incorporates a game-playing supplier that improves done a trial-and-error process called reinforcement learning (SN: 3/25/24). The strategy periodically saves nan agent’s advancement successful an archive, past later picks interesting, saved states and progresses from there. But selecting absorbing states relies connected hand-coded rules, specified arsenic choosing locations that haven’t been visited much. It’s an betterment complete random action but is besides rigid.

Clune’s laboratory has now created Intelligent Go-Explore, which uses a ample connection model, successful this lawsuit GPT-4, alternatively of nan hand-coded rules to prime “promising” states from nan archive. The connection exemplary besides picks actions from those states that will thief nan strategy research “intelligently,” and decides if resulting states are “interestingly new” capable to beryllium archived.

LLMs tin enactment arsenic a benignant of “intelligence glue” that tin play various roles successful an AI strategy because of their wide capabilities, says Julian Togelius, a machine intelligence astatine New York University who was not progressive successful nan work. “You tin conscionable move it into nan spread of, like, you request a novelty detector, and it works. It’s benignant of crazy.”

The researchers tested Intelligent Go-Explore, aliases IGE, connected 3 types of tasks that require multistep solutions and impact processing and outputting text. In one, nan strategy must put numbers and arithmetic operations to nutrient nan number 24. In another, it completes tasks successful a 2-D grid world, specified arsenic moving objects, based connected matter descriptions and instructions. In a third, it plays solo games that impact cooking, wealth hunting aliases collecting coins successful a maze, besides based connected text. After each action, nan strategy receives a caller study — “You get successful a pantry…. You spot a shelf. The support is wooden. On nan support you tin spot flour…” is an illustration from nan cooking crippled — and picks a caller action.

The researchers compared IGE against 4 different methods. One method sampled actions randomly, and nan others fed nan existent crippled authorities and history into an LLM and asked for an action. They did not usage an archive of absorbing crippled states. IGE outperformed each comparison methods; erstwhile collecting coins, it won 22 retired of 25 games, while nary of nan others won any. Presumably nan strategy did truthful good by iteratively and selectively building connected absorbing states and actions, frankincense echoing nan process of productivity successful humans.

IGE could thief observe caller narcotics aliases materials, nan researchers say, particularly if it incorporated images aliases different data. Study coauthor Cong Lu of nan University of British Columbia says that uncovering absorbing directions for exploration is successful galore ways “the cardinal problem” of reinforcement learning. Clune says these systems “let AI spot further by opinionated connected nan shoulders of elephantine quality datasets.”

AI invents caller tasks

The 2nd caller strategy doesn’t conscionable research ways to lick assigned tasks. Like children inventing a game, it generates caller tasks to summation AI agents’ abilities. This strategy builds connected different created by Clune’s laboratory past twelvemonth called OMNI (for Open-endedness via Models of quality Notions of Interestingness). Within a fixed virtual environment, specified arsenic a 2-D type of Minecraft, an LLM suggested caller tasks for an AI supplier to effort based connected erstwhile tasks it had aced aliases flubbed, frankincense building a program automatically. But OMNI was confined to manually created virtual environments.

So nan researchers created OMNI-EPIC (OMNI pinch Environments Programmed In Code). For their experiments, they utilized a physics simulator — a comparatively blank-slate virtual situation — and seeded nan archive pinch a fewer illustration tasks like kicking a shot done posts, crossing a span and climbing a formation of stairs. Each task is represented by a natural-language explanation on pinch machine codification for nan task.

OMNI-EPIC picks 1 task and uses LLMs to create a explanation and codification for a caller variation, past different LLM to determine if nan caller task is “interesting” (novel, creative, fun, useful and not excessively easy aliases excessively hard). If it’s interesting, nan AI supplier trains connected nan task done reinforcement learning, and nan task is saved into nan archive, on pinch nan recently trained supplier and whether it was successful. The process repeats, creating a branching character of caller and much analyzable tasks on pinch AI agents that tin complete them. Rocktäschel says that OMNI-EPIC “addresses an Achilles’ bottommost of open-endedness research, that is, really to automatically find tasks that are some learnable and novel.”

An array of learning challenges generated by OMNI-EPIC are shown here. The challenges are some caller and appropriately difficult for these systems.M. FALDOR ET AL./ARXIV.ORG 2024

OMNI-EPIC needn’t beryllium confined to beingness tasks, nan researchers constituent out. Theoretically, it could delegate itself tasks successful mathematics aliases literature. (Zhang precocious created a tutoring strategy called CodeButter that, she says, “employs OMNI-EPIC to present endless, adaptive coding challenges, guiding users done their learning travel pinch AI.”)  The strategy could besides constitute codification for simulators that create caller kinds of worlds, starring to AI agents pinch each kinds of capabilities that mightiness transportation to nan existent world. 

Should we moreover build open-ended AI?

“Thinking astir nan intersection betwixt LLMs and RL is very exciting,” says Jakob Foerster, a machine intelligence astatine nan University of Oxford. He likes nan papers but notes that nan systems are not genuinely open-ended, because they usage LLMs that person been trained connected quality information and are now static, some of which limit their inventiveness. Togelius says LLMs, which benignant of mean everything connected nan internet, are “super normie,” but adds, “it whitethorn beryllium that nan inclination of connection models towards mediocrity is really an plus successful immoderate of these cases,” producing thing “novel but not excessively novel.”

Some researchers, including Clune and Rocktäschel, spot open-endedness arsenic basal for AI that broadly matches aliases surpasses quality intelligence. “Perhaps a really bully open-ended algorithm — possibly moreover OMNI-EPIC — pinch a increasing room of stepping stones that keeps innovating and doing caller things everlastingly will depart from its quality origins,” Clune says, “and sail into uncharted waters and extremity up producing wildly absorbing and divers ideas that are not rooted successful quality ways of thinking.”

Many experts, though, interest astir what could spell incorrect pinch specified superintelligent AI, particularly if it’s not aligned pinch quality values. For that reason, “open-endedness is 1 of nan astir vulnerable areas of instrumentality learning,” Lu says. “It’s for illustration a ace squad of instrumentality learning scientists trying to lick a problem, and it isn’t guaranteed to attraction connected only nan safe ideas.”

But Foerster thinks that open-ended learning could really summation safety, creating “actors of different interests, maintaining a equilibrium of power.” In immoderate case, we’re not astatine superintelligence yet. We’re still mostly astatine nan level of inventing caller video games.