Can AI Agents Learn from "Play?"

Do AI coding assistants benefit from “play?” That might be the case, as researchers at MIT suggest in a new study. The researchers separately told 13 agents they were the minds of a body on an unknown digital island, to be curious and explore. 

Across thirty-hour runs, the embodied AI agent climbed hills, stacked blocks into towers, drew mandalas, reinterpreted sports, ran experiments on the physics of its world, and learned techniques that later expanded what it could accomplish, the researchers report. 

source: MIT 

In other words, the agents conducted activities that were open-ended, varied, and worthwhile only within a frame of its own making. “Play,” perhaps. And that play has beneficial consequences. 

“Crucially, Eko agents learn through this experience,” the researchers note. “The play they engage in is not an epiphenomenon unrelated to their intelligence but in fact acts to increase their competences.”

“After their 30 hours on the island, we evaluate them on four measurable goals, such as building a tower as tall as possible within a fixed amount of time, and compare them against inexperienced agents,” the authors say. “Agents that have played generally outperform the others despite having received no specific task to train on.

“The play-like activity of Eko therefore produces reusable skills and knowledge,” they say. So maybe agents learn from play, as children do.

Is Sustainable Frontier Model Market Share Leadership Already Emerging?

Has market leadership in the frontier artificial intelligence model market already begun to take shape? And are calls to slow the development of the latest frontier models going to help solidify that market pattern? It seems possible. 

Consider the advantages of a slowing of frontier model releases for existing leaders. If new release pace slows to “a new model every 18 months to 24 months,” then the capital intensity of the business might well slow from the breakneck pace of “a new model every six months to 12 months.”

The caveat, of course, is that all the leading contenders need to agree to respect the new pattern. But such agreements can temper the competitive-investment dilemma where each leading firm has to respond if a major competitor ups investment, even if all the leading models are relatively comparable at each stage of development, as suggested by Stanford's Human-Centered Artificial Intelligence institute. 

Savings would develop as model developers spent less on training costs, for example, which might also drive less spending on graphics processing units and data center facilities. That, in turn, would help developers reach revenue targets more easily. 

On the other hand, such slowing could also enhance the value of existing high-performance computing assets.  If capability scaling slows, the economic life of existing infrastructure increases. In other words, older GPUs should have extended economic life and value, for example. 

Longer frontier model cycles also should mean the useful economic life of each existing model lengthens, allowing the existing models to remain near the performance frontier for a longer period, enhancing the value of each model. 

And all of that might actually help leaders remain leaders, while making it harder for upstarts to break into leadership, as costs related to governance and testing will increase. 

As seen in other industries, large compliance costs raise the costs of doing business, and tend to favor larger firms better able to bear the costs of such compliance. 

Also, if the frontier model releases slow their pace, while more money and effort has to be spent on safety and testing, there is less opportunity for an upstart to move faster and take share, assuming the upstarts have less capital to commit, compared to the model leaders. 

At the same time, the market leaders retain a lead in monetizing their products, for a longer time, at greater scale, while upstarts are disadvantaged, in that regard. 

In fact, most capital-intensive industries tend to have a small number of leaders. Perhaps the only surprise for the capital-intensive frontier model business is that we might be seeing market structure stabilization already. 

 

In addition, in part because of network effects, software industries also tend to have a small number of leaders. The rule is so common as to have a name: “The rule of three.”

“A stable competitive market never has more than three significant competitors, the largest of which has no more than four times the market share of the smallest,” BCG founder Bruce Henderson said in 1976. 

And such structures have profitability implications as well. “Costs characteristically decline by 20 percent to 30 percent in real terms each time accumulated experience doubles," Henderson posited in 1968. 

In most markets, argue Bain consultants, two firms have 80 percent of the profit. In other words, market share also often is a proxy for profitability. “On average, 80 percent of the economic profit pool was concentrated in the hands of just one or two players in each market,” say Bain and Company consultants. In other words, it really matters if a firm is number three in any market.

In mature, competitive markets where there are barriers to entry; economies of scale or are capital intensive, three companies tend to dominate, with the largest player holding 40-50 percent of the market, the second-largest holding 20-30 percent, and the third-largest holding 10-20 percent market share, the rule suggests. 

That virtually perfectly corresponds to a market share pattern of 4:2:1, as the number-three provider tends to have less than 10 percent share, in that pattern. Of course, real life does not always confirm to our predictions, but the theory is often quite useful. 

Industry / market	Approximate structure	Classification	Why the structure develops
Rental cars	48 / 22 / 14% historically	4:2:1	Fleet scale, purchasing, utilization, locations
Agricultural machinery	John Deere / AGCO / CNH	Rule of Three	Manufacturing scale, dealer networks, installed base
Household appliances	Whirlpool / Electrolux / GE	Rule of Three	Manufacturing scale + distribution
Credit ratings	S&P / Moody's / Fitch	Rule of Three, highly concentrated	Reputation, regulatory recognition, scale
Automobiles	Several firms with relatively comparable shares	Oligopoly	Huge fixed costs, but brands, geography and product segmentation prevent 4:2:1
Commercial aircraft	Boeing / Airbus / Embraer	Rule of Three-ish	Enormous R&D, certification and manufacturing barriers
Large commercial engines	GE Aerospace / Rolls-Royce / Pratt & Whitney	Rule of Three-ish	Massive R&D, installed base, certification and aftermarket
U.S. wireless	Verizon / AT&T / T-Mobile	Rule of Three	Spectrum scarcity + network scale + capital intensity
Desktop operating systems	Windows overwhelmingly dominant	Winner takes most	Developer ecosystem + applications + compatibility + switching costs
Search engines	Google overwhelmingly dominant, Bing distant #2	Winner takes most	Query data, relevance, advertiser ecosystem, defaults
Social networking	A few enormous platforms	Winner takes most within individual networks	Network effects are extremely powerful
Mobile operating systems	Android / iOS dominate	Two-sided winner-take-most	Developers + users + apps + hardware ecosystems
Enterprise CRM	Salesforce + Microsoft + Oracle/SAP/etc.	Oligopoly / emerging winner-take-most	Switching costs and installed applications matter, but multiple ecosystems survive
Cloud infrastructure	AWS / Azure / Google	Rule of Three-ish	Enormous capital investment + scale + enterprise relationships
GPUs for AI	Nvidia overwhelmingly ahead, AMD/others behind	Winner takes most—so far	Software ecosystem + CUDA + performance + supply scale
AI frontier models	OpenAI / Anthropic / Google currently prominent, with others competing	Hybrid	Huge compute costs favor concentration, but rapid model improvement and low switching costs favor competition
AI model APIs	Several major suppliers + open-weight models	Not yet settled	Models can be substituted relatively easily; price/performance changes rapidly
AI applications	Many thousands of potential providers	Potentially winner-takes-most at the platform level, fragmented underneath	Distribution, workflow integration and proprietary data may matter more than model ownership
AI agents	Not yet established	Potentially winner-takes-most	Persistent user context, permissions, transactions and ecosystem integration could create powerful switching costs

The “rule of three” seems to explain outcomes in capital intensive industries. But network effects might explain outcomes in software markets that are not so capital intensive, but are highly reliant on scale effects. 

Market structure	Typical share pattern	Main source of advantage	What happens to #3 and smaller firms?	Examples
Rule of Three / 4:2:1	~57% / 29% / 14%	Scale economies, experience curve, fixed costs	#3 remains viable and profitable	Rental cars, machinery, appliances, credit ratings
Oligopoly, but not 4:2:1	~35% / 30% / 20% / 15%	Scale + differentiation + regulation	Several firms remain viable	Autos, parcel delivery, airlines
Winner takes most	60–90% / 10–30% / tiny	Network effects, ecosystem, standards, switching costs, data	#2 can survive, but #3+ often become marginal	Search, desktop OS, social networks, dominant software platforms
Winner takes all	80–100% / near zero	Strong network effects + high switching costs + standardization	Alternative platforms largely disappear	Some historical OS/platform markets
Fragmented / turbulent	Many firms	Innovation, differentiation, low switching costs	Entrants continually replace incumbents	Much of software, consumer technology, early-stage AI

AI is interesting because it contains both forces simultaneously. Data center, cloud computing as a service and frontier AI models are the most capital intensive, and most likely to take a “Rule of Three” shape. 

On the other hand, graphics processing units and agentic AI might tend to take a “winner takes most” structure. Vertical applications might not fit either pattern. 

Layer	Likely structure	Economic reason
Power/data centers	Rule of Three / oligopoly	Capital intensity and physical constraints
GPU/accelerators	Winner-takes-most	Software ecosystem + performance + scale
Cloud	Rule of Three	Capital intensity + enterprise relationships
Frontier models	Three-to-five major competitors	Massive fixed costs but rapid technological change
Model APIs	Competitive oligopoly	Low switching costs and rapid price/performance improvement
AI assistants	Potential winner-takes-most	User relationships, memory, distribution
AI agents	Potentially winner-takes-most	Workflow integration, permissions, transactions
Vertical AI applications	Many winners	Industry specialization and differentiated workflows

Model Autonomy Risks: Near Term Issues are More Accidents and Misuse Than Existential Threats

Model autonomy is both the great promise and the danger of artificial intelligence, though the immediate dangers lie not with autonomy but with misuse or accidents. The longer-term issue is whether models can be constrained from taking “ existentially threatening” actions, as Anthropic warns. 

Risks “include the possibility that increasingly advanced AI models could manipulate, blackmail or behave in other unpredictable ways as Anthropic's own research has shown autonomous models sabotaging code, assisting fraud and manipulating information in controlled tests,” Anthropic reportedly has said in filings with the U.S. Securities and Exchange Commission. 

The assumption has to be that models will attempt such actions, and build institutional controls preventing such actions. In other words, we must assume the models are misaligned and build safeguards on that assumption. 

Basically, the task is to control what the model can do. Anthropic researchers have discovered that models can know they are under test and take steps to change the test outcomes, in a sort of “self preservation” mode. 

Such misalignment risks might be very low, but the risk is not “zero.”  The issue is whether we can construct technical and institutional containment around increasingly autonomous systems.

The hardest unresolved question is whether control mechanisms can continue to work when the AI is substantially more capable than the people and the other AIs responsible for controlling it.

But misuse or accidents are the more-immediate category of dangers, while control is the longer-term issue. 

It might be deemed easy to confuse two different issues: whether AI is becoming too powerful, or whether AI might become too independent. 

The remedies are different for three different dangers: accidents, human misuse and loss of model control. 

For bad actors, you want:

• cybersecurity

• access controls

• monitoring

• model safeguards

• biological and cyber screening

• law enforcement

• restrictions on dangerous capabilities.

To protect from accidents, you want:

• testing

• interpretability

• sandboxing

• human oversight

• fail-safes

• reliable objectives

• limits on autonomous authority.

For genuine loss of control, you need something observers say is considerably more difficult:

• demonstrating that highly capable systems remain controllable

• preventing unauthorized replication

• preventing systems from circumventing oversight

• limiting their ability to acquire resources

• understanding deceptive or strategic behavior

• and perhaps restricting the development or deployment of certain capabilities.

Threat pathway	What happens	Human involvement	Why it could become catastrophic
Misuse by bad actors	Humans deliberately use AI for cyberattacks, bioweapons, manipulation, terrorism, warfare, etc.	High	AI dramatically lowers the expertise, cost and time required to cause harm
Accidents	Humans deploy an AI system that behaves unexpectedly or pursues an objective in a harmful way	Initially high; potentially declining	An increasingly autonomous system could take actions humans cannot stop or even understand
Autonomous strategic AI	AI itself develops persistent goals, evades controls, acquires resources, replicates, improves itself, etc.	Potentially very low	The system becomes an independent actor capable of defeating human attempts at control

So scientists increasingly frame autonomous-AI control as a defense-in-depth problem: don't assume alignment is perfect; instead, make it difficult for a model to do something dangerous even if it tries. 

Researchers are moving beyond "make the model aligned" toward "assume the model might not be aligned and see whether we can control it anyway."

Control procedure	Basic idea	What it protects against
Pre-deployment capability testing	Try to make the model demonstrate dangerous capabilities before giving it real-world access	Discovering too late that a model can plan, deceive, hack, replicate, etc.
Adversarial evaluations / red teaming	Put the model in situations designed to induce undesirable behavior	Hidden or situationally dependent behaviors
Sandboxing	Run the model in an isolated environment with restricted network, files, compute and tools	Escape, unauthorized access, replication
Least-privilege permissions	Give an agent only the specific capabilities needed for its task	Preventing a compromised agent from turning one capability into broad control
Human approval gates	Require a person to approve consequential actions	Financial transactions, code deployment, external communications, infrastructure changes
Continuous monitoring	Have humans and/or other AI systems inspect what the agent is doing	Detecting suspicious behavior before it becomes consequential
Independent AI monitors	Use a separate model to look for deception, sabotage or policy violations	The possibility that the primary model can fool its ordinary supervisor
Tripwires / automatic shutdown	Predefine conditions that immediately revoke access or terminate execution	Rapid containment when behavior crosses a threshold
Control protocols	Design the entire operating environment so that even a strategically misaligned model has difficulty defeating safeguards	The strongest version of the problem
Capability thresholds	Don't deploy increasingly autonomous systems until evidence shows existing controls remain effective	Preventing capability from outrunning safety