Microsoft's Nadella Hits Out at AI Rivals Over Distillation

Satya Nadella, Microsoft CEO, has publicly criticised, though not named, AI labs throughout the tech industry who use AI distillation to train their AI models.

The AI distillation process involves training a smaller AI model with the outputs of a larger, more powerful one.

In a post on X, Satya says that model makers complaining about the distillation process are being hypocritical over the AI training process.

He says that while "great innovation... is needed" from model providers having fair use rights to train on public data, he finds it "ironic" that the consensus among AI model makers is to impose "restrictive terms" on distillation, while still reserving the right to learn from customer usage and interaction data themselves.

“If learning flows in only one direction, economic value converges toward the owners of the learning infrastructure rather than the creators of the knowledge itself,” he adds.

Industry criticisms of AI distillation

Anthropic, OpenAI and Google DeepMind rely on work created by others to train their own AI models. ChatGPT, Claude and Gemini source their information from publicly available writing, images and forms of data. 

Many have voiced ethical and copyright-related concerns regarding this training system, with companies like the BBC suing model makers like Anthropic for non-consensual information sourcing.

While Satya did not name Anthropic outright in the post, there is evidence to suggest it was strongly directed at the AI firm. Satya’s criticism of AI model makers complaining about distillation follows Anthropic’s post earlier this year on “detecting and preventing distillation attacks.”

In June, Anthropic wrote a letter to South Carolina Senator Tim Scott and Massachusetts Senator Elizabeth Warren, saying that Alibaba had recently carried out the “largest known distillation attack” against the AI firm's data.

The statement said that competitors can use distillation to acquire “powerful abilities from other labs in a fraction of the time and at a fraction of the cost, that it would take to develop them independently”.

Alibaba is yet to respond to Anthropic’s accusations.

Business risks with outsourcing AI models

In the same post, he also warns that companies relying on major AI models are giving AI firms access to their own proprietary data.

He adds that companies need to own their AI infrastructure and industry knowledge instead of relying on a single model vendor.

Satya says they should also conduct their own evaluations and their own “learning loop” to allow their AI technology to develop and improve over time.

“That is why enterprises need a real trust boundary for their human capital and token capital to compound," he says, adding that the boundary must not be crossed by anyone or anything, “not even the intelligence exhaust, without consent”.

Similarly, Elon Musk has also publicly criticised Anthropic’s use of data sourcing to train its own AI models.

In a February post on X, Elon accused Anthropic of “stealing training data at a massive scale”, saying it will have to pay “multi-billion dollar settlements” for theft.

“This is just a fact,” he adds.