This AI Model Is Behind the Agents Feature in Windows 11 Settings

Microsoft has launched Mu, a brand new synthetic intelligence (AI) mannequin that may run regionally on a tool. Final week, the Redmond-based tech large launched new Home windows 11 options in beta, amongst which was the brand new AI brokers characteristic in Settings. The characteristic permits customers to explain what they wish to do within the Settings menu, and makes use of AI brokers to both navigate to the choice or autonomously carry out the motion. The corporate has now confirmed that the characteristic is powered by the Mu small language mannequin (SLM).

Microsoft’s Mu AI Mannequin Powers Brokers in Home windows Settings

In a blog post, the tech large detailed its new AI mannequin. It’s at the moment deployed totally on-device in suitable Copilot+ PCs, and it runs on the system’s neural processing unit (NPU). Microsoft has labored on the optimisation and latency of the mannequin and claims that it responds at greater than 100 tokens per second to fulfill the “demanding UX necessities of the agent in Settings state of affairs.”

Mu is constructed on a transformer-based encoder-decoder structure that includes 330 million token parameters, making the SLM match for small-scale deployment. In such an structure, the encoder first converts the enter right into a legible fixed-length illustration, which is then analysed by the decoder, which additionally generates the output.

Microsoft stated this structure was most popular as a result of excessive effectivity and optimisation, which is important when functioning with restricted computational bandwidth. To maintain it aligned with the NPU’s restrictions, the corporate additionally opted for layer dimensions and optimised parameter distribution between the encoder and decoder.

Distilled from the corporate’s Phi fashions, Mu was skilled utilizing A100 GPUs on Azure Machine Studying. Usually, distilled fashions exhibit greater effectivity in comparison with the dad or mum mannequin. Microsoft additional improved its effectivity by pairing the mannequin with task-specific knowledge and fine-tuning by way of low-rank adaptation (LoRA) strategies. Curiously, the corporate claims that Mu performs at an analogous degree because the Phi-3.5-mini regardless of being one-tenth the scale.

Optimising Mu for Home windows Settings

The tech large additionally needed to remedy one other drawback earlier than the mannequin may energy AI brokers in Settings — it wanted to have the ability to deal with enter and output tokens to alter tons of of system settings. This required not solely an enormous data community but additionally low latency to finish duties nearly instantaneously.

Therefore, Microsoft massively scaled up its coaching knowledge, going from 50 settings to tons of, and used methods like artificial labelling and noise injection to show the AI how individuals phrase frequent duties. After coaching with greater than 3.6 million examples, the mannequin turned quick and correct sufficient to reply in below half a second, the corporate claimed.

One vital problem was that Mu carried out higher with multi-word queries over shorter or imprecise phrases. For example, typing “decrease display brightness at night time” offers it extra context than simply typing “brightness.” To resolve this, Microsoft continues to indicate conventional keyword-based search outcomes when a question is just too imprecise.

Microsoft additionally noticed a language-based hole. In cases when a setting may apply to greater than a single performance (as an illustration, “enhance brightness” may confer with the system’s display or an exterior monitor). To deal with this hole, the AI mannequin at the moment focuses on probably the most generally used settings. That is one thing the tech large continues to refine.