Microsoft plans to break free from NVIDIA and deploy its next-generation Maia 300 AI accelerator in 2027, targeting 40% more tokens per megawatt than leading GPU-based systems. The chip will power Microsoft’s expanding AI infrastructure, with a 96-accelerator rack configuration designed to deliver up to seven times the token throughput of its predecessor.
The announcement marks another step in Microsoft’s effort to build custom silicon for large-scale AI inference. Its Maia 200 accelerator already handles production workloads, including Microsoft Copilot. Microsoft expects GitHub Copilot and Microsoft Foundry to adopt the hardware by the end of 2026.
Maia 300 takes a system-level approach to inference efficiency. Its architecture targets memory-intensive workloads, long-context reasoning, and AI agents that execute complex tasks across multiple steps.
Memory bandwidth drives design
Maia 300 pairs eight HBM4 memory stacks with 18 terabytes per second of peak bandwidth. The system operates within a 950-watt thermal design power envelope, addressing the growing memory demands of large AI models.
Microsoft has also adopted a multi-chiplet design that separates compute, memory controllers, and input/output functions. Compute chiplets use TSMC’s N2P manufacturing process, which aims to improve performance per watt.
The accelerator also supports enhanced MXFP4 low-precision computing. This approach reduces memory requirements for model calculations and key-value cache storage. Those savings can help systems accommodate longer context windows and serve more requests within the same power budget.
More from AI and Robotics
See AllAI and RoboticsMeta claims new Muse Wonder models are world’s best at handling complex agentic tasksAI and RoboticsChina’s tote-moving wheeled humanoid, 110-pound robot arm target factory automationAI and RoboticsVideo: Worm-like robot shifts its own weight to crawl over rocks, climb steps and swimAI and RoboticsChinese EV giant’s 31-joint humanoid robot could soon greet you at the car dealershipAI and RoboticsAmazon draws on fleet of 1 million robots to help machines learn and act in real worldSoftware plays an important role in the design. Microsoft’s software-defined dataflow architecture aims to keep data close to compute resources, reducing unnecessary transfers. AI-assisted kernel optimization also helps improve hardware utilization across different workloads.
Networking connects 96 accelerators
Maia 300 extends its architecture beyond individual chips to entire racks. Each configuration combines 96 accelerators within a 167-kilowatt power envelope.
Microsoft lists 27.6 terabytes of HBM capacity and 1.7 petabytes per second of aggregate memory bandwidth across the rack’s high-speed scale-up domain. That combination targets larger inference workloads and more simultaneous AI interactions within a single deployment.
Custom networking connects accelerators through high-bandwidth links optimized for tensor parallelism. An AI-focused transport layer uses Ethernet-based infrastructure to support communication across accelerators and racks. The design aims to scale workloads without treating networking as a separate bottleneck.
MicrosoftsaysMaia 300 should deliver 40% more tokens per megawatt than leading GPU-based systems. Its projected rack-scale throughput also represents a substantial increase over Maia 200, although the company has not detailed independent benchmarks for these comparisons.
Maia 200 expands production
Microsoft is already deploying Maia 200 across the United States, Europe and Australia. The company reports up to 1.7 times more tokens per watt for built-in AI features in Teams, Copilot, and Cowork experiences compared with leadingGPUinfrastructure.
The existing accelerator runs OpenAI models,Microsoft’sMAI models, and open-source model families. Microsoft uses performance data from these services to guide hardware development and optimize real-world workloads.
Maia 300 will support Microsoft’s model development efforts, cloud inference, and agent-based coding tools. Microsoft also expects the hardware to help scale Dragon Copilot, its clinical AI assistant.
Production deployments will begin in 2027. Microsoft’s longer-term plan calls for multi-gigawatt Maia deployments starting that year, positioning custom silicon as a major component of its expanding AI computing capacity.
Get the latest in engineering, tech, space & science - delivered daily to your inbox.
By subscribing, you agree to our
Terms of Useand
PoliciesYou may unsubscribe at any time.
0
COMMENT
Subscribe to
Today!
Access to exclusive content, expert insights and a deeper dive into engineering and tech. No ads, no limits.
Explore Now!By
Aamir KhollamAamir is a seasoned tech journalist with experience at Exhibit Magazine, Republic World, and PR Newswire. With a deep love for all things tech and science, he has spent years decoding the latest innovations and exploring how they shape industries, lifestyles, and the future of humanity.
- 1Scientists develop new catalyst for zinc-air batteries to run for over 300 hours
- 2US scientists grow better perovskite crystals, boost solar cell efficiency to 26%
- 3Video: Worm-like robot shifts its own weight to crawl over rocks, climb steps and swim
- 4Energy-importing countries record 3x faster clean energy generation than exporters
- 5Chinese EV giant's 31-joint humanoid robot could soon greet you at the car dealership
Microsoft plans to break free from NVIDIA and deploy its next-generation Maia 300 AI accelerator in 2027, targeting 40% more tokens per megawatt than leading GPU-based systems. The chip will power Microsoft’s expanding AI infrastructure, with a 96-accelerator rack configuration designed to deliver up to seven times the token throughput of its predecessor.
The announcement marks another step in Microsoft’s effort to build custom silicon for large-scale AI inference. Its Maia 200 accelerator already handles production workloads, including Microsoft Copilot. Microsoft expects GitHub Copilot and Microsoft Foundry to adopt the hardware by the end of 2026.
Maia 300 takes a system-level approach to inference efficiency. Its architecture targets memory-intensive workloads, long-context reasoning, and AI agents that execute complex tasks across multiple steps.
Memory bandwidth drives design
Maia 300 pairs eight HBM4 memory stacks with 18 terabytes per second of peak bandwidth. The system operates within a 950-watt thermal design power envelope, addressing the growing memory demands of large AI models.
Microsoft has also adopted a multi-chiplet design that separates compute, memory controllers, and input/output functions. Compute chiplets use TSMC’s N2P manufacturing process, which aims to improve performance per watt.
The accelerator also supports enhanced MXFP4 low-precision computing. This approach reduces memory requirements for model calculations and key-value cache storage. Those savings can help systems accommodate longer context windows and serve more requests within the same power budget.
More from AI and Robotics
See AllAI and RoboticsMeta claims new Muse Wonder models are world’s best at handling complex agentic tasksAI and RoboticsChina’s tote-moving wheeled humanoid, 110-pound robot arm target factory automationAI and RoboticsVideo: Worm-like robot shifts its own weight to crawl over rocks, climb steps and swimAI and RoboticsChinese EV giant’s 31-joint humanoid robot could soon greet you at the car dealershipAI and RoboticsAmazon draws on fleet of 1 million robots to help machines learn and act in real worldSoftware plays an important role in the design. Microsoft’s software-defined dataflow architecture aims to keep data close to compute resources, reducing unnecessary transfers. AI-assisted kernel optimization also helps improve hardware utilization across different workloads.
Networking connects 96 accelerators
Maia 300 extends its architecture beyond individual chips to entire racks. Each configuration combines 96 accelerators within a 167-kilowatt power envelope.
Microsoft lists 27.6 terabytes of HBM capacity and 1.7 petabytes per second of aggregate memory bandwidth across the rack’s high-speed scale-up domain. That combination targets larger inference workloads and more simultaneous AI interactions within a single deployment.
Custom networking connects accelerators through high-bandwidth links optimized for tensor parallelism. An AI-focused transport layer uses Ethernet-based infrastructure to support communication across accelerators and racks. The design aims to scale workloads without treating networking as a separate bottleneck.
MicrosoftsaysMaia 300 should deliver 40% more tokens per megawatt than leading GPU-based systems. Its projected rack-scale throughput also represents a substantial increase over Maia 200, although the company has not detailed independent benchmarks for these comparisons.
Maia 200 expands production
Microsoft is already deploying Maia 200 across the United States, Europe and Australia. The company reports up to 1.7 times more tokens per watt for built-in AI features in Teams, Copilot, and Cowork experiences compared with leadingGPUinfrastructure.
The existing accelerator runs OpenAI models,Microsoft’sMAI models, and open-source model families. Microsoft uses performance data from these services to guide hardware development and optimize real-world workloads.
Maia 300 will support Microsoft’s model development efforts, cloud inference, and agent-based coding tools. Microsoft also expects the hardware to help scale Dragon Copilot, its clinical AI assistant.
Production deployments will begin in 2027. Microsoft’s longer-term plan calls for multi-gigawatt Maia deployments starting that year, positioning custom silicon as a major component of its expanding AI computing capacity.
Get the latest in engineering, tech, space & science - delivered daily to your inbox.
By subscribing, you agree to our
Terms of Useand
PoliciesYou may unsubscribe at any time.
0
COMMENT
Subscribe to
Today!
Access to exclusive content, expert insights and a deeper dive into engineering and tech. No ads, no limits.
Explore Now!By
Aamir KhollamAamir is a seasoned tech journalist with experience at Exhibit Magazine, Republic World, and PR Newswire. With a deep love for all things tech and science, he has spent years decoding the latest innovations and exploring how they shape industries, lifestyles, and the future of humanity.
- 1Scientists develop new catalyst for zinc-air batteries to run for over 300 hours
- 2US scientists grow better perovskite crystals, boost solar cell efficiency to 26%
- 3Video: Worm-like robot shifts its own weight to crawl over rocks, climb steps and swim
- 4Energy-importing countries record 3x faster clean energy generation than exporters
- 5Chinese EV giant's 31-joint humanoid robot could soon greet you at the car dealership
