Agent Evaluation with Custom Graders in Copilot Studio
Monday, August 31, 2026 · Maximilian Kenfenheuer
#MicrosoftAI #Microsoft365Copilot #CopilotStudio #AIGovernance #AgenticAI
"The demo worked" and "users liked it" are not enough once agents support customer interactions, approvals or system actions.
Copilot Studio now offers custom graders for agent evaluations. Organizations assess agents against their own criteria: approved terminology, correct internal sources, escalation at low confidence and compliance with process boundaries.
In this article, I explain why evaluation and governance belong together and how organizations can define and measure agent quality in five steps.
Ten Generic Tools: The Work IQ MCP Server and Agent Architecture
Sunday, August 30, 2026 · Maximilian Kenfenheuer
#Microsoft365Copilot #MicrosoftAI #CopilotStudio #AI #EnterpriseAI
The Work IQ API endpoints reach general availability on June 16. Work IQ combines Chat, Context, Tools and Workspaces and turns intelligence into a reusable service layer for agents.
One design choice stands out: the Work IQ MCP server reduces hundreds of operations to 10 generic tools, and getSchema lets agents discover data structures dynamically. Usage is independent of Copilot licensing and billed on consumption.
In this article, I explain what this architecture means for consistency, speed and governance of custom agents.
Evaluating Models Instead of Benchmarks: Model Choice in Copilot
Saturday, August 29, 2026 · Maximilian Kenfenheuer
#Microsoft365Copilot #MicrosoftAI #AI #Copilot #EnterpriseAI
Microsoft 365 Copilot now offers more models, including Anthropic models and new frontier reasoning options for agentic work.
A sales summary, a policy review and a multi-step agent workflow place different demands on a model. Enterprises do not want endless model sprawl. They want controlled choice inside a governed platform.
In this article, I explain why structured evaluation matters more than benchmark narratives and how organizations can group use cases for their model strategy.
Domain Exclusion in Practice: Ownership, Maintenance and Limits
Saturday, August 29, 2026 · Maximilian Kenfenheuer
#Microsoft365Copilot #MicrosoftAI #Copilot #AIGovernance #ResponsibleAI
AI governance used to ask whether employees may use AI and which model is approved. Today, organizations need finer controls, including which external sources may influence Copilot answers.
Domain exclusion lets admins exclude up to 1,000 websites from web grounding in Microsoft 365 Copilot. The scope is limited to web page results, and the list is maintained via PowerShell and CSV.
In this article, I explain why source controls shape answer quality and who in the organization should own the exclusion list.
Copilot Tasks: AI That Works in the Background and Asks Before Acting
Wednesday, August 26, 2026 · Maximilian Kenfenheuer
#Microsoft365Copilot #MicrosoftAI #Copilot #AI #EnterpriseAI #DigitalTransformation
With Copilot Tasks, AI works in the background with its own browser, runs tasks once, on a schedule or recurring, and reports back when the work is done. Before sending messages or spending money, it asks for consent.
Tasks starts as a research preview. The signal for enterprises is clear: users will soon expect AI to do things, not only to describe them.
In this article, I explain which tasks suit this execution model, where approval boundaries belong and how to measure success.