Scoped agents help Copilot search 10,000 SharePoint sites

Michael Slater, vice president of technology at Improving, put a number on the enterprise search problem during a recent interview on The Improving Edge, the company's podcast. Improving, which Slater described as a medium sized firm that is sometimes called a boutique, runs more than 10,000 SharePoint sites. His point was aimed at the Fortune 500 operator: if a company that size has 10,000 sites, a colleague saying "I put that in SharePoint" at a far larger enterprise means the document is, in practice, gone.

That matters for AI because the model has nothing to go on. Slater told the show's host that training data gives a large language model context about the world, and public material such as a website, SEC filings and press releases may be in there. Everything a company actually runs on is not.

The model only repeats what it finds, so finding is the whole job

Slater was blunt about what an AI assistant does when it answers an internal question. It retrieves something and repeats it back, which means the user could have found it too. The value is that the model finds it faster, and that only holds when there is a system telling it where to look.

The cost of building that system is where budgets slip. According to Dataversity, citing HyperFRAME research, 84% of surveyed enterprise infrastructure and operations leaders said AI deployment had consumed more budget and operational resources than originally planned. Dataversity attributes much of that overrun to work that happens before a model reaches production: finding, understanding, preparing and governing the data the model depends on.

For the CIO signing off on an AI line item, that reframes the purchase. The model license is the smaller number. The larger one is the registry of where data lives, the connections to structured stores, and the security that decides which user, and which agent, can see what.

An agent scoped to three sites instead of 10,000

Slater gave Microsoft its due. Copilot out of the box searches SharePoint and, in his words, does a pretty good job, and he uses it as his own search tool. The problem is scale. A question thrown at every site in the tenant has to compete with duplicate uploads, stale copies and documents nobody shared.

The fix he described lives in the premium Office 365 license. A build agent button lets a user create an agent that talks only to the SharePoint sites they care about, and can also point at a website link, a database, a Fabric connection or a OneDrive folder. Slater would not call the result guaranteed, because it is still AI, but said a scoped agent has a strong likelihood of returning the right document rather than an old copy someone re uploaded elsewhere.

He was clear this is not a Microsoft only path. Google Workspace has comparable tooling, and Slater named Glean as another provider that does the same job. What matters is the approach: give employees an enablement zone to configure their own narrow agents, because they already know which data their role needs from doing the work manually, and an IT team trying to scope for everyone at once tends to go too broad and miss.

Microsoft's own documentation for SharePoint Advanced Management shows why the scale problem is real for governance too. In its product material, Microsoft says the sensitivity label snapshot report can be downloaded as a CSV containing up to 10,000 sites, and it recommends running permission and label snapshot reports quarterly with monthly follow ups on sharing activity. A tenant at Improving's size sits right at that export ceiling, which is a useful check for any facilities or operations lead who assumes their own tenant is small.

Bronze, silver and gold are old disciplines with new labor

When the host asked how the data pipelines behind these agents get built, Slater reached back to the data warehouse era and the years of Hadoop, Spark and MapReduce. The stages now carry metal names. Bronze is raw data in its source form, which Slater said is a fancy way of saying a PDF or a Word document.

Silver is extraction. Using an invoice as the example, he described pulling line items, order number, total, and the customer's name and address into a structured database table, lightly validated. Invoice 55 becomes row 55, but nobody has yet checked whether that customer exists in the customer database.

Gold is where cross cutting validation happens. Customer, order, line item and address are extracted as entities, checked against each other, and hooked to Power BI, Tableau or whatever the reporting layer is. Slater warned that the self service reporting era produced so many reports that nobody knew which one was right, and he expects the same sprawl with agents unless enterprises apply the lesson this time.

The new part is who does the moving. Slater said AI can now look at bronze layer data, propose the silver table, write the Python, drop it into Databricks, Fabric or Snowflake, and then propose a schema across all the resulting tables. Projects that used to be long data lake or lakehouse efforts can run at rapid speed. He also pushed back on the idea that this is something other than software engineering.

The data engineer becomes the quality gate

Getting the pipeline built once is not the hard part, in Slater's account. Every time source data changes, the update has to cascade through bronze, silver and gold, and each layer has its own quality bar that cannot drop just because something moved. Otherwise the board deck is wrong and, as he put it, somebody's job might be on the line.

That changes what a data engineer is for. If the model can transform data and design the model, the human writes the gates that check the model's output.

Slater said data engineers can spend more time, especially at the gold layer, writing data quality checks to look for edge cases, and working out how to trace a wrongly processed invoice back to the source system and flag it there. A CIO.com opinion piece by Nalini Garg makes a parallel argument from the governance side: a capable model can still get the answer wrong when the business cannot agree on what its data means. Garg's framing puts the advantage in a better semantic layer rather than a bigger model, which is the same place Slater puts the data engineer.

The uncomfortable evidence is that layered strategies are common advice and projects still stall. TSG, a technology consulting firm, wrote in its own blog post that Gartner research suggests 50% of AI projects fail to move beyond the pilot stage, often because of scalability, integration and operationalization problems. TSG itself frames enterprise AI as three interconnected layers of infrastructure, platforms and applications. Layered strategies are common on paper. Slater's argument is that the quality bar at each layer, held every time the source data changes, is where the real work sits.

Personal agents first, departments second, decisions last

Slater laid out what he calls the AI triangle. Personalized AI sits at the base, where most enterprises are today. Departmental AI systems form the middle, and authoritative decision making for employees, executives and customers sits at the peak, with the heaviest security, login, guardrail and governance requirements.

The middle layer is where cross functional actions live: reading from one system, writing to another, issuing a refund, checking an order against warehousing. Slater said that is no longer a self serve agent an employee builds alone. An IT team makes the advanced configurations, often low code, matching a customer record ID in System A to the same customer in System B and putting credentials in the right place.

Context is what makes any of it useful. Slater said a well built system should know the user's job title, department and even why they are asking today. A finance leader at quarter end is probably asking about revenue or the P and L, and a slightly mistuned system will happily return last quarter's numbers or the same month a year ago. That is the CFO who gets a confident wrong answer, and the operations director who has to explain why.

Slater's advice to operators is to start small: decide which questions people should be able to ask first, give employees a place to build narrow agents, and let departmental systems grow from there. The test he points to is whether a change in source data cascades through every layer with the quality bar held at each one. If it does not, the answer coming out of the agent is the same stale document that was already lost in SharePoint.