RAG or Retrieval Augmented Generation services some standard use cases.
Use Case: Customised AI system to address externally facing processes and to build business automation and prompt-query-response systems.
RAG is a customised approach and should not be confused with CoPilot out of the box. CoPilot for internal information search and gathering is built on a Graph database which accessess all information from Microsoft applications including emails, Sharepoint, Teams, Wikis etc.
| Feature | M365 Copilot (Internal Use) | Custom RAG |
| Data Scope | The Microsoft Graph (massive and diverse). Accesses everything the user has permission to see: Outlook emails, Teams chats, OneDrive files, SharePoint sites, meeting transcripts, and your organization’s Web Index. | A single, specialized data source. Typically one or more Azure AI Search indexes populated with specific, structured documents (PDFs, Markdown, etc.). |
| Security & Filtering | Native Graph Security (Real-Time). Every single query is filtered by the user’s specific identity and permissions defined in Microsoft Entra ID (formerly Azure AD). This is automatic, immediate, and non-negotiable. | Developer-Implemented Metadata Filtering. The developer must manually configure the search index with security metadata (e.g., access_group) and write the code to retrieve the user’s group and apply that filter to the search query. |
| Retrieval Focus | Federated Search. It doesn’t just search one index; it orchestrates simultaneous searches across dozens of M365 applications and data stores (called Graph Connectors). | Single-Source Retrieval. Focused on finding the best chunks from a predefined set of documents (e.g., HR Policies, Product Manuals). |
RAG is a fundamentally different approach and use case.
There are issues with RAG. It will require manual interventions, pipeline setup and maintenance and QA on all outputs and answers. The data will need to be cleaned and scrubbed and your data lake or data management will be the first and most important step in building a RAG.
For an average sized RAG deployment of some 5-10.000 semi-structured pages of data (PDFs, Docs), it will take about 3 months to design, POC, build, QA and deploy the first version.
