Enterprise AI (RAG): compliance, reliability, costs, etc. The complete guide.

Companies have a vast documentary heritage: procedures, contracts, meeting minutes, knowledge bases, support tickets, CRM content, product sheets, HR documents, internal standards, and security policies. Yet this knowledge often remains difficult to find, interpret, and reuse.

The RAG, for Retrieval-Augmented Generation, connects generative AI to the company’s internal data to produce more contextualized, verifiable, and business-appropriate responses.

This guide explains what a RAG architecture is, when to use it, how to design it, its potential costs, which risks to anticipate, and how to address security, confidentiality, and compliance challenges.

Generative AI has introduced a new way to interact with information: asking a question in natural language and receiving a structured answer. For businesses, the promise is significant: save time, reduce reliance on a few internal experts, streamline access to knowledge, and improve the quality of operational decisions.

However, a general-purpose model used alone is unaware of the company’s internal documents. It can assist with writing, rephrasing, or structuring, but it cannot spontaneously answer based on your contracts, procedures, quality database, or technical documentation. This is precisely the role of RAG: to provide the model with the right documentary excerpts at the right time, so it can respond based on a controlled business context.

RAG is therefore not simply about “connecting ChatGPT to files.” It is a comprehensive architecture that combines data, document retrieval, security, governance, user experience, evaluation, and maintenance. Recent sources confirm that enterprise RAG implementations must address complex issues: query understanding, multi-source access, token constraints, response times, access rights, and content governance.

In this white paper, you will learn:

• what RAG is, simply put;
• when it is relevant;
• when it should not be used;
• which technical components make up a RAG architecture;
• which structural decisions to make;
• which costs to anticipate;
• how to address security, GDPR, AI Act, and confidentiality;
• how to launch a RAG project in a gradual and realistic way.

Key takeaways
RAG is a pragmatic answer to a very common problem: businesses have a wealth of knowledge, but it is scattered, hard to access, and rarely usable in natural language.

The principle is simple:

1. the user asks a question;
2. the system searches internal documents for the most relevant passages;
3. these passages are added to the context sent to the language model;
4. the model generates a response based on these excerpts;
5. the response can cite the sources used.

AWS describes RAG as a technique that augments a large language model with external data, such as internal documents, to provide the model with the context needed to produce a useful response in a specific use case.

A simple analogy

Imagine an expert who needs to answer a complex question. Without documents, they rely on their memory. With RAG, the right files are automatically opened to the right pages before they respond.

The language model acts as the writer who formulates the answer. The search engine acts as the archivist who retrieves the sources. The RAG architecture organizes the collaboration between the two.

RAG vs. Traditional Chatbot

A traditional chatbot responds based on what it learned during training or its initial prompt. It can be fluent, but it risks producing overly generic or misaligned responses with your internal rules.

A RAG chatbot, however, first searches your internal sources: HR policies, product documentation, quality procedures, contracts, FAQs, support tickets, commercial documents. The response is thus more grounded in your operational reality.

RAG vs. Fine-Tuning

The fine-tuning consists of retraining or specializing a model using examples. It can be useful for learning a style, response structure, classification, or specific behavior.

But for leveraging internal documents that change regularly, RAG is often more suitable. It allows knowledge updates by reindexing or syncing sources, without retraining the model. It also facilitates source citation, traceability, and rights management.

ApproachUseful forLimitations

Simple prompt

One-off tasks, writing, rephrasing

Not connected to internal data

Fine-tuning

Style, classification, recurring behaviors

Less suited to evolving knowledge

RAG

Document search, sourced answers, internal knowledge

Highly dependent on data quality and retrieval

Key takeaway
RAG is generally more relevant than fine-tuning when the main challenge is leveraging an evolving internal knowledge base with sourced answers.

Issue: employees frequently ask the same questions about leave, remote work, expense reports, benefits, and onboarding processes.

RAG solution: connect an assistant to HR policies, internal agreements, FAQs, and procedures.

Business value: reduced repetitive requests, greater employee autonomy, consistent and traceable answers.

Augmented customer support

Issue: support teams must search through past tickets, product sheets, escalation procedures, and knowledge bases.

RAG solution: provide suggested answers from internal sources, with citations of the documents used.

Business value: reduced processing time, improved response quality, leveraging past incidents.

Intelligent document search engine

Problem: documents are scattered across SharePoint, Google Drive, Notion, Confluence, ERP, CRM, or PDF files.

RAG solution: index priority sources and enable users to ask questions in natural language.

Business value: faster access to information, reduced dependency on experts, better knowledge reuse.

Legal or compliance assistant

Problem: teams need to quickly retrieve clauses, obligations, procedures, or internal decisions.

RAG solution: query contracts, policies, reference frameworks, internal notes, and validated archives.

Business value: accelerated search, more effective analysis preparation, better traceability.

Key consideration: this type of assistant must be carefully framed. It can help retrieve and synthesize information but should not be presented as automated legal advice.

Sales assistant

Problem: sales teams waste time finding the right sales pitches, customer cases, offer sheets, objection responses, or proposal elements.

RAG solution: create an assistant connected to sales documentation and validated content.

Business value: faster responses to prospects, message consistency, faster onboarding of new hires.

Technical knowledge base

Issue: technical documentation is extensive, partially outdated, and difficult to navigate.

RAG solution: query manuals, tickets, changelogs, schemas, maintenance procedures, and internal guides.

Business value: faster diagnostics, reduced errors, and field team support.

Employee onboarding

Issue: a new employee must quickly absorb a large volume of documents.

RAG solution: an onboarding assistant capable of answering questions based on validated internal documents.

Business value: smoother integration, reduced time spent by managers answering repetitive questions.

Key takeaway
The best RAG use cases are those where answers already exist in documents but are hard to find, synthesize, or contextualize.

5. When RAG is not the right solution

RAG is not a one-size-fits-all solution.

Poorly structured or insufficient data

If documents are incomplete, contradictory, outdated, or scattered without logic, RAG may produce subpar answers. A RAG assistant does not magically fix a weak knowledge base.

Need for complex business reasoning

RAG excels at retrieving and synthesizing information. It is less suited for applying complex business rules, calculating scenarios, arbitrating between multiple constraints, or executing transactional processes.

In such cases, a dedicated business application, a rules engine, or a structured workflow may be more appropriate.

Unreliable data

If no one knows which document is authoritative, the AI won’t either. The project must start by clarifying authorized sources, outdated documents, and update owners.

Automation rather than document search

If the task mainly involves sending reminders, filling out forms, routing tickets, or updating a CRM, a standard automation may suffice.

Absolute guarantees required

RAG can reduce hallucinations, improve traceability, and provide sources. However, it does not guarantee that every answer will be perfect. For critical decisions, human validation, deterministic rules, or control processes must be implemented.

Key takeaway
Good scoping also means knowing when to say no to RAG. If the need is better suited to a workflow, a rules engine, or a traditional business application, it should be acknowledged.

Poor or poorly structured data

If documents are incomplete, contradictory, outdated, or scattered without logic, RAG may produce subpar answers. A RAG assistant does not magically fix a weak knowledge base.

Need for complex business reasoning

RAG excels at retrieving and synthesizing information. It is less suited for applying complex business rules, calculating scenarios, arbitrating between multiple constraints, or executing transactional processes.

In such cases, a business application, a rules engine, or a structured workflow may be more appropriate.

Unreliable data

If no one knows which document is authoritative, the AI won’t either. The project must start by clarifying authorized sources, outdated documents, and update owners.

Automation over document search

If the task mainly involves sending reminders, filling out forms, routing tickets, or updating a CRM, a standard automation may suffice.

Absolute guarantees required

RAG can reduce hallucinations, improve traceability, and provide sources. However, it does not guarantee that every answer will be perfect. For critical decisions, human validation, deterministic rules, or control processes must be implemented.

Key takeaway
Good scoping also means knowing when to say no to RAG. If the need is better suited to a workflow, a rules engine, or a traditional business application, it should be acknowledged.

Internal sources
├─ PDF, Word, PowerPoint
├─ SharePoint / Google Drive / Notion / Confluence
├─ CRM / ERP / support tickets
└─ business databases

↓

Document ingestion
├─ text extraction
├─ OCR if needed
├─ cleaning
├─ structuring
└─ metadata detection

↓

Chunking
├─ by sections
├─ by headings
├─ by paragraphs
└─ with controlled overlap

↓

Embeddings + indexing
├─ embedding model
├─ vector database
├─ optional lexical index
└─ metadata and permissions

↓

User query
↓

Retrieval
├─ vector search
├─ keyword search / BM25
├─ hybrid search
├─ access rights filters
└─ reranking

↓

Enriched prompt
├─ question
├─ relevant excerpts
├─ response instructions
└─ citation rules

↓

LLM
↓

User response
├─ summary
├─ citations
├─ limitations
└─ possible actions

↓

Logs, feedback, evaluation, monitoring

Data sources

Sources can be simple, such as a folder of PDFs, or complex, like a set of internal systems. The priority is not to connect everything, but to connect the right sources.

Document ingestion

Ingestion involves extracting, cleaning, and transforming documents to make them usable. Scanned PDFs, tables, images, diagrams, and poorly formatted documents may require OCR, advanced parsing, or manual reprocessing.

AWS Bedrock Knowledge Bases, for example, describes managed workflows covering ingestion to retrieval, including text chunking, embeddings, and storage in compatible vector databases.

Chunking

Chunking involves splitting documents into segments. Overly long chunks dilute information. Overly short chunks lose context. The strategy depends on the document type: procedures, contracts, FAQs, technical documentation, or support tickets.

LlamaIndex notes that changing chunk size and overlap alters the embeddings generated, and that smaller chunks may be more precise while larger chunks may be more general.

Embeddings and vector database

Embeddings transform text into numerical vectors, enabling semantic proximity measurement between a query and documentary passages.

The vector database stores these vectors and allows retrieval of the passages most relevant to a query.

Hybrid search and reranking

Vector search is useful, but not always sufficient. Modern architectures often combine:

• vector search;
• keyword search;
• metadata filters;
• reranking;
• citations;
• access control.

Microsoft outlines common RAG pipeline techniques: full-text search, vector search, chunking, hybrid search, query rewriting, and reranking. Pinecone also explains that two-step reranking can improve quality by combining large-scale search with more precise re-ranking.

Access rights management

A critical point: a user must not receive an answer based on a document they do not have access to. The architecture must enforce permissions at the search stage, not just in the interface.

Microsoft highlights the need for granular access control when private content is exposed to LLMs, using mechanisms such as security trimming, query-time filters, and network isolation.

Key takeaway
A robust RAG architecture is a complete chain: ingestion, document quality, search, permissions, prompt, model, citations, logs, and continuous improvement.

The choice of model depends on several criteria:

• response quality;
• cost per token;
• latency;
• maximum context;
• regional availability;
• contractual commitments;
• multilingual capability;
• compatibility with security requirements.

A highly powerful model is not always necessary. For many RAG use cases, the best trade-off is to use a fast, cost-effective model for simple queries and a more advanced one for complex cases.

External API or self-hosted model?

Option|Advantages|Limitations

External API

Quick to deploy, high-performing models, reduced maintenance

Vendor dependency, contractual clauses to review, variable costs

Self-hosted model

Enhanced control, environment mastery, potential sovereignty

Infrastructure, MLOps, monitoring, higher costs and required expertise

Hybrid

Optimisation by use case

More complex architecture

Which vector database?

The choice depends on volume, latency, filtering, security model, cloud environment, and in-house expertise. An SME with a few thousand documents does not have the same needs as an international group with millions of chunks.

Which chunking strategy?

Testing is required. The right segmentation depends on the documents. A FAQ can be split question by question. A contract must retain clauses and sub-clauses. A procedure must keep steps together.

How to manage metadata?

Metadata is essential:

• document type;
• date;
• version;
• author;
• department;
• confidentiality;
• language;
• client or project concerned;
• access rights;
• status: draft, validated, obsolete.

They help filter, prioritise, and explain responses.

How to limit hallucinations?

No method eliminates them entirely. However, they can be reduced:

• high-quality retrieval;
• mandatory citations;
• instruction not to answer without a source;
• confidence threshold;
• displaying limitations;
• regular evaluation;
• user feedback;
• adversarial testing;
• human validation for sensitive cases.

How to assess quality?

A RAG system should be evaluated with real questions. Criteria may include:

• accuracy;
• completeness;
• faithfulness to sources;
• ability to say “I don’t know”;
• relevance of citations;
• latency;
• satisfaction rate;
• reduced search time;
• no data leakage.

Key takeaway
Technical decisions are not just technical. They impact cost, security, answer quality, maintenance, and user adoption.

Main cost drivers

ItemDescriptionImpact

Scoping

business workshops, objectives, success criteria

essential to avoid a useless POC

Document audit

sources, quality, rights, obsolescence

often underestimated

Data preparation

cleaning, structuring, OCR, metadata

can become significant

Development

ingestion, retrieval, interface, orchestration

core of the project

Integrations

SSO, SharePoint, CRM, ERP, ticketing

depends on the IS

LLM

cost of API calls or model infrastructure

depends on volume

Embeddings

initial indexing and updates

depends on document volume

Vector database

storage, queries, availability

depends on scale

Security

IAM, logs, encryption, testing

non-negotiable

Monitoring

cost tracking, errors, quality

essential in production

Maintenance

fixes, reindexing, improvements

recurring cost

Indicative costs of major LLM providers

Rates change frequently. The figures below should be read as orders of magnitude observed as of May 27, 2026, and verified before any contractual commitment.

ProviderExample modelsIndicative public prices

OpenAI

GPT-5.5, GPT-5.4, GPT-5.4 mini

GPT-5.5: $5 / 1M tokens input and $30 / 1M tokens output; GPT-5.4: $2.50 / 1M input and $15 / 1M output; GPT-5.4 mini: $0.75 / 1M input and $4.50 / 1M output.

Anthropic

Claude Opus 4.7, Sonnet 4.6, Haiku 4.5

Opus 4.7: $5 / MTok input and $25 / MTok output; Sonnet 4.6: $3 / MTok input and $15 / MTok output; Haiku 4.5: $1 / MTok input and $5 / MTok output.

Google

Gemini 3.5 Flash, Gemini 2.5 Pro, Gemini 2.5 Flash, Flash-Lite

Gemini 3.5 Flash standard: $1.50 / 1M input and $9 / 1M output; Gemini 2.5 Pro: $1.25 / 1M input up to 200k tokens and $10 / 1M output; Gemini 2.5 Flash: $0.30 / 1M input text/image/video and $2.50 / 1M output; Flash-Lite: $0.10 / 1M input and $0.40 / 1M output.

Mistral AI

Mistral Small 4, Large 3, Medium 3.5

Small 4: $0.15 / M tokens input and $0.60 / M tokens output; Large 3: $0.50 / M input and $1.50 / M output; Medium 3.5: $1.50 / M input and $7.50 / M output.

Simplified cost calculation example

Assume:

• 200 users;
• 20 questions per user per business day;
• 22 days per month;
• 1,500 input tokens per question, including context;
• 500 output tokens per answer.

This amounts to approximately:

• 132 million input tokens per month;
• 44 million output tokens per month.

Monthly costs will vary significantly depending on the model chosen. With an economy model, they can remain manageable. With a premium model, they can become substantial. The key is to control costs based on actual usage, not just the listed price.

What drives costs up

• sending too much context to the model;
• use a premium model for all queries;
• unnecessarily reindex the entire document base;
• multiply LLM calls in an agentic workflow;
• omit caching;
• fail to limit retrieved documents;
• neglect usage monitoring;
• connect non-priority sources;
• process large volumes of scanned PDFs without a strategy.

How to control costs

• select the cheapest model that meets the expected quality level;
• limit the context sent;
• use reranking to send fewer but more relevant chunks;
• cache recurring responses or contexts;
• separate simple and complex queries;
• track costs by team, usage, source, and query type;
• set quotas;
• start with a limited document scope.

Key takeaways
The cost of RAG depends less on the “model price” than on the full architecture: document volume, number of users, context length, usage frequency, retrieval quality, and integration level.

As soon as a RAG system processes personal data, GDPR must be considered: purpose, legal basis, data minimisation, retention period, data subject rights, security, subcontracting, and documentation.

The CNIL notes that GDPR does not prevent AI innovation but requires addressing risks to individuals when personal data is used. It emphasises the need for a legal basis, minimisation, and documentation of processing activities.

In a RAG project, the questions to ask are practical:

• do the documents contain personal data?
• sensitive data?
• HR data?
• customer data?
• health data?
• confidential contractual information?
• information covered by professional secrecy?
• do all users have the right to access this information?

AIPD

A Data Protection Impact Assessment (DPIA) may be necessary or strongly recommended when processing poses high risks. The CNIL notably recommends conducting a DPIA for AI system development in several scenarios, such as when sensitive data is collected, personal data is processed on a large scale, data from vulnerable individuals is used, or datasets are cross-referenced.

AI Act

The AI Act introduces a risk-based approach. Obligations vary depending on whether the system is prohibited, high-risk, subject to transparency requirements, or minimal risk. The European Commission states that prohibited practices and AI literacy obligations have applied since 2 February 2025, that rules for general-purpose AI models have applied since 2 August 2025, and that the timeline for high-risk systems was adjusted under the Digital Omnibus, with some rules set for 2 December 2027 and others for 2 August 2028, depending on the categories.

For a company deploying an internal RAG assistant, the main challenge is to qualify the use case: simple document assistant, HR tool, decision support, tool used in recruitment, training, compliance, credit, healthcare, safety, or other sensitive areas. European guidelines on classifying high-risk systems were still under consultation and adjustment as of May 2026.

RAG system security

The main risks are:

• sensitive data leaks;
• incorrect access rights enforcement;
• prompt injection;
• indirect prompt injection via documents;
• system prompt exposure;
• document poisoning;
• unsourced responses;
• logs containing sensitive data;
• over-permissioned agents;
• vendor lock-in;
• lack of auditability.

The OWASP Top 10 for LLM Applications 2025 ranks prompt injection, disclosure of sensitive information, supply chain risks, data/model poisoning, excessive agency, system prompt leakage, vector and embedding weaknesses, disinformation, and unbounded resource consumption among the major risks of LLM applications.

The UK’s NCSC notes that prompt injection is not comparable to a simple SQL injection: LLM systems do not inherently separate data from instructions. It recommends treating LLMs as “inherently confusable” components and mitigating risks through design, privilege limitation, monitoring, and deterministic safeguards.

Recommended measures

• SSO and strong authentication;
• user-based access control;
• document filtering at query time;
• encryption in transit and at rest;
• environment segmentation;
• separation of dev, test, and prod;
• controlled logging;
• log retention policy;
• masking or pseudonymisation where relevant;
• security testing;
• AI red teaming;
• internal usage policy;
• contractual review of vendors;
• subprocessing clauses;
• monitoring of transfers outside the EU;
• documentation of decisions.

The NIST AI RMF Generative AI Profile provides a voluntary framework for integrating trust, governance, measurement, and risk management considerations into generative AI systems. The Cloud Security Alliance also recommends applying Zero Trust principles to LLM environments: least privilege, micro-segmentation, continuous monitoring, identity management for both human and non-human entities, and governance.

Key takeaways
RAG security does not rely on a magic prompt. It depends on architecture: access rights, isolation, logging, encryption, monitoring, testing, and governance.

Selecting a vector database or model before clarifying the business need often leads to an impressive but useless proof of concept.

Connect all data

The more sources there are, the higher the risks: document noise, obsolescence, unclear rights, costs, latency. It’s better to start with a limited, reliable corpus.

Overlooking document quality

A high-performing RAG relies on reliable, up-to-date, and well-structured documents. Corpus quality is often more important than model choice.

Ignoring access rights

This is one of the most dangerous mistakes. An assistant must never become a backdoor to access confidential documents.

Omitting citations

Without sources, users cannot verify answers. Citations improve trust and make corrections easier.

Not testing responses

You need to create a set of real questions with expected answers and evaluation criteria. Otherwise, quality remains subjective.

Underestimating maintenance

Documents change, sources move, models evolve, pricing varies, users discover new use cases: a RAG must be maintained.

Confusing prototype with product

A prototype can overlook certain aspects: fine-grained security, monitoring, UX, error handling, costs. A product cannot.

Launching a POC without success criteria

A POC must answer clear questions: does it save time? Are the answers reliable? Do users adopt it? Is the cost acceptable?

Key takeaways
Most RAG failures stem not from the model, but from poor scoping, a weak corpus, overlooked security, or a lack of evaluation.

1. Identify the business need

Frame the problem without mentioning AI: “support teams spend 20 minutes finding a procedure”, “new hires keep asking the same questions”, “sales teams don’t know which document to use”.

2. Prioritise use cases

Not all use cases are equal. Focus on those that combine business value, document feasibility, and manageable risk.

3. Audit available data

List sources, their quality, freshness, format, owner, and confidentiality level.

4. Check security/GDPR constraints

Identify personal, sensitive, or confidential data. Define rights, roles, and documentation obligations.

5. Design a target architecture

Select components: ingestion, embeddings, vector database, model, interface, logs, monitoring, authentication.

6. Develop a limited prototype

Limit scope: one team, one corpus, one use case, a simple interface.

7. Test with real cases

Use actual user questions, not just ideal demonstrations.

8. Measure quality

Track accuracy, citations, justified non-response rate, satisfaction, time saved, errors, costs.

9. Deploy gradually

Expand in waves: more users, more sources, more features.

10. Maintain and improve

Reindex, correct, monitor, train, adjust prompts, review costs, improve sources.

Key takeaways
A successful RAG project resembles a business product more than an isolated technical experiment.

Business problem

• Who is losing time today?
• What problem do we want to solve?
• Which decision or task will be improved?
• What would be a useful result in 3 months?

Users

• Who will use the tool?
• How many users?
• What user profiles?
• What different access rights?
• What level of training?

Data

• Which sources will be used?
• Are the documents reliable?
• Are they up to date?
• Who maintains them?
• Are there duplicates?
• Are there outdated documents?
• Are the formats usable?

Security and compliance

• Do the documents contain personal data?
• Sensitive data?
• Confidential information?
• Are access rights clearly defined?
• Is a DPIA required?
• Which vendors will have access to the data?
• Where will the data be hosted?
• What is the retention period for logs?

Response and quality

• Should the model cite its sources?
• Should it refuse to answer without a source?
• What level of accuracy is expected?
• Who validates the answers?
• How do you correct a wrong answer?
• How do you measure quality?

Costs

• What initial budget?
• What recurring budget?
• How many queries are expected?
• Which model to use?
• What usage thresholds?
• What monitoring?

Deployment

• Prototype or product?
• Which team leads?
• What timeline?
• What user support?
• What maintenance plan?

Key takeaways
If you cannot answer this checklist, the project is likely not ready for development.

A B2B services company with 600 employees has:

• 1,200 internal procedures;
• 300 contract templates;
• 450 support FAQs;
• 20,000 historical tickets;
• project committee minutes;
• a poorly structured SharePoint database;
• several outdated but still accessible documents.

Support, sales, and operations teams spend considerable time searching for information. Answers vary depending on the person asked. New employees rely heavily on internal experts.

Initial problem

The company wants to enable its teams to ask questions in natural language:

• “What is the escalation procedure for a critical incident?”
• “Which clause template should be used for a public sector client?”
• “How to handle an out-of-time refund request?”
• “What are the prerequisites before going live?”

It requires sourced answers, limited to validated documents, while respecting access rights.

Chosen approach

The project starts with a limited scope: validated support procedures and FAQs. Contracts and historical tickets are excluded from the first batch as they contain more sensitive data and document noise.

Architecture

• SSO via internal identity;
• ingestion of validated documents from SharePoint;
• text extraction and cleaning;
• metadata: department, date, version, owner, status;
• chunking by title and section;
• embeddings;
• vector database with rights-based filters;
• hybrid search;
• reranking;
• answer with citations;
• anonymised logs where possible;
• user feedback;
• quality/cost dashboard.

Identified risks

• outdated documents still present;
• inconsistent SharePoint permissions;
• contradictory procedures;
• out-of-scope questions;
• risk of copying and pasting customer data into prompts;
• unrealistic user expectations.

Measures taken

• manually validated corpus;
• exclusion of unapproved documents;
• refusal to answer without a source;
• display of documents used;
• usage policy;
• short user training;
• cost thresholds;
• weekly feedback review at launch.

Expected benefits

• reduced search time;
• greater consistency in responses;
• faster onboarding;
• reduced requests to experts;
• identification of documentation gaps.

Limitations

• the assistant does not replace experts;
• it does not yet handle contracts;
• complex answers require validation;
• quality depends on document updates.

Next steps

After 8 weeks of pilot:

1. measure quality;
2. identify unanswered questions;
3. correct the corpus;
4. gradually integrate non-sensitive contract templates;
5. add simple workflows, such as opening a ticket or suggesting a procedure.

Key takeaways
A realistic RAG project starts small, proves its value, corrects its data, then scales gradually.

RAG is one of the most useful architectures for applying generative AI to a company’s internal data. It transforms a hard-to-exploit document base into a conversational interface capable of searching, synthesising, and citing its sources.

But a successful RAG project is not just about connecting a chatbot to documents. Its success depends on several factors:

• a clear business need;
• a reliable corpus;
• a robust retrieval architecture;
• strict access control;
• a security-by-design approach;
• compliance with GDPR and the AI Act;
• cost management;
• a simple user experience;
• continuous evaluation;
• real maintenance.

RAG isn’t magic. But when designed properly, it can become a highly practical tool to reduce search time, leverage internal knowledge, streamline operations, and improve the quality of business responses.

Key takeaway
The value of RAG doesn’t come from the model alone. It comes from the whole: data, architecture, security, use cases, governance, and adoption.

Scroll helps you frame the need, assess feasibility, design the architecture, and develop a solution tailored to your business, technical, and regulatory constraints.

Our approach is pragmatic:

• start with the business need;
• audit available data;
• identify risks;
• choose a suitable architecture;
• develop a useful prototype;
• measure quality;
• prepare for secure deployment.

Let’s discuss your RAG project

Before developing, have your AI project scoped: use cases, data, architecture, costs, security, compliance, and deployment roadmap.

Sources used

Project brief

User-provided brief for the white paper’s positioning, structure, target audience, tone, and constraints.

RAG and modern architectures

Microsoft Learn, “Retrieval-augmented generation in Azure AI Search”, accessed May 27, 2026. Source used for enterprise RAG challenges: query understanding, multi-source access, token constraints, governance, security, classic RAG, and agentic retrieval.

Microsoft Cloud Blog, “Common retrieval augmented generation techniques explained”, February 4, 2025. Source used for RAG techniques: full-text search, vector search, chunking, hybrid search, query rewriting, and reranking.

AWS Prescriptive Guidance, “Understanding Retrieval Augmented Generation”. Source used for the definition of RAG, basic steps, and the role of embeddings, vector databases, and orchestrators.

AWS, “Amazon Bedrock Knowledge Bases”. Source used for managed RAG workflows, ingestion, vector stores, advanced chunking, GraphRAG, reranking, and source attribution.

LlamaIndex, “Basic Strategies”. Source used for considerations on chunk size, chunk overlap, embeddings, and hybrid search.

Pinecone, “Rerankers and Two-Stage Retrieval”. Source used to explain the value of reranking and two-stage retrieval.

GDPR, CNIL, and personal data

CNIL, “Developing AI systems: CNIL recommendations for GDPR compliance”, 22 July 2025. Source used for GDPR principles applicable to AI: legal basis, data minimisation, security, sensitive data, DPIA, and documentation.

CNIL, sections on legal basis, minimisation, and the responsibilities of data controllers and processors.

CNIL, sections on DPIA and risk criteria.

European Data Protection Board, thematic page “Artificial intelligence”, including Opinion 28/2024 on certain data protection aspects related to AI models.

AI Act

European Commission, “AI Act | Shaping Europe’s digital future”. Source used for the risk-based approach, GPAI obligations, transparency requirements, implementation timeline, and adjustments linked to the Digital Omnibus.

AI Act Service Desk, “Timeline for the Implementation of the EU AI Act”. Source used for the phased implementation timeline and the 2025–2027 milestones, with reference to the Digital Omnibus context.

European Commission, consultation of 19 May 2026 on guidelines for classifying high-risk AI systems. Source used to note that the classification of high-risk systems remains a case-by-case assessment.

European Commission, “Standardisation of the AI Act”. Source used for the link between high-risk rules, harmonised standards, and the adjusted timeline.

AI security, LLMs, and RAG

OWASP GenAI Security Project, “2025 Top 10 Risk & Mitigations for LLMs and Gen AI Apps”. Source used for key LLM risks: prompt injection, disclosure of sensitive information, supply chain, poisoning, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, and unbounded consumption.

OWASP, “LLM01:2025 Prompt Injection”. Source used to clarify that RAG and fine-tuning do not fully eliminate prompt injection risks.

NIST, “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile”, published 26 July 2024. Source used for the generative AI risk management framework.

NCSC, “Prompt injection is not SQL injection (it may be worse)”, 8 December 2025. Source used to explain why prompt injection is a structural risk of LLMs and why it must be managed through architecture, impact limitation, and monitoring.

Cloud Security Alliance, “Using Zero Trust to Secure Enterprise Information in LLM Environments”, 3 February 2026. Source used for Zero Trust recommendations applied to LLM environments: least privilege, micro-segmentation, continuous monitoring, IAM, and governance.

CISA, “New Best Practices Guide for Securing AI Data Released”, 22 May 2025. Source used for best practices in securing data used to train and operate AI systems.

LLM pricing and API costs

OpenAI, “API Pricing”, accessed 27 May 2026. Source used for GPT-5.5, GPT-5.4, GPT-5.4 mini, batch API pricing, and cost-related options.

Anthropic, “Claude API Pricing”, accessed 27 May 2026. Source used for Claude Opus 4.7, Sonnet 4.6, and Haiku 4.5 pricing.

Google AI for Developers, “Gemini Developer API pricing”, accessed 27 May 2026. Source used for Gemini 3.5 Flash, Gemini 2.5 Pro, Gemini 2.5 Flash, Gemini 2.5 Flash-Lite, and Gemini embeddings pricing.

Mistral AI Docs, model sheets for Mistral Small 4, Mistral Large 3, and Mistral Medium 3.5, accessed 27 May 2026. Sources used for indicative pricing per million tokens and context characteristics.