A team connects a generative assistant to its internal documents. Within three weeks, it summarizes, drafts, answers clients. No one on that team can say which data it has indexed, which answers it has returned, or to whom.
The security reflex, faced with this scene, is to lock down access: who can open the tool, with which authentication. A generative AI has no perimeter to lock down, though. It ingests data, transforms a model, produces text, calls tools, then learns from what is fed back to it. Risk travels through this workflow. It exits through the link no one has governed.
Here is the thesis of this article: a generative AI's security is distributed across the six steps of its workflow, from data acquisition to the feedback loop, and the whole system is only as strong as its least-governed link.
In short: a generative AI's security is governed across six steps along its workflow (data acquisition, model, generation, deployment, monitoring, feedback loop). At each step, a single question stands in for the control: what could leave here without anyone knowing, and could you prove it? The inventory that answers that question is the one ISO/IEC 42001, the NIST AI RMF, the AI Act and Loi 25 already require.
Where does risk travel within a generative AI?
Data crosses the system end to end. It enters through the training or indexing corpus, passes through an often third-party model, comes out as text, transits through tools that agents call, then returns, at times, into the model to improve it. Each passage is a place where sensitive data can escape or a result becomes impossible to reconstruct. Securing this whole means following it link by link, where the data actually passes, rather than at the network's front door.
What are the six steps to govern?
| Step | The risk specific to the step | The minimal control |
|---|---|---|
| 1. Data acquisition | Sensitive, personal or licensed data pulled untriaged into the training or indexing corpus. | Classification and minimization before ingestion; register of sources. |
| 2. Model | Opaque third-party model, unpinned version, a dependency whose contractual guarantees remain unknown. | Inventory of models and versions; model cards retained. |
| 3. Generation | Prompt injection, data leakage in the prompt or the output, hallucination presented as fact. | Input and output filtering; guardrails on sensitive data. |
| 4. Deployment | Over-broad access, agents calling tools without control, silent privilege escalation. | Least privilege; review agent rights as you would humans'. |
| 5. Monitoring and compliance | No usable trace; impossible to reconstruct what was produced, for whom, from what. | Logging of prompts and outputs; mapping to applicable frameworks. |
| 6. Feedback loop | Reintroduction of usage data into the model or index without consent or review. | Explicit reuse policy; human checkpoint before reinjection. |
None of these six lines can be bought. They are decisions, made and written down, at precise points in the workflow. That is what separates a defensible security program from a reassuring dashboard.
Active protection takes over from monitoring
The first reflex faced with a new risk is to observe it: install monitoring, collect alerts, wait. Observing a leak lets it happen. Maturity is measured by the move from monitoring to active protection, the capacity to act at the moment data tips over: masking personal data in a prompt before it reaches the model, blocking an output that contains a secret, tightening access that has grown too broad. A control that merely records always arrives after the fact.
Operationalization, the link programs underestimate
The tool that detects a problem does not resolve it. Someone validates the alert, assigns ownership, decides the action and carries it out. That is where most programs stall: the technology is budgeted, the people and the decision flow are not. A finding with no owner is a finding that sleeps. Before any purchase, three answers matter: who receives the alert, who decides, in how much time.
A security control exists only if it carries a name: the person who answers when it fires.
Where to start without locking yourself in?
Covering all six steps at once exhausts a program before it produces an effect. Practice teaches the opposite: pick the most exposed link first, often generation and deployment, where your data meets a third-party model, and instrument it thoroughly before broadening. Two principles hold the whole together: align every control with a legible business outcome, and favour interoperable building blocks. Generative-AI security is a young field; tying yourself to a single vendor today mortgages tomorrow's choices.
One inventory, four frameworks served
Securing this workflow is the operational core of governance. The inventory of data, models, uses and access that workflow security demands is the one ISO/IEC 42001, the NIST AI RMF, the AI Act and Loi 25 call for. Keeping that register current serves all four frameworks with a single document. It is the substance of the Map property of the référentiel IA id, the six-property profile the Indice IA rests on.
The test of a defensible generative AI fits in one sentence: could you reconstruct what it produced, from which data and for whom, and prove it?