The question we hear from our clients is no longer “should we allow AI?”, it is “what has already been pasted into it?”. The answer to give your teams fits in one sentence: anything that identifies a customer does not leave your perimeter, and nothing the assistant produces goes out without human review. The choice of tool comes afterwards.
The real failure mode
The model is not the problem; the path the data takes is. An excerpt from a support ticket carrying the customer's name and address, a contract dropped in to “get a summary of it”, a whole production log pasted in to make sense of an error: each time, the information leaves your systems and lands with a third party, often under a personal account, on terms of use nobody has read.
It is almost never malicious. It is someone trying to finish their work faster with the tool they have to hand. That is precisely why a blanket ban does not hold: it pushes the usage onto the personal phone, where you can no longer see any of it.
- Write down what never leaves. A short, readable list: customer identities, contracts, payment and health data, technical secrets, production extracts. One page, not a twenty-page charter.
- Company accounts, not personal ones. A business subscription brings a contractual framework, access administration and an audit trail. A personal account brings none of the three.
- Anonymise at source. What is needed is almost always the structure of the problem, not the identity: replace names, contract numbers and addresses before pasting. A demonstration dataset settles a good share of the cases.
- Name who signs off. Nothing goes out to a customer and nothing is written into a business system until a named person has reviewed it. The assistant prepares, the human decides.
What it changes on the technical side
Once those rules are set, the architecture questions become answerable again: where the service runs, which data actually transits, who holds the keys, what stays in the logs and for how long. We run production for others, in France and in Switzerland, and the discipline is the same as for any sensitive processing: a defined perimeter, named access, segregation, logging, an exit path. An assistant does not invent a new category of risk; it adds one more data egress to map, and that one is triggered by a copy-paste rather than by a flow you designed. The mistakes we still see in the cloud show up here unchanged: shared accounts, no segregation, logs nobody reads.
How to check it this week
- Ask who in the company is paying for an AI subscription on a personal card. The answer is rarely the expected one.
- Look at whether production logs can be read from an office workstation. If they can, they can be pasted.
- Take three real use cases and redo them on anonymised data. If the result holds up, your rule is workable; if it does not, the rule will be worked around, and it is the rule that needs revisiting.
The framework of use itself, and the habits that go with it, are covered in the questions to settle before deploying and in what separates two teams given the same tool. On the operations side, see our cloud managed services and our consulting and audit.