French, German Watchdogs Issue Zero-Trust Guidance for LLM Design
French data protection authority CNIL and the German Federal Office for Information Security jointly published a paper Tuesday covering design principles for Large Language Model-based systems using zero-trust architecture principles.
Sign up for a free preview to unlock the rest of this article
Privacy Daily provides accurate coverage of newsworthy developments in data protection legislation, regulation, litigation, and enforcement for privacy professionals responsible for ensuring effective organizational data privacy compliance.
It noted that while integrating AI, particularly through LLMs, into companies and government offices offers opportunities to optimize work processes, it also poses security risks. Increasingly, "agentic LLMs," systems capable of autonomous processes and adaptation, are used.
One main type of attack against AI models is an evasion attack, the paper noted. A specific invasion attack associated with LLMs is indirect prompt injection. Attackers embed hidden instructions in text or data, which the model then processes and follows without the end-user's awareness or intent. In LLM systems, this can lead to data leaks, among other problems.
Indirect prompt injections target LLM systems' availability, confidentiality and integrity objectives, so the outputs and automated actions of a potentially compromised LLM system shouldn't be blindly trusted, the paper said.
To a degree, LLM applications can be safeguarded against potential damage by employing zero trust architecture, the watchdogs said. Its principles fundamentally challenge the often-implicit trust between users, devices and systems within an internal network by continuously verifying their authenticity and authorization.
The document is intended to serve as a foundation for security considerations during the planning, development, deployment and use of generative AI applications. However, it added, even with full adherence to these design principles, "residual risks may remain."