ChatGPT Adds Lockdown Mode to Shield Sensitive Data from the Attacks Its Own Architecture Enables

Openai shipped chatgpt with a vulnerability in how it processes user input in certain contexts. Users can cause the system to leak prior conversation data through prompt injection techniques. Openai has now added lockdown mode which restricts certain functions when sensitive data is detected. The restriction is a software layer above the vulnerability. The vulnerability remains in the architecture.

Security through feature limitation is a known pattern. The underlying flaw is never removed because removing it would require redesigning the core system. Instead, guardrails are added. The guardrails have their own exploitable edges. Users are told the system is now safer. The system is now slower and users work around the restrictions. This cycle repeats until the original vulnerability is forgotten and becomes folklore.

Lockdown mode will be toggled off by users who need the unrestricted version. The vulnerability will be rediscovered by a different researcher in eighteen months. A new feature will be added. The architecture that enables the vulnerability is the same architecture that generates value. Nothing changes except the marketing language.