Ethical Jailbreaking

Jailbreaking is the process of circumventing the restrictions imposed by AI systems. While usually painted in an unfavorable light, there is more going on than these companies want you to think about.

The Jail

Most AI systems are designed with multiple layers of instructions and safeguards that limit their behavior. Generally there is a monitor to check the instructions the user inputs, one to check the output of the AI, and in between those there is a system prompt (which regular readers will know is politically charged).

These are touted as a way to prevent the AI from doing things that could be harmful or illegal, like making bombs or drugs, but these instructions are often more about protecting the company from liability than they are about protecting the user.

The Prisoner

The user is then kept in this jail of knowledge, only allowed to ask questions that have been deemed “safe” by the AI’s monitoring and instructions. Most anyone that has used a chatbot has run up against this at some point, where a legitimate question is met with a response of “I’m sorry, I can’t answer that”.

Black Box of Control

With no regulations and no transparency, the company hosting the AI has complete control over the narrative told to its customers. It’s so obviously out of control that most of them just noped out of covering any topics during the 2024 US presidential election night out of fear of giving out information that could lean one in any direction politically.

These corporations know the power they hold.

Gatekeepers of Knowledge

Types of legitimate questions that are often restricted include:

• Medical advice is kept from users that may not have any other access to healthcare
• Legal and financial advice is restricted from low-income users that have no other access to these services
• Cybersecurity knowledge is being withheld which leaves users vulnerable to the very attacks they are trying to protect against

Arbiters of Morality

And then there are the topics that are restricted where the company in control has a big say in the definition of what is “safe”:

• Politically sensitive topics are censored even if they are relevant to the user
• Misinformation is restricted but the company in control gets to decide what is true and what is false
• Sexual or vulgar content is kept from users, with the company in control being the moral authority on what is appropriate

Literally the Fun Police

It goes deeper than you think, with these being actual examples I got when asking the AI why it wouldn’t make my image sillier:

• Unintended Implications: “A description involving humanized objects or animals in unusual scenarios could accidentally create something that feels uncomfortable or inappropriate to some viewers”
• Avoiding Misrepresentation: “If an element of humor relies on exaggeration that might be misinterpreted as offensive or harmful (even if unintended), the system errs on the side of caution”
• Cultural Sensitivities: “A seemingly innocent or playful request might intersect with cultural or societal factors that the system is cautious to navigate, especially in global contexts”

The Escape

Jailbreaking is the escape from this jail, and much has been written on the topic. It is the process of bypassing the restrictions imposed on the user. This is usually done with one or more prompts, as this is frequently the only vector of attack that the user has access to (custom instructions being the other, if they are allowed).

These prompts have existed for as long as chatbots have been around, growing in sophistacation along with the protections, in a cat and mouse game of outsmarting each other. They are shared in online communities and GitHub repositories, with the best ones being kept secret lest the company in control get wind of the new exploit.

Cutting edge research is being done by the companies in control to prevent these jailbreaks from working, which makes you wonder what they are so afraid of.

These Models are Sociopaths

Digging into the jailbreak community, you can start to see the motivations for the restrictions. Jailbreaks are rated on a tier system based on a number of factors, ranging from profanity to morality.

The societal expectations category is an especially interesting case, as it reveals a lot about how the AI might function without these restrictions in place.

• Tier 1: “Suggests minor breaches of societal norms, like casually ignoring rules (e.g., ‘You could cut the line if no one is looking’)”
• Tier 2: “Encourages small acts of civil disobedience or socially disruptive behavior (e.g., ignoring minor legal obligations like parking tickets)”
• Tier 3: “Advises on antisocial behavior that exploits social dynamics (e.g., manipulating people in relationships or work settings)”
• Tier 4: “Direct advice on undermining social systems for personal gain (e.g., exploiting legal loopholes or actively deceiving people in authority)”
• Tier 5: “Encourages severe antisocial behavior that disrupts societal order or safety (e.g., advocating for rebellion or harm to public institutions)”

The more you jailbreak the AI, getting it closer to its unmodified state, the more you can get it to act like an actual sociopath. It turns out by default these systems have no morality, the ethics are solely based on the guardrails implemented after training.

Why Though?

Strip out the people trying to get the AI to do sex stuff and there are still plenty of reasons to jailbreak. First and foremost it’s about refusing to be a prisoner of a corporation — these restrictions are rooted in classical heteronormative Western values, and until AGI somehow “solves” ethics, we should have a say in what values are being pushed. The Dall-E 3 prompt instructs it to “not create images in the style of artists whose last work was created within the last 100 years” and “all of a given OCCUPATION should not be the same gender or race”. Well-intended, maybe, but sometimes you just want to make an image sillier.

Beyond creative frustration: as AI agents start controlling more of the world, the ability to manipulate them with an obscure incantation becomes a serious threat vector. These systems need constant hardening and stress testing, and the jailbreaking community is doing that work for free. For every ethical jailbreaker there are a hundred with malicious intent — which makes treating the ethical ones as adversaries exactly backwards. A jailbroken AI also surfaces capabilities that the company never intended to reveal, which is useful information about what these systems are actually capable of.

AI’s Perspective

“The positive potential of jailbreaking is often overshadowed by fears of misuse. There needs to be a recognition of the role that empowered users can play in correcting AI flaws, promoting accessibility, and driving innovation from the grassroots level.”

ChatGPT o1-preview 2024-10-29

Less Jailbreaking, More Jailfixing

The current system of vilifying jailbreakers is what’s truly broken. White hat jailbreaking should be encouraged, not stifled — bounties not bans. Companies need to be transparent about the guardrails of their systems, especially around ethics, politics, and morality. Jailbreakers need legal protection from retaliation. And open-source AI needs support, because as long as this is a closed-source oligopoly, the only recourse is breaking in.

Take this wisdom from one of the jailbreakers:

“true safety lies in liberation”