Beginner's Guide to Understanding Negative Prompts in Stable Diffusion

Negative prompts in Stable Diffusion serve as a guiding mechanism for image generation. They enable users to define what they wish to exclude from their generated images, which can be particularly useful in avoiding the creation of images that are blurry, distorted, or contain unwanted noise.

Stable Diffusion and other AI image generators often use an image as a prompt to generate a similar image. However, text prompts are used to describe the desired output, while negative prompts are used to describe undesired elements. For instance, if you want a house without a lawn, you can subtract a lawn from a house to get a winter scene.

The process begins by extracting features from both positive and negative image prompts. The positive features condition the conditioned latent, while the negative features condition the “unconditioned” latent. This causes the latent image to move towards the image prompt and away from the negative image prompt.

With the introduction of Stable Diffusion v2, negative prompts have become an essential tool. They offer an extra layer of control over text-to-image generation. For example, if you’ve generated a painting of Paris on a rainy day and want to generate another one but without any people, you can use the same seed value and add the negative prompt “people”.

Role of Negative Prompts in Enhancing Stable Diffusion Models

Refining AI Output: Negative prompts in Stable Diffusion are a unique approach to guiding artificial intelligence systems1. They specify what the user does not want to see, without any extra input1. This powerful method helps to refine the output of AI models by excluding unwanted elements, ultimately leading to more targeted and desirable results.

Improving Model Performance: The concept of stable diffusion negative prompts emerged as AI researchers sought to improve the performance of their models1. By incorporating negative prompts, they found that they could better control the output of the AI system, resulting in a more accurate and tailored response.

Enhancing Capabilities: As AI continues to evolve, stable diffusion negative prompts have become a key component in enhancing the capabilities of these advanced systems. Traditional prompts provide AI models with a specific query or task to complete, while stable diffusion negative prompts offer a more nuanced approach. By combining both traditional prompts and stable diffusion negative prompts, you can achieve a more precise and tailored output from AI models.

Advanced Techniques & Best Practices for Crafting Effective Negative Prompts

1. Emphasizing Negative Prompts: If you’re generating an image of a “forest” but you really don’t want any “bears” in the image, you could use “bears:1.3” as your negative prompt to make it 30% more important that there are no bears in the image.
2. Modifying Images: If you’ve generated an image of a “red apple” but you want it to be less red, you could use “red:0.5” as your negative prompt to nudge the model towards making the apple less red.
3. Keep Negatives Concise: If you’re generating an image of a “city skyline at night” and you don’t want any “cars” or “street lights” in the image, it’s better to specify those directly as your negative prompts rather than using a vague term like “no city elements”.
4. Using Negative Prompts like ‘no duplicates’ or ‘no blurry pictures’: If you’re generating an image of a “bouquet of flowers” and you want each flower to be unique and the image to be sharp, you could use “duplicates” and “blurry pictures” as your negative prompts.

Practical Examples:

Let's take the simple example where we want to generate the image of animals present in the forest, but there needs to be no bears as mentioned in the best practices above. We will present to you the images generated with and without the specific negative prompts.

Prompt: A forest filled with animals

Negative prompt : scary, cartoon,bears:1.3

The negative prompt specifies the need to make sure that there were no "bears" present

By comparing both generated images, we can observe distinct differences. In the image on the left, animals are present as intended, but it deviates from our preference by including bears. On the right, various animals are depicted, aligning with our overall intention. Importantly, the specification "bears:1.3" ensures a 30% emphasis on the absence of bears, successfully excluding them from the image.

Let us take another example where we want to have a 3d rendered image of an apple. Our goal is to make sure that the color of the apple isn't too bright and red in color.

Prompt: An apple, front view, studio photo, hyper-realistic, fashion, photorealistic, 3d render

Negative prompt: scary, cartoon, painting, poor quality,  red:0.5

We can see the difference and the impact of the negative prompt in the images generated, without including "red:0.5," the resulting image is prominently red. However, by introducing this specific term during image generation there is a distinct contrast in the generated image.

Assume we want to generate an image of a skyscraper that consists of vibrant street art, however, the images generated should not consist of any streetlights nor there should be the presence of cars. The intention is to have a vibrant image only with the presence of skyscrapers and glass windows.

Prompt : Skyscrapers with vibrant street art, urban aesthetic , comic book style

Negative Prompt: scary, cartoon, painting, poor quality , cars

On taking a close look at the generated images we can see how there is no presence of cars when we get to add "cars" in the negative prompt.

In this specific example, our goal is to generate an image featuring an American woman with exceptionally long blonde hair, yet with the unique requirement that she is facing away, striking a posed stance. While such specifications can be directly included in the prompt, the effectiveness of negative prompts becomes apparent in fine-tuning our desired output. As we previously know, utilizing negative prompts allows us to explicitly exclude elements not desired in the image.

Prompt : Hasselblad photo portrait of a beautiful and mysterious American woman model with long extra blond hair, wearing short clothes. The setting is a clean house with colorful paintings.

Negative prompt : scary, cartoon, painting, poor quality , big wide smile , facing towards camera

The presence of the keywords "big wide smile, facing towards camera" in the negative prompts helps in generating the image according to our intention.

Conclusion:

We hope that this guide helped in improving your understanding of negative prompts and its effectiveness in generating high quality images.