Story Diffusion: Create Comic Strips Effortlessly with AI

Shanmukha Karthik -

Story diffusion transforms your written narratives into stunning visual experiences. Imagine crafting a comic strip simply by describing your story with text prompts. Story Diffusion achieves consistency across images in a series through a method called Consistent Self-Attention. This method is inserted into the diffusion backbone and works by calculating self-attention across a batch of images rather than a single image. This allows the model to build connections between the images and ensure that the characters and their attributes are consistent throughout the series. For example, if a character is wearing a black suit in one image, they will also be wearing a red shirt in subsequent images.

Try Story Diffusion for Free*

* Get $0.50 daily free credits on Segmind.

Comic Strip generated with Story Diffusion

How to use Story Diffusion?

Using story diffusion is fairly easy. We have broken things down for you, so that you can use start creating amazing comic strips with story diffusion. You can generate comic strips either with just text prompts or a with a combination of text prompt and a reference image.

With out Reference Image

  1. Character Description: Give a general description of the character. eg - a man, wearing black suit.
  2. Comic Description: This is where you write your story board. Each frame is divided by a new line. You can control the captions appearing on these frames. By default, the prompt is used as the caption for each image. If you need to change the caption, add a '#' at the end of each line. Only the part after the '#' will be added as a caption to the image. The [NC] symbol is used as a flag to indicate that no characters should be present in the generated scene images. If you want do that, prepend the '[NC]' at the beginning of the line.
💡
Here is an example for comic description:at home, read new paper #at home, The newspaper says there is a treasure house in the forest.on the road, near the forest[NC] The car on the road, near the forest #He drives to the forest in search of treasure.[NC]A tiger appeared in the forest, at nightvery frightened, open mouth, in the forest, at nightrunning very fast, in the forest, at night[NC] A house in the forest, at night #Suddenly, he discovers the treasure house!in the house filled with treasure, laughing, at night #He is overjoyed inside the house.
Comic Strip generated with a text prompt

3. Choose comic style: You can control the number of frames in the comic strip. The four pannel give you four frames, where as classic comic style gives outputs of 8 frames.

4. Choose Style Name: You choose the style in which you want to generate your comic strips. For examples, anime style or line art style and so on.

With Reference Image

You can use a reference image for your character. There are a few things you should consider when using a reference image, other than that everything remains same as generating comic strip with just text prompt.

In the general description of the character. If reference image above is provided, making sure to follow the class word you want to customize with the trigger word 'img', such as: 'man img' or 'woman img' or 'girl img'. Along with this make sure to remove [NC] when using the reference image.

In the advanced settings, you can adjust the Style Strength ratio based on your reference image. The higher the value, stronger the influence.

💡
Lets look at an example:Character Description: a policeman img, wearing a white shirtComic Description:Directing traffic on the road.walking on the streets.Chasing a man on the street.At the police station.
Comic Strip generated with a text prompt & a reference image

Four Pannel vs Classic Comic Style

Based on your story board, you can decide the frames by choosing either Four Pannel- which give 4 frames or classic comic style which gives 8 frames.

Four Pannel (R) vs Classic Comic Style (R)

Style Names

You can generate comic strips in a variety of styles based on the theme of your comics.

  • Disney
  • Anime
  • Comic book
  • Line art
  • Photographic
  • Cinematic
  • Photographic
Japanese Anime Style
Line Art Style
Comic Book Style
Try Story Diffusion for Free on Segmind