How to use Stable Diffusion to Create Comic Strips

Comic strips have been a core part of human entertainment since the earliest edition of the newspapers. In today's digital world, they are now in most if not all parts of digital media, e-newspapers, Twitter(X), newsletters, emailers. For decades artists have spent an enormous amount of time trying to sketch art for comic strips either physically or digitally. Art creation was essentially the most cumbersome and time consuming aspect of this avenue.

But today, with the evolution of generative AI, art for comic strips can be created in a matter of minutes for any imaginable script. Not just the art, GenAI can be leveraged to even come up with a script for a general idea. In this article, I am going to show you an entire workflow for GenAI based comic strip creation.

But first lets understand what are the important factors that need to be controlled in order to get to a successful output.

What are the important factors to control?

1. Script- To arrive at a funny and witty comic strip,  it is important that we first write/generate a script. Traditionally this would involve a lot of thought and creativity but thanks to the age of LLMs we can generate multiple of these instantaneously and proceed to pick the best out of the lot.
2. Image quality- Quality of images generated needs to not only be high resolution but also consistent. Text to Image models such as SDXL and Mid-journey allow you to generate images in line with your script instantly eliminating the need for an artist to spend hours on sketching.
3. Image Consistency- As we move through the different panels of the comic strip we need to maintain consistency in character and scene. The elements of the scene and the look of the characters cannot change as we go from panel to  panel else, the reader would be lost in translation. This can be controlled by using the same 'style' while generating images for eg: "comic" or "anime". The characters can be consistent by using a characters face from one image and replacing it across all images using a simple crop tool

What tools do we need?

1. ChatGPT - Firstly we will need to use ChatGPT for generating a script for our comic strip.
2. Segmind SDXL 1.0- Next we use the Segmind's SDXL 1.0 model to generate our images.
3. Segmind Background Removal-We will need to remove the background of some images to maintain character consistency. I will explain this in detail in the next section of this article.
4. Clipdrop CleanUp tool- We will use this tool to erase/remove any unwanted objects or bad spots from our images.
5. Canva- Yes that's right, for the final step, which is assembling our comic strip in accordance with our script and adding text bubbles with text, we will use Canva.

How do I use these tools to create a comic strip?

Now I'll take you through the detailed process of how you can create your first AI generated comic strip only by using the above mentioned tools-

1. Generate a Script- Log in/Create an account on ChatGPT by OpenAI.  You will be able to get a good script with the free version. Just plug in a simple prompt with a generic theme for your script such as "Comic Strip on Albert Einstein" and the model will throw out a panel-by-panel script. You can go a step further by adding to your prompt in order to get a script that you desire with words such as "funny yet quirky" or "dark".
2. Generate Images- To generate images for our comic strip you have to go to segmind.com, log in/create your accountSegmind currently offers 100 free credits per day which should be sufficient for us to create a comic strip. Select the SDXL 1.0 model and choose "comic" style from the drop down in the settings section. Enter a prompt that describes your panels and randomise seed till you achieve something that looks amazing. Try to keep the same seed and generate images for all your panels.
3. Achieve consistency- Once you have all your images, you may notice that your characters are not consistent across all images. To fix this you will have to pick once of the images that your character/s looks the most ideal to be reused. Crop out the characters head and save as a separate image. Now, go to Segmind's Background Removal model and remove any background from your cropped out image so only your charters head/face remains. You can now use this image of your character head and replace it over other existing images where your character is inconsistent.
4. Fix your Images- There could be instances where some of your images have certain unwanted elements or random noise created by the model. To fix this you will need to use the CleanUp tool by Clipdrop. Just brush over the parts of the image that need fixing and voila! your images are almost perfect.
5. Assemble your Comic Strip- In the final step we will use Canva to assemble all our images in order of the script generated earlier. Add the cropped out, background removed character heads wherever necessary. Add text bubbles and add text from the script within the text bubbles.

Thats it! Save the file in your preferred format and you would have successfully generated your first comic strip in a matter of less than 20 minutes if not faster.

What are the Possibilities?

Honestly, the possibilities are endless and are only limited to as far human creativity can take us. From creating a simple witty comic strip to designing and publishing an entire comic book, it can all be done in 1/1000th of the time that it traditionally takes us. Some parallel use cases/applications that can be explored are-

1. Anime clips- For this, you will use the "Anime" style on SDXL instead of  "comic". The images generated can be stitched together into a short video. With an exciting background score, you can acheive wonders.
2. Poster Design- One or many of the comic panels generated can be printed onto posters/banners for internal decor.
3. T-shirt Design- The Images generated can also be printed on T-shirt or apparel/mugs/anything printable. It could serve as the perfect gift which is essentially only one of a kind.
4. Online Blog/Newsletter- The comic Strips can be used in Blogs/Newsletters to catch the readers attention and impart an important message.

Try SDXL 1.0 for free to generate comics

These are only a few use cases and applications for this workflow of the many. It's important to note that the workflow described here is not a static, finalised version but rather a dynamic part of our ongoing exploration and discovery of impactful use-cases. If you're inspired by the potential of this use case in reshaping the fashion and design sector, Segmind offers fast APIs for a range of Generative models, all of which can be integrated into a customised pipeline tailored to your specific needs.