Use Cases of ControlNet SoftEdge

In this blog post, we will take a look at the use cases of ControlNet SoftEdge. This particular variant of ControlNet places a unique emphasis on extracting edges from images, crafting a sketch-like representation that excels in transforming diverse patterns into intricate line drawings. The selective extraction process ensures the preservation of fine details, setting ControlNet SoftEdge apart.

In the previous post on ControlNet SoftEdge, we got to see about the parameters present in the model and how to use the ControlNet SoftEdge API  to generate Images, we also got insight on the best settings of the model to maximize its performance. We will now explore the real-world scenarios where the ControlNet SoftEdge emerges as a powerful tool.

Converting Images to Illustrations:

The primary work of ControlNet SoftEdge is to focus on extracting edges from images and creating a sketch-like representation. It excels in extracting line drawings from diverse patterns. We will use this aspect of the model to our advantage.

The reference image will be displayed on the left while the converted image will be displayed on the right. We will take a look at various images with varying rates of complexity to understand the capability of the model.

Let's explore a practical example. Imagine converting an image of a woman in a pilot uniform into an illustration in the style of manga.

Prompt : A beautiful girl in manga style

Imagine if we wanted to transform the image of a model standing in front of a building into an illustration of a woman surrounded by flowers in an anime style

Prompt : anime style illustration of woman standing in a beautiful field of flowers, colorful flowers everywhere, perfect lighting.

Consider having an image of a young child attempting to use a computer, and we would like to transform it into an image depicting a little kid with a Pixar animation style

Prompt : pixar style illustration of a little kid sitting in front of computer

Transform line drawings:

In the case of line drawings, ControlNet SoftEdge first analyzes your line drawings and extracts the edges and the contours present in the image.  It's like tracing the characters, objects, and the background present in the images.

Once the above process is over it tends to add texture and breathe life into the image based on the prompt given and then finally it tends to polish the edges and refines the details.

Example:  Let’s take a line drawing of a football player as the image. The goal is to transform this specific line drawing into an illustration portraying the player actively engaged on the ground. With the provided image and prompt, the model initiates by extracting the existing edges in the input image. It then translates the extracted edges, aligning the illustration with the specified prompt which results in the final image.

Prompt: illustration of a soccer player hitting the ball on the ground

Example: imagine starting with a line drawing featuring a blonde woman adorned with freckles and dressed in intricate embroidery. Our goal is to morph this image into an illustration characterized by a combination of both saturated and muted colors.

Prompt : Half-body illustration of a woman, with dark blonde hair, wearing a long boho sweater, comfortable , casual, laid-back, frontal facing, beautiful light sky, low saturation, muted colors, animation

Or  Let's take an example with the illustration of a line drawing portraying a man donned in modern, uber-cool attire. We aim to impart a sketch-like appearance, precisely infusing colors into the line drawing.

Prompt : A  beautiful man anxious , challenging societal expectations hiding in the corner of a dance club sketch

Modifying Texture, Color, and Tone

Here we will see how the ControlNet SoftEdge model can be used for not only converting images into illustrations or sketches but also for modifying the tone and the color palette of the image.

This is made possible due to the ability of the model to extract edges with precision. Once the initial process of edge extraction of not only the character present in the image but also the background and the objects, it then proceeds to colorize the resultant image. Hence we have a model that provides precision and control over the finer details of the existing image.

Example:  Let us take an image of a gorgeous Chinese-style high-rise villa standing in a valley and transform the image into an image where it has a more ancient-like structure and is also surrounded by greenery and a sunny blue sky.

Prompt : ancient structure of a hotel , surrounded by various type of greens, aerial shot, wide shot, with sunny blue sky

Now let us take an image of a fantastical scene: a white cat adorned with intricate line art, set against the backdrop of an exploding nebula. The transformation that we now seek is to shift from its current bright and vibrant state to a more mysterious and textured ambiance in the style of 90’s anime.

Prompt : 90's vintage anime of a cat surrounded by darkness and shooting stars

Imagine an illustration capturing the essence of Washington Square Park, resembling a sunny and vibrant poster. Now envision a complete 360 turn as we turn this lively scene into a dystopian and ominous atmosphere. The once-sunny hues give way to a palette dominated by shades of red and black, ushering in a mood that evokes a sense of darkness and uncertainty.

Prompt : Night city street, downtown, with street lamps, buildings and palms lined up. Dramatic lightning, in red and black. high quality, centered perspective. Utopian city feel

Conclusion:

This model stands out as the go-to choice, not just enhancing images but infusing them with a touch of artistry. It goes beyond the usual, holding onto the unique features of each image while gently refining brush strokes. The result? More than just captivating visuals

Throughout this blog, we got to look into 3 wonderful use cases of the model and how it elevated the quality of the existing image depending upon our choices.