Guides

Crafting Photorealistic Portraits Using Stable Diffusion XL (SDXL): Advanced Prompt Techniques

Discover the art of creating AI-generated, photorealistic portraits with our comprehensive guide. Learn how to blend traditional photography principles with advanced prompt techniques for mastering the creation of lifelike portrait images using Stable Diffusion.

Shanmukha Karthik

Jan 25, 2024 • 10 min read

In the ever-evolving world of digital art, Generative image models such as SDXL 1.o has marked a revolutionary leap forward, especially in the realm of photorealistic portraits. For artists, hobbyists, and AI enthusiasts alike, mastering the art of creating lifelike images using SDXL is both an exciting challenge and an opportunity to push the boundaries of creativity. Building on our previous guide Generating Photo-realistic Images with Stable Diffusion XL, in this blog we'll explore strategies that bring an unprecedented level of realism and depth to your digital portraits.

Understanding key Elements of Portrait Photography

Before we construct any prompts, let's first try to understand the principles that drive portrait photography. We will use these principles in constructing prompts to generate photorealistic portrait images with Stable Diffusion.

In portrait photography, lighting plays a crucial role in setting the mood and accentuating features. Natural light, particularly when soft and diffused, offers a flattering and authentic feel, gently highlighting facial contours. Artificial light, like a softbox, provides even, shadow-reducing illumination suitable for professional looks, while direct flash creates dramatic, high-contrast effects for modern portraits.

The choice of background significantly influences the portrait's focus and atmosphere. Simple, neutral-colored backgrounds ensure the subject remains the central point without distractions, ideal for classic portraits. Textured backgrounds, like brick walls or curtains, add depth and context without taking attention away from the subject.

Camera settings are pivotal for achieving the desired aesthetic. A wide aperture, such as f/1.8, blurs the background, drawing focus to the subject's eyes, perfect for creating depth. A fast shutter speed like 1/200th of a second freezes motion, ensuring clarity. ISO settings vary; a low ISO ensures high quality in bright conditions, while a high ISO is beneficial in low light despite the risk of noise.

Focal length choice can dramatically alter a portrait's perspective. A 50mm lens offers a natural, eye-like view with some background blur, versatile for various portraits. In contrast, an 85mm lens provides more background blur and facial feature compression, ideal for detailed close-ups.

Composition techniques, such as the rule of thirds, create balanced and engaging images by strategically placing the subject's eyes. Filling the frame emphasizes facial expressions and minimizes distractions, excellent for expressive portraits.

Pose and expression are key to conveying the subject's personality and the portrait's mood. Natural poses capture relaxed, authentic emotions, while guided poses are useful for specific moods or aesthetics.

Finally, post-processing adjustments like exposure correction, color balancing, and retouching enhance the final image's quality, ensuring a polished and natural look. These adjustments correct brightness, color fidelity, and remove imperfections, tailoring the image to the desired outcome.

We've created a handy cheat sheet to guide you in crafting prompts for generating photorealistic portrait images with SDXL.

Element	Details	Example Input
Subject Description	Describe the subject (age, gender, notable features, clothing, etc.)	"Portrait of a young woman in a summer dress"
Lighting	- Natural Light: Specify direction/quality - Artificial Light: Specify type/effect	"Natural light from the right creating a soft glow"
Background	- Simple: Plain or neutral-colored - Textured: Detailed background like brick wall, curtain, etc.	"Blurred green garden"
Camera Settings	- Aperture: Wide/narrow - Shutter Speed: Fast/slow - ISO: Low/high	"Wide aperture for background blur"
Focal Length	Specify preferred focal length	"50mm lens perspective"
Composition	- Rule of Thirds - Filling the Frame	"Subject’s eyes aligned with the rule of thirds"
Pose and Expression	Describe the subject's pose and expression	"Sitting on a garden bench, smiling naturally"
Post-Processing	- Exposure Adjustment - Color Correction - Retouching	"Slight exposure adjustment for brightness"
Mood/Atmosphere	Describe the overall mood or atmosphere	"Warm, cheerful atmosphere"

Lets put together a Portrait Prompt Example:

"Portrait of a young woman in a summer dress, natural light from the right creating a soft glow. Background: blurred green garden, wide aperture. 50mm lens perspective, subject’s eyes aligned with the rule of thirds. Pose: sitting on a garden bench, smiling naturally. Warm, cheerful atmosphere, slight exposure adjustment for brightness"

We will be using Realvis XL, an SDXL checkpoint that excels in generating images which are very close to realism. In fact, with images generated by Realvis XL, it's hard to tell if they are AI-generated or real.

Classic Studio Portraits

These portraits are taken in a controlled studio setting with professional lighting and backdrops, focusing on capturing timeless and elegant images of the subject.

Prompt: Elegant studio portrait of a woman in a vintage dress, classic three-point lighting setup for a timeless look. Neutral, seamless backdrop, 85mm lens for flattering features, formal seated pose with a gentle smile.

Prompt: Professional studio headshot of a businessman, key light and fill light creating a balanced exposure. Clean, grey background, 50mm lens for a natural look, confident stance with arms crossed, looking directly at the camera.

Environmental Portraits

These portraits capture individuals in their natural surroundings or work environments, highlighting their relationship with the setting and often telling a story about their life or occupation.

Prompt: Portrait of a chef in a bustling kitchen, natural ambient light mixed with kitchen fluorescents. Kitchen environment as background, 50mm lens capturing the chef in action. Candid expression while cooking, showcasing personality and profession.

Prompt: Farmer portrait in a field at sunset, warm natural backlighting highlighting the fields. Wide-angle lens to include expansive farm background, farmer leaning on a tractor, proud and relaxed demeanor reflecting a connection to the land.

Lifestyle Portraits

Lifestyle portraits depict people in real-life events or milestones in an artistic manner, capturing candid moments and everyday situations.

Prompt: Casual portrait of a young couple in a coffee shop, natural window light creating a cozy atmosphere. Café interior as a vibrant background, 50mm lens for a candid feel. Subjects laughing and sharing coffee, capturing a moment of everyday joy.

Prompt: Portrait of a group of friends on a city rooftop, dynamic urban skyline at dusk. Mixed lighting from city lights and twilight, wide aperture for focus on group. Relaxed, spontaneous poses, capturing the essence of urban life.

Conceptual Portraits

These are creative and artistic portraits that often involve complex ideas, themes, and elaborate setups, aiming to convey a specific concept or narrative.

Prompt: Surreal portrait of a person with painted face, abstract patterns creating a unique look. Studio lighting with colored gels for a dramatic effect, plain dark background. 85mm lens, intense gaze into the camera, conveying deep emotions

Prompt: Artistic portrayal of an individual emerging from a cocoon of colorful fabric, symbolizing transformation and rebirth. Vibrant colors against a dark background, creating a striking contrast. Creative lighting to enhance the dramatic effect of the metamorphosis.

Street Portraits

Taken in outdoor urban settings, these portraits capture the essence of street life, often featuring candid and spontaneous shots of strangers or everyday city life.

Prompt: Candid street portrait of an elderly man sitting on a city bench, natural light with buildings in the background. 50mm lens capturing the hustle of city life around him. Unposed, reflective expression, immersed in the urban setting

Prompt: Street portrait of a vendor at a bustling food market, surrounded by colorful produce and busy shoppers. Overhead natural light mixed with the glow of market lamps, captured with a 50mm lens for a close yet contextual shot. Warm, inviting smile, embodying the lively market atmosphere

Fine Art Portraits

These portraits are artistic and highly stylized, often focusing on aesthetic quality and creative expression, with the subject frequently serving as a component of a larger artistic vision.

Prompt: Artistic portrait of a ballet dancer, dramatic side lighting emphasizing form and posture. Black background for high contrast, 85mm lens for elegant body lines. Poised, mid-dance movement, creating a sense of fluidity and grace

Prompt: Monochromatic portrait of a model with dramatic makeup, chiaroscuro lighting for a classic art feel. Minimalist background, 50mm lens for focused facial expression. Intense, contemplative look, blending modern fashion with classic art aesthetics.

Documentary Portraits

These are candid portraits that capture real-life events or situations with an aim to tell a story or document social and cultural issues.

Prompt: Portrait of a young activist at a protest, holding a sign, amidst a crowd of demonstrators. Dynamic, raw emotion on the face, shot with a 50mm lens to focus on the individual yet capturing the collective spirit. A moment of passion and commitment to a cause.

Prompt: Portrait of a veteran in a room filled with military memorabilia, medals, and photographs. Natural afternoon light streaming in, 50mm lens capturing the mixture of pride and nostalgia. Thoughtful, distant gaze, telling a story of service and sacrifice.

Prompt: Portrait of a factory worker in an industrial setting, machinery and the bustle of production in the background. Harsh, artificial lighting highlighting the sweat and determination. 35mm lens capturing the essence of hard labor and the industrial environment.

Children's Portraits

These portraits capture the innocence, charm, and spontaneity of children, often focusing on natural expressions and moments of childhood.

Prompt: Playful portrait of a child in a park, natural sunlight with trees and playground in the background. Candid expression while playing, 35mm lens to capture the child in a natural environment. Bright, colorful clothing, a moment of pure joy.

Prompt: Portrait of a newborn baby sleeping, soft diffused light in a nursery setting. Close-up with a 50mm lens, capturing delicate features. Soft, pastel-colored bedding as background, peaceful and tender mood.

Travel Portraits

Capturing people in various travel destinations, these portraits reflect the culture, ambiance, and essence of the place, often highlighting the experiences and emotions associated with travel.

Prompt: *Portrait of a local Japanese* artisan in a market, vibrant stalls and goods in the background. Natural light filtering through the market, 50mm lens for environmental storytelling. Authentic expression, showcasing local culture and lifestyle

Prompt: Sunset portrait of a hiker on a mountain trail, panoramic view of the valley below. Warm, golden hour light illuminating the landscape, shot with a 35mm lens for a balance of subject and majestic scenery. Reflective and triumphant expression, capturing the spirit of exploration.

Sports Portraits

These portray athletes or sporting events, focusing on capturing the intensity, motion, and spirit of sports and physical activities.

Prompt: Dynamic portrait of a basketball player on the court, action shot with stadium lighting. Focused expression, mid-jump with the ball, 85mm lens for sharp subject against a blurred audience. High shutter speed to capture the intensity of the game.

Prompt: Portrait of a swimmer poolside, reflective water and pool lights in the background. Natural daylight mixed with pool lighting, 50mm lens capturing the athlete's determination. Wet hair, goggles on, embodying the spirit of competition

Pet Portraits

These portraits capture the personality and charm of pets, often highlighting the unique bond between the pet and its owner or showcasing the animal in its natural behavior.

Prompt: Portrait of a dog in a home setting, lying comfortably on a rug. Natural light from a nearby window, 85mm lens for a detailed close-up. Warm, cozy atmosphere, capturing the pet's relaxed and content demeanor

Prompt: Action shot of a cat playing in a garden, mid-leap caught with high shutter speed. Natural outdoor lighting, 50mm lens for a dynamic, playful scene. Vivid greenery in the background, showcasing the pet's agility and energy

Couple Portraits

These portraits can range from candid and spontaneous shots to more posed and stylized images, highlighting the couple's connection in diverse settings and contexts.

Prompt: Casual couple portrait in an urban setting, graffiti wall as a vibrant backdrop. Natural, relaxed poses, 35mm lens to include some urban context. Laughing together, candid and spontaneous, reflecting a modern relationship

Prompt: Rainy day street portrait of a couple under an umbrella, cityscape blurred in the rain. Moody, atmospheric lighting, 50mm lens capturing their laughter as they navigate puddles. A playful, joyful moment, undampened by the weather.

Conclusion

Wrapping up, we've taken a deep dive into the classic rules of portrait photography and seen how they translate into the world of AI with Stable Diffusion. It's been an eye-opening experience to see just how lifelike and realistic AI-generated images can be when we apply these time-honored principles. This journey has been a perfect blend of old-school technique and cutting-edge tech, showing us the incredible potential lying at the intersection of traditional photography and artificial intelligence.