Image-to-Image and ControlNet
What you'll learn: How to transform existing images into new variations and gain precise structural control over AI-generated outputs using img2img, inpainting, and ControlNet.
Beyond Text-to-Image: Working with Source Images
While text-to-image generates pictures from scratch, image-to-image (img2img) uses an existing image as a starting point. Think of it like giving an artist a rough sketch instead of just verbal instructions—you're providing visual guidance that constrains the creative process.
Three Key Techniques
Img2img transforms an existing image based on your prompt while preserving some of the original's composition. You control the "strength" parameter: low values keep the original mostly intact, high values create something more dramatically different. It's perfect for style transfers, variations, or artistic reinterpretation.
Inpainting lets you selectively edit portions of an image by masking areas you want to regenerate. The model fills in the masked region while maintaining coherence with the surrounding context—like Photoshop's content-aware fill, but guided by your text prompt.
ControlNet takes this further by extracting structural information from a source image (edges, depth maps, poses, or line art) and using it as a "skeleton" for generation. The AI follows this structure precisely while creating entirely new content. It's analogous to building a new house on an existing foundation—the layout is fixed, but everything else is fresh.
When to Use Each
Use img2img for iterative refinement and style variations. Use inpainting for targeted edits and fixing specific regions. Use ControlNet when you need precise compositional control—matching a pose, preserving architectural lines, or following a specific spatial layout.
Key Takeaway: Image-to-image techniques let you guide generation with visual input rather than text alone, giving you precise control over composition, structure, and selective editing while maintaining coherence.