Course contentsShow
← AI Engineering
Lesson 1737 of 1,88642. Multimodal SystemsFree lesson

Image-to-Image and ControlNet

Use existing images as guidance with img2img, inpainting, and ControlNet for precise control.

Image-to-Image and ControlNet

What you'll learn: How to transform existing images into new variations and gain precise structural control over AI-generated outputs using img2img, inpainting, and ControlNet.

Beyond Text-to-Image: Working with Source Images

While text-to-image generates pictures from scratch, image-to-image (img2img) uses an existing image as a starting point. Think of it like giving an artist a rough sketch instead of just verbal instructions—you're providing visual guidance that constrains the creative process.

Three Key Techniques

Img2img transforms an existing image based on your prompt while preserving some of the original's composition. You control the "strength" parameter: low values keep the original mostly intact, high values create something more dramatically different. It's perfect for style transfers, variations, or artistic reinterpretation.

Inpainting lets you selectively edit portions of an image by masking areas you want to regenerate. The model fills in the masked region while maintaining coherence with the surrounding context—like Photoshop's content-aware fill, but guided by your text prompt.

ControlNet takes this further by extracting structural information from a source image (edges, depth maps, poses, or line art) and using it as a "skeleton" for generation. The AI follows this structure precisely while creating entirely new content. It's analogous to building a new house on an existing foundation—the layout is fixed, but everything else is fresh.

When to Use Each

Use img2img for iterative refinement and style variations. Use inpainting for targeted edits and fixing specific regions. Use ControlNet when you need precise compositional control—matching a pose, preserving architectural lines, or following a specific spatial layout.

Key Takeaway: Image-to-image techniques let you guide generation with visual input rather than text alone, giving you precise control over composition, structure, and selective editing while maintaining coherence.