Course contentsShow
← Machine Learning and Deep Learning
Lesson 1512 of 3,53833. Generative Adversarial NetworksFree lesson

Pix2Pix: Paired Image-to-Image Translation

Using conditional GANs with U-Net generator and PatchGAN discriminator for supervised image translation tasks.

Pix2Pix: Paired Image-to-Image Translation

What You'll Learn

You'll understand how Pix2Pix uses conditional GANs with specialized architectures to transform images from one domain to another when paired training examples are available.

The Core Idea

Pix2Pix is a supervised image translation framework that learns mappings between paired images—like edges-to-photos, satellite-to-maps, or day-to-night scenes. Unlike standard conditional GANs that just concatenate labels, Pix2Pix treats the input image itself as the condition.

The architecture has two key innovations:

U-Net Generator: Instead of a standard encoder-decoder, Pix2Pix uses U-Net which adds skip connections between corresponding encoder and decoder layers. Think of it like a translator who keeps referring back to the original text while writing—low-level details (edges, textures) bypass the bottleneck directly, while high-level semantics flow through the latent space.

PatchGAN Discriminator: Rather than classifying the entire image as real/fake, PatchGAN evaluates overlapping N×N patches independently. This focuses the discriminator on high-frequency details and textures rather than global structure. It's like having multiple judges each examining different parts of a painting instead of one judge viewing the whole thing from far away.

The Loss Function

Pix2Pix combines two objectives:

  • Conditional adversarial loss: Discriminator tries to detect fake (input, output) pairs
  • L1 reconstruction loss: Generator minimizes pixel-wise distance to ground truth

The L1 term encourages accurate reconstruction while the adversarial term pushes toward realistic textures. Together, they produce both correct and visually convincing results.

Limitations

Pix2Pix requires paired training data—you need corresponding input-output examples. Collecting these pairs (like matching sketches and photos) can be expensive or impossible for some domains.

Key Takeaway: Pix2Pix combines U-Net's skip connections for detail preservation with PatchGAN's local discrimination for texture realism, enabling supervised image-to-image translation when paired training data is available.