GAN Fundamentals  |  April 13, 2023

Inside GANs: Understanding the Duel of Generator and Discriminator

Generative Adversarial Networks (GANs) are a type of artificial neural network developed by Ian Goodfellow and other researchers in 2014. At its most fundamental level, GANs consist of two distinct neural networks referred to as the Generator and the Discriminator. These two models work together to progressively improve each other’s performance and generate new, realistic data from a given input.

The Generator is the model responsible for generating new data. This model typically begins with some sort of noise, such as a random vector of numbers or a vector of noise. The noise is then input into the network and the model creates a new image based on the input. The Generator is effectively creating data, such as images, from a random noise.

The Discriminator is the model responsible for differentiating between the real images and the generated ones. It looks at the generated images and attempts to decide whether they are real or created by the Generator. This is done by inputting both real and generated images into the network and analyzing the output. The outputs are then used to train the model, updating the weights to enable better differentiation between real and generated images.

The two models take turns training and competing with each other in a min-max game, referred to as an adversarial relationship. The Generator builds its skills by learning from the Discriminator, while the Discriminator continues to develop its skills by looking at the results generated by the Generator. The Generator strives to create realistic images that the Discriminator cannot tell apart from the real images, while the Discriminator works to correctly identify the real and generated images. This min-max game is referred to as a Nash equilibrium and continues until the two neural networks reach an equilibrium point.

An important aspect of GANs is their resilience to overfitting. Overfitting occurs when a model attempts to learn the underlying data too well, resulting in situations where the model does not generalize well to unseen data. GANs are able to avoid this issue due to the adversarial relationship between the Generator and the Discriminator. While the Discriminator is evaluating the generated images, it forces the Generator to create a wider range of images, thus preventing the model from overfitting and resulting in better generalization performance.

In summary, GANs employ a two-model architecture consisting of a Generator and a Discriminator. The Generator takes a random noise as input and creates new data, such as images, from it. The Discriminator then evaluates those generated images and attempts to differentiate the real from the fake. This min-max game enables the two models to continually improve each other’s performance, enabling GANs to create increasingly realistic data far beyond what was previously possible. Additionally, the adversarial relationship between the two models helps to avoid overfitting, resulting in better generalization performance.