DAY 82-100 DAYS MLCODE: GAN

In the pervious blogs, we discussed Object detection and segmentation, in this blog we’ll start GAN (Generative Adversarial Networks)

What is GAN (Generative Adversarial Networks)?

As per wikipedia:

Generative adversarial networks (GANs) are a class of artificial intelligence algorithms used in unsupervised machine learning, implemented by a system of two neural networks contesting with each other in a zero-sum game framework. They were introduced by Ian Goodfellow et al. in 2014.^[1]
Wikipedia

GAN was introduced in a paper by Ian Goodfellow and other researchers at the University of Montreal, including Yoshua Bengio, in 2014.

GANs are deep neural net architectures comprised of two models fighting one against the other and thus called “adversarial”.

Generator and Discriminator

To understand the GANs, we have to understand the Generator and Discriminator algorithms. When we are training the GANs, actually Generator is fighting with discriminator. Like Generator generates the image of Dog and pass it to discriminator, now its discriminator to identify whether generated image is dog or not.

Another way to define these two modesl like below:

Generative models model the distribution of individual classes and pass the generated output to a discriminator
Discriminative models learn the boundary between classes and work as a classifier. It confirms the assumption which was made by Generative model. Like if Generative model generated the Dog’s image. This model will confirm whether the image is of Dog or not.

GANs Training:

A typical GANs training works like below:

The generator model generates an image.
This generated image from the step above is fed into the discriminator model alongside a stream of images taken from the actual dataset.
The discriminator model uses both real and fake images and returns probabilities, a number between 0 and 1, with 1 representing a prediction of authenticity and 0 representing fake.

Actually GAN will have a double feedback loop:

The discriminator is in a feedback loop with the ground truth of the images
The generator is in a feedback loop with the discriminator.

Below is the example of Generative Adversarial Network framework

Let’s create a simple GAN using MNIST Data:

Discriminator

Let’s define the discriminator. Given an image, Discriminator has to determine whether the image is fake or not. The input of our discriminator network is a (28x28x1) input which is the same as single MNIST dataset’s image. The output is a single node with a probability of 1 for real, 0 for a fake.

def discriminator():
    
    model = Sequential()
    input_shape = (28, 28, 1)
    dropout_prob = 0.4

    model.add(Conv2D(64, 5, strides=2, input_shape=input_shape, padding='same'))
    model.add(LeakyReLU())
    
    model.add(Conv2D(128, 5, strides=2, padding='same'))
    model.add(LeakyReLU())
    model.add(Dropout(dropout_prob))
    
    model.add(Conv2D(256, 5, strides=2, padding='same'))
    model.add(LeakyReLU())
    model.add(Dropout(dropout_prob))
    
    model.add(Conv2D(512, 5, strides=1, padding='same'))
    model.add(LeakyReLU())
    model.add(Dropout(dropout_prob))
    
    model.add(Flatten())
    model.add(Dense(1))
    model.add(Activation('sigmoid'))
    
    return model

Verify the Discriminator model

discriminator = discriminator()
discriminator.summary()

Output:
Layer (type) Output Shape Param #
=================================================================
conv2d_1 (Conv2D) (None, 14, 14, 64) 1664
_________________________________________________________________
leaky_re_lu_1 (LeakyReLU) (None, 14, 14, 64) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 7, 7, 128) 204928
_________________________________________________________________
leaky_re_lu_2 (LeakyReLU) (None, 7, 7, 128) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 7, 7, 128) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 4, 4, 256) 819456
_________________________________________________________________
leaky_re_lu_3 (LeakyReLU) (None, 4, 4, 256) 0
_________________________________________________________________
dropout_2 (Dropout) (None, 4, 4, 256) 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 4, 4, 512) 3277312
_________________________________________________________________
leaky_re_lu_4 (LeakyReLU) (None, 4, 4, 512) 0
_________________________________________________________________
dropout_3 (Dropout) (None, 4, 4, 512) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 8192) 0
_________________________________________________________________
dense_1 (Dense) (None, 1) 8193
_________________________________________________________________
activation_1 (Activation) (None, 1) 0
=================================================================
Total params: 4,311,553
Trainable params: 4,311,553
Non-trainable params: 0
_________________________________________________________________

Generator

Let’s define the generator model. Main aim of generator model to generate the images something like Model will say that assume this image is of number 1 of MINST data and will ask discriminator to verify whether features of the generated images is matching with original or not.

Below generator will start with a random vector of noise with size of 100 and gradually upsample. To improve the output of the generator model using “UpSampling2D“ normal convolutions instead of transposed convolutions.

def generator():

model = Sequential()
dropout_prob = 0.4

model.add(Dense(7*7*256, input_dim=100))
model.add(BatchNormalization(momentum=0.9))
model.add(LeakyReLU())
model.add(Reshape((7,7,256)))
model.add(Dropout(dropout_prob))

model.add(UpSampling2D())
model.add(Conv2D(128, 5, padding=’same’))
model.add(BatchNormalization(momentum=0.9))
model.add(LeakyReLU())

model.add(UpSampling2D())
model.add(Conv2D(64, 5, padding=’same’))
model.add(BatchNormalization(momentum=0.9))
model.add(LeakyReLU())

model.add(Conv2D(32, 5, padding=’same’))
model.add(BatchNormalization(momentum=0.9))
model.add(LeakyReLU())

model.add(Conv2D(1, 5, padding=’same’))
model.add(Activation(‘sigmoid’))

return model

Verify the architecture of generator:

generator = generator()
generator.summary()

Output of Generator is like below

Now define the model using architecture defined above

optimizer4discriminator = RMSprop(lr=0.0008, clipvalue=1.0, decay=1e-10)
discriminator_model = Sequential()
discriminator_model.add(discriminator)
discriminator_model.compile(loss=’binary_crossentropy’, optimizer=optimizer4discriminator, metrics=[‘accuracy’])

discriminator_model.summary()

Output:

Layer (type) Output Shape Param # ================================================================= sequential_1 (Sequential) (None, 1) 4311553 ================================================================= Total params: 4,311,553
Trainable params: 4,311,553
Non-trainable params: 0

optimizer4adversarial = Adam(lr=0.0004, clipvalue=1.0, decay=1e-10)
generator_model = Sequential()
generator_model.add(generator)

# Disable layers in discriminator
for layer in discriminator.layers:
layer.trainable = False

generator_model.add(discriminator)
generator_model.compile(loss=’binary_crossentropy’, optimizer=optimizer4adversarial, metrics=[‘accuracy’])

generator_model.summary()

Read the MNIST dataset for the trianing

# Read MNIST data
x_train = input_data.read_data_sets(“mnist”, one_hot=True).train.images
x_train = x_train.reshape(-1, 28, 28, 1).astype(np.float32)

Train GANs

batch_size = 256

vis_noise = np.random.uniform(-1.0, 1.0, size=[16, 100])

loss_adv = []
loss_dis = []
acc_adv = []
acc_dis = []
plot_iteration = []

for i in range(10001):

# Select a random set of training images from the mnist dataset
images_train = x_train[np.random.randint(0, x_train.shape[0], size=batch_size), :, :, :]
# Generate a random noise vector
noise = np.random.uniform(-1.0, 1.0, size=[batch_size, 100])
# Use the generator to create fake images from the noise vector
images_fake = generator.predict(noise)

# Create a dataset with fake and real images
x = np.concatenate((images_train, images_fake))
y = np.ones([2*batch_size, 1])
y[batch_size:, :] = 0

# Train discriminator for one batch
d_stats = discriminator_model.train_on_batch(x, y)

# Train the generator
# The input of th adversarial model is a list of noise vectors. The generator is ‘good’ if the discriminator classifies
# all the generated images as real. Therefore, the desired output is a list of all ones.
y = np.ones([batch_size, 1])
noise = np.random.uniform(-1.0, 1.0, size=[batch_size, 100])
a_stats = generator_model.train_on_batch(noise, y)

if i % 50 == 0:
plot_iteration.append(i)
loss_adv.append(a_stats[0])
loss_dis.append(d_stats[0])
acc_adv.append(a_stats[1])
acc_dis.append(d_stats[1])

clear_output(wait=True)

fig, (ax1, ax2) = plt.subplots(1,2)
fig.set_size_inches(16, 8)

ax1.plot(plot_iteration, loss_adv, label=”loss Generator”)
ax1.plot(plot_iteration, loss_dis, label=”loss Discriminator”)
ax1.set_ylim([0,5])
ax1.legend()

ax2.plot(plot_iteration, acc_adv, label=”acc Generator”)
ax2.plot(plot_iteration, acc_dis, label=”acc Discriminator”)
ax2.legend()

plt.show()

# Optional, print losses instead of plotting with:
# print(“{}: [Dis. loss: {:.4f}, acc: {:.4f}] [Gen. loss: {:.4f}, acc: {:.4f}]”.format(i, d_stats[0], d_stats[1], a_stats[0], a_stats[1]))

if i % 500 == 0:
# Visualize the performance of the generator by producing images from the test vector
images = generator.predict(vis_noise)
# Map back to original range
#images = (images + 1 ) * 0.5
plt.figure(figsize=(10,10))

for im in range(images.shape[0]):
plt.subplot(4, 4, im+1)
image = images[im, :, :, :]
image = np.reshape(image, [28, 28])

plt.imshow(image, cmap=’gray’)
plt.axis(‘off’)

plt.tight_layout()
plt.savefig(r’output/mnist-normal/{}.png’.format(i))
plt.close(‘all’)

Output:

Output of Generative models loos like below

In conclusion, this was a very simple GANs using the Keras high-level API. You can find today’s code here.

#100DaysofMLCode #GAN

DAY 82-100 DAYS MLCODE: GAN