VGG16 → from scratch using Transfer Learning with Keras and TensorFlow 2

VGG16 Model

If we are gonna build a computer vision application, i.e. for example, let’s take an example like Image Classification, we could use Transfer Learning instead of training from the scratch. By this way we often make faster progress in training the model since we are just making use of someone else’s trained model and we can use that to do new tasks for us.

The Computer Vision research community has been posting a lot of data sets on the internet like Image Net or MS COCO or PASCAL types of data sets. Well, these are some of the data sets that are famous across the globe as of now on which people have done enough research and are being used practiced in day to day life with real time solutions.

In Transfer Learning the trick is very simple: we don’t train all the layers of the model. We just freeze all the layers and just train the lower layers of the model, i.e. making use of the trained model weights and so this makes retraining very simple.

Well, here in this example I have actually done a Dog and Cat classification using VGG-16 and used ImageNet in this example.

Before starting the example, let me tell you about ImageNet.

What is ImageNet ?

ImageNet is a large image dataset which has been organized according to the WordNet hierarchy. They actually have around 14,197,122 images totally dated today, i.e. 14 Million images. There are actually around 100,000 synsets in WordNet, where each synset has around 1000 images just for illustration.

This is actually designed by people with really good IQ. Well that’s just an indirect way to say proudly that “NOT ME…!!!”

Moreover, every single image that is actually used in ImageNet is manually annotated by us humans. We can be proud of that since bots ain’t doing this job.

Well, now the Cat and Dog dataset actually has around 1000 → cat and dog training images and for validation it has around 500 → cat and dog images.

Link for dataset

Once you have downloaded all the dataset you can proceed with the implementation. Here I have built from scratch, according to the VGG-16.

We will be using TensorFlow and Keras.

Points to be noted here,

If you’re using TensorFlow Version 2.x then there ain’t any changes with the code.

If you’re deploying it in your Local Machine make sure you have enough RAM.

If you’re using cloud, then you need not worry. Since I’ve deployed it in Google Colab.

All the links which you need to implement this I have uploaded in my GitHub

narenltk - Overview

First thing is importing the dataset, if you’ve downloaded the dataset manually and then importing the data set you can use OpenCV with Glob.

If you want to use your Local Machine then I recommend you guys to create a Virtual Environment and get the work done. It helps in solving a lots of problem and if your don’t know or couldn’t remember how to do it you can refer the link below.

Why do we need a virtual environment ?

Here I have downloaded and unzipped the following way and it is applicable in both Linux and Windows Machine. I have been working with Windows.

Dataset

Here most of the coding will be based on only Keras which uses TensorFlow at the back-end.

Once you’re done with this you need to use the Training Data and Validation Data for which I am going to use ImageDataGenerator.

Keras the Deep Learning Library by default provides us a way to fit models using Image Data Augmentation with ImageDataGenerator class.

ImageDataGenerator actually expands the training dataset so the performance improves.

ImageDataGenerator

Here I have given the target_size as 224, this is because the input to the ConvNets is a fixed size which is 224*224 RGB image as stated in the VGG 16 architecture paper.

You can also have a look at the details given in the documentation page of Keras.

ImageDataGenerator

Keras documentation: Image data preprocessing

There are 2 ways to my knowledge for implementing the VGG-16.

1. You can either write code from scratch with the help of Keras.

2. You can just import the VGG-16 function from Keras

Keras supports you in all ways.

Here in this example I have done with method 1 as stated above. However, I will show the easy way first.

Simple Way

Now in order to show you how to create from scratch is done as shown below

VGG 16 model

Here you can do a summary of the model and verify it.

If you’re the vgg16 by importing keras then you need to pop up the last layer which is the final Fully Connected layer.

To find the Summary of the model you need to give the model name i,e, model.summary() as shown below

model.summary()

Now that we have come this far without any bugs, all we need now is a Checkpoint, Early stopping and fitting the model for training.

ModelCheckPoint

ModelCheckPoint is which saves the model by monitoring the specific parameter of the model.

Here I am monitoring the validation accuracy by passing the val_loss to ModelCheckPoint. The model will be saved when the current epoch validation loss is lesser than the previous epoch.

ModelCheckPoint

EarlyStopping

EarlyStopping will stop the training of the model earlier itself when there is no increase in the parameters which we have set for monitoring. You can define 100 epochs or 1000 epochs or even 10000 epochs, but training will stop based on this function alone.

EarlyStopping

Here we have set the monitor as val_accuracy in EarlyStopping and val_loss, you can do your research which method to use for monitoring it can be either same or else different but make sure to find out the answer yourself.

But you can also stop even before there are no changes in monitoring value, by simply adding a parameter patience = 20 / patience = 10. Which stops the training once the model reaches 20 / 10 epochs.

model.fit

Now the final part here is model.fit_generator since we have used ImageDataGenerator class.

In the fit_generator we need to pass the training data, validation data, steps_per_epoch along with the epochs, the steps and other things you can do based on the system requirements and based on that the training time will extend.

But unfortunately this fit_generator actually works with TensorFlow 1.x version and for 2.x version it is different.

For TensorFlow 2.x you need to simply call model.fit and not model.fit_generator. Well problem solved I guess. TensorFlow 2.x is actually small minor changes in the codes here and there.

Nothing much of a change as shown below.

model.fit

Once you’re done with this you have to save the weights here I have been working with Google Colab and so i have saved the model weights in google drive and the format to which it has to be saved you can mention i.e. you can save the weights in .hdf5, .h5, .pkl etc..

Ref:

https://keras.io/api/

http://image-net.org/about-overview

If you wish to stay connected,

Narendiran Krishnan - Bengaluru, Karnataka, India | Professional Profile | LinkedIn

you can also just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!