Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Computer Vision with TensorFlow, Getting Started

Computer Vision with TensorFlow, Getting Started

I plan to first start with the idea of CNNs and why they should be used when we already have common or dense NNs. I would then move on to show the clear win which CNN creates for image classification as it is all about picking out features. I then plan to show attendees how an image journeys through convolutional layers and how it extracts features, I would also show them a visualization of the same. Having done this I plan to show the attendees how TensorFlow makes it easy to load and label images in the runtime, I would also show them the use of image augmentation with TF and how it works in TF to save resources in the disk space and also dropouts to reduce potential overfitting. Then, I plan to show them how to use the indispensable Transfer Learning so we can efficiently utilize the work done by other people. At the last I also plan to show the different output layers one should use for binary classification and categorical.

Rishit Dagli

January 04, 2021
Tweet

More Decks by Rishit Dagli

Other Decks in Programming

Transcript

  1. Computer Vision with TF Rishit Dagli High School, TEDx, Ted-Ed

    speaker rishit.tech @rishit_dagli Rishit-dagli
  2. • High School Student • TEDx and Ted-Ed Speaker •

    ♡ Hackathons and competitions • ♡ Research • My coordinates - www.rishit.tech $whoami rishit_dagli Rishit-dagli
  3. rishit.tech Activity Recognition if(speed<4){ status=WALKING; } if(speed<4){ status=WALKING; } else

    { status=RUNNING; } if(speed<4){ status=WALKING; } else if(speed<12){ status=RUNNING; } else { status=BIKING; }
  4. rishit.tech Activity Recognition if(speed<4){ status=WALKING; } if(speed<4){ status=WALKING; } else

    { status=RUNNING; } if(speed<4){ status=WALKING; } else if(speed<12){ status=RUNNING; } else { status=BIKING; } // Oh crap
  5. rishit.tech Activity Recognition 0101001010100101010 1001010101001011101 0100101010010101001 0101001010100101010 Label = WALKING

    1010100101001010101 0101010010010010001 0010011111010101111 1010100100111101011 Label = RUNNING 1001010011111010101 1101010111010101110 1010101111010101011 1111110001111010101 Label = BIKING 1111111111010011101 0011111010111110101 0101110101010101110 1010101010100111110 Label = GOLFING (Sort of)
  6. rishit.tech What to expect??? • Building data input pipelines using

    the tf.keras.preprocessing.image.ImageDataGenerator class to efficiently work with data on disk to use with the model. • Build and test model. • Overfitting —How to identify and prevent it. • Data augmentation and dropout —techniques to fight overfitting
  7. rishit.tech Images Training Cats Dogs Validation Cats Dogs 1.jpg 2.jpg

    3.jpg 4.jpg 5.jpg 6.jpg 7.jpg 9.jpg 8.jpg 10.jpg
  8. rishit.tech Images Training Cats Dogs 1.jpg 2.jpg 3.jpg 4.jpg 5.jpg

    6.jpg 7.jpg 9.jpg 8.jpg 10.jpg Validation Cats Dogs
  9. rishit.tech Images Training Cats Dogs Validation Cats Dogs 1.jpg 2.jpg

    3.jpg 4.jpg 5.jpg 6.jpg 7.jpg 9.jpg 8.jpg 10-.jpg
  10. rishit.tech total training cat images: 1000 total training dog images:

    1000 total validation cat images: 500 total validation dog images: 500 -- Total training images: 2000 Total validation images: 1000
  11. rishit.tech import tensorflow as tf from tensorflow.keras.models import Sequential from

    tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D from tensorflow.keras.preprocessing.image import ImageDataGenerator import os import numpy as np import matplotlib.pyplot as plt
  12. rishit.tech Data Preparation 1. Read images from the disk. 2.

    Decode contents of these images and convert it into proper grid format as per their RGB content. 3. Convert them into floating point tensors. 4. Rescale the tensors from values between 0 and 255 to values between 0 and 1, as neural networks prefer to deal with small input values.
  13. rishit.tech # Generator for our training data train_image_generator = ImageDataGenerator(rescale=1./255)

    train_data_gen = train_iamge_generator.flow_from_directory( batch_size=batch_size, directory=train_dir, shuffle=True, target_size=(IMG_HEIGHT,IMG_WIDTH) class_mode='binary')
  14. rishit.tech # Generator for our training data train_image_generator = ImageDataGenerator(rescale=1./255)

    train_data_gen = train_image_generator.flow_from_directory( batch_size=batch_size, directory=train_dir, shuffle=True, target_size=(IMG_HEIGHT,IMG_WIDTH) class_mode='binary')
  15. rishit.tech # Generator for our training data train_image_generator = ImageDataGenerator(rescale=1./255)

    train_data_gen = train_image_generator.flow_from_directory( batch_size=batch_size, directory= train_dir, shuffle=True, target_size=(IMG_HEIGHT,IMG_WIDTH) class_mode='binary')
  16. rishit.tech # Generator for our training data train_image_generator = ImageDataGenerator(rescale=1./255)

    train_data_gen = train_image_generator.flow_from_directory( batch_size=batch_size, directory= train_dir, shuffle=True, target_size=(IMG_HEIGHT,IMG_WIDTH) class_mode='binary')
  17. rishit.tech # Generator for our training data train_image_generator = ImageDataGenerator(rescale=1./255)

    train_data_gen = train_image_generator.flow_from_directory( batch_size=batch_size, directory= train_dir, shuffle=True, target_size=(IMG_HEIGHT,IMG_WIDTH) class_mode='binary')
  18. rishit.tech # Generator for our training data train_image_generator = ImageDataGenerator(rescale=1./255)

    train_data_gen = train_image_generator.flow_from_directory( batch_size=batch_size, directory= train_dir, shuffle=True, target_size=(IMG_HEIGHT,IMG_WIDTH) class_mode='binary')
  19. rishit.tech # Generator for our validation data validation_image_generator = ImageDataGenerator(rescale=1./255)

    val_data_gen = validation_imadata_generator.flow_from_directory ( batch_size=batch_size, directory= validation_dir, shuffle=True, target_size=(IMG_HEIGHT,IMG_WIDTH) class_mode='binary')
  20. model = Sequential([ Flatten( input_shape = (150,150,3), Dense(512 , activation

    = ‘relu’, Dense(2, activation=’softmax’) ])
  21. model = Sequential([ Flatten( input_shape = (150,150,3), Dense(512 , activation

    = ‘relu’, Dense( 2 , activation=’softmax’) ])
  22. model = Sequential([ Flatten( input_shape = (150,150,3), Dense( 512 ,

    activation = ‘relu’, Dense(2, activation=’softmax’) ])
  23. rishit.tech 0 64 128 48 192 144 142 226 168

    -1 0 -2 .5 4.5 -1.5 1.5 2 -3 Current Pixel Value is 192 Consider neighbor Values Filter Definition CURRENT_PIXEL_VALUE = 192 NEW_PIXEL_VALUE = (-1 * 0) + (0 * 64) + (-2 * 128) + (.5 * 48) + (4.5 * 192) + (-1.5 * 144) + (1.5 * 42) + (2 * 226) + (-3 * 168)
  24. model = Sequential([ Conv2D(16, (3,3), padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),

    MaxPooling2D(2,2), Conv2D(32, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Conv2D(64, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Flatten(), Dense(512, activation='relu'), Dense(1, activation='sigmoid') ])
  25. model = Sequential([ Conv2D(16, (3,3), padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),

    MaxPooling2D(2,2), Conv2D(32, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Conv2D(64, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Flatten(), Dense(512, activation='relu'), Dense(1, activation='sigmoid') ])
  26. rishit.tech 0 64 128 128 48 192 144 144 142

    226 168 0 255 0 0 64 0 64 48 192 192 128 128 144 144 144 142 226 255 0 255 168 0 0 64 168 192 144 255 168
  27. model = Sequential([ Conv2D(16, (3,3), padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),

    MaxPooling2D(2,2), Conv2D(32, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Conv2D(64, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Flatten(), Dense(512, activation='relu'), Dense(1, activation='sigmoid') ])
  28. model = Sequential([ Conv2D(16, (3,3), padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),

    MaxPooling2D(2,2), Conv2D(32, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Conv2D(64, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Flatten(), Dense(512, activation='relu'), Dense(1, activation='sigmoid') ])
  29. model = Sequential([ Conv2D(16, (3,3), padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),

    MaxPooling2D(2,2), Conv2D(32, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Conv2D(64, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Flatten(), Dense(512, activation='relu'), Dense(1, activation='sigmoid') ])
  30. Layer (type) Output Shape Param # ================================================================= conv2d_3 (Conv2D) (None,

    150, 150, 16) 448 _________________________________________________________________ max_pooling2d_3 (MaxPooling2 (None, 75, 75, 16) 0 _________________________________________________________________ conv2d_4 (Conv2D) (None, 75, 75, 32) 4640 _________________________________________________________________ max_pooling2d_4 (MaxPooling2 (None, 37, 37, 32) 0 _________________________________________________________________ conv2d_5 (Conv2D) (None, 37, 37, 64) 18496 _________________________________________________________________ max_pooling2d_5 (MaxPooling2 (None, 18, 18, 64) 0 _________________________________________________________________ flatten_1 (Flatten) (None, 20736) 0 _________________________________________________________________ dense_2 (Dense) (None, 512) 10617344 _________________________________________________________________ dense_3 (Dense) (None, 1) 513 ================================================================= Total params: 10,641,441 Trainable params: 10,641,441 Non-trainable params: 0
  31. rishit.tech 1. Add more data 2. Use data augmentation 3.

    Use architectures that generalize well.
  32. rishit.tech Data Augmentation Apply some random transformation. The Goal is

    the model never sees the exact same picture twice. Use ImageDataGenerator to perform transformation
  33. rishit.tech Let’s Put this together. image_gen_train = ImageDataGenerator( rescale=1./255, rotation_range=45,

    width_shift_range=.15, height_shift_range=.15, horizontal_flip=True, zoom_range=0.5 )
  34. rishit.tech DropOut Another technique to overcome overfitting. It drops some

    of the output units from the applied layer during the training process It takes values such as 0.1, 0.2, 0.3, etc. which means 10%, 20%, 30% of the output dropping off.
  35. rishit.tech Creating a network with Dropouts model = Sequential([ Conv2D(16,

    (3,3), padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)), MaxPooling2D(2,2), Dropout(0.2), Conv2D(32, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Conv2D(64, (3,3), padding='same', activation='relu'), MaxPooling2D(2,2), Dropout(0.2), Flatten(), Dense(512, activation='relu'), Dense(1, activation='sigmoid') ])
  36. rishit.tech Testing import numpy as np from google.colab import files

    from keras.preprocessing import image uploaded = files.upload() for fn in uploaded.keys(): path = '/content/' + fn img = image.load_img(path, target_size=(150, 150)) x = image.img_to_array(img) x = np.expand_dims(x, axis=0) images = np.vstack([x]) classes = model.predict(images, batch_size=10) print(classes[0]) if classes[0]>0.5: print(fn + " is a dog") else: print(fn + " is a cat")