Introduction to Image Processing: 2.Spatial

Transcript

1. Digital Image Processing Spatial Processing (Week 2,3) NHSM - 4th year - Fall 2024 - Prof. Mohammed Hachama hachamam@gmail.com http://hachama.github.io/home/
2. Outline Intensity Transformations Linear Filtering Non-linear filtering NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 2/18
3. Intensity Transformations NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 3/18
4. Types of transformations Transforming an image: g = T ◦ f New pixel value depends only on the input one New pixel value depends on the surrounding input pixels New pixel value depends on the whole input image NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 4/18
5. Types of transformations Transforming an image: g = T ◦ f • Point transformations: g(x, y) = T[f (x, y)] % Import libraries import cv2 as cv import numpy as np % Read data im = cv.imread(’data/lake.jpg’,0) mn, mx, mnx = im.min(), im.max(), mx-mn h,w=im.shape[0], im.shape[1] out =im.copy() % Processing NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 4/18
6. Types of transformations Transforming an image: g = T ◦ f • Point transformations: g(x, y) = T[f (x, y)] • Processing: two approaches • Naive approach: iterate over pixels for i in range(h): for j in range(w): out[i,j] = 255 * (im[i,j]-mn)/mnx out = out.astype(np.uint8) Complexity (# of operations)? NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 4/18
7. Types of transformations Transforming an image: g = T ◦ f • Point transformations: g(x, y) = T[f (x, y)] • Processing: two approaches • Gray level r ∈ [0, 255] is mapped to s ∈ [0, 255] • Using a reference table (lookup table). NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 4/18
8. Types of transformations Transforming an image: g = T ◦ f • Point transformations: g(x, y) = T[f (x, y)] • Processing: two approaches // Initialization of the LUT LUT = 255*(np.array(range(256))-mn)/mnx for i in range(h): for j in range(w): out[i,j] = LUT(im[i,j]) out = out.astype(np.uint8) Complexity (# of operations)? NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 4/18
9. Point transformations Example 1: Negative (or complement) of images g(x, y) = 255 − f (x, y) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 5/18
10. Point transformations Example 2: Contrast stretching g(x, y) = T (f (x, y)) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 5/18
11. Point transformations Example 2: Contrast stretching g(x, y) = T (f (x, y)) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 5/18
12. Point transformations Example 3: Thresholding g(x, y) ∈ {0, 1} or {0, 255}. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 5/18
13. Point transformations Example 3: Thresholding g(x, y) ∈ {0, 1} or {0, 255}. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 5/18
14. Point transformations Example 4: Logarithmic transformations s = c log(1 + r) • Stretching light values and compressing dark values. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 5/18
15. Point transformations Example 5: Exponential transformations s = crγ • γ < 1: Compress light values and stretch dark values. γ = 1 γ = 3 γ = 0.4 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 5/18
16. Local Transformations • The intensity of the image output at a pixel is obtained as a function of those pixels belonging to the chosen core NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 6/18
17. Local Transformations • The intensity of the image output at a pixel is obtained as a function of those pixels belonging to the chosen core NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 6/18
18. Local Transformations • The intensity of the image output at a pixel is obtained as a function of those pixels belonging to the chosen core NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 6/18
19. Local Transformations • The intensity of the image output at a pixel is obtained as a function of those pixels belonging to the chosen core NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 6/18
20. Local Transformations • The intensity of the image output at a pixel is obtained as a function of those pixels belonging to the chosen core See next (linear and nonlinear filtering, image restoration) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 6/18
21. Global Transformations (1) Image Histogram • Many global techniques are histogram-based. • Histogram = ”graphical” representation of the gray-level distribution of an image. • H(k) = Number of pixels with a gray-level k. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 7/18
22. Global Transformations (1) Image Histogram NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 7/18
23. Global Transformations (1) Image Histogram • Histogram provides a good indication of the photometric composition of the image: uniform, unimodal, multimodal. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 7/18
24. Global Transformations (1) Image Histogram • Two different images having the same histogram: no information on spatial distribution. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 7/18
25. Global Transformations (1) Image Histogram • Color image: one histogram for each channel, or • Color images → Gray-level images • Mean value: I = 1 3 (R + G + B) • Luminance: L = 0.299R + 0.587G + 0.114B Weights derived according to the perception of the human eye. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 7/18
26. Global Transformations (2) Example 1: Histogram Equalization Automatic determination of a transformation T to achieve a uniform histogram NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 8/18
27. Global Transformations (2) Example 1: Histogram Equalization Automatic determination of a transformation T to achieve a uniform histogram NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 8/18
28. Global Transformations (2) Example 1: Histogram Equalization Automatic determination of a transformation T to achieve a uniform histogram NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 8/18
29. Global Transformations (2) Example 1: Histogram Equalization • Step 1: Compute the histogram h(k), k ∈ [0, 255] • Step 2: Histogram normalization hn (k) = h(k) N , k ∈ [0, 255], N := #of Pixels • Step 3: Cumulative probability distribution C(k) = k j=0 hn (k), k ∈ [0, 255] • Step 4: Transformation of the intensities f ′(x, y) = 255 × C(f (x, y)) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 8/18
30. Global Transformations (2) Example 1: Histogram Equalization A 4-bits image of size 4 × 4 and the equalized version 13 14 2 14 10 2 5 9 15 15 3 15 15 8 13 1 9 11 3 11 7 3 5 7 15 15 4 15 15 5 9 1 n 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ni 0 1 2 1 0 1 0 0 1 1 1 0 0 2 2 4 fi 0 .0625 .125 .0625 0 .0625 0 0 .0625 .0625 .0625 0 0 .125 .125 0.25 fi 0 .0625 .1875 .25 .25 .3125 .3125 .3125 .375 .4375 .5 .5 .5 .625 .75 1 n′ 0 1 3 4 4 5 5 5 6 7 8 8 8 9 11 15 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 8/18
31. Global Transformations (3) Example 2: Adaptive Histogram Equalization (AHE) Input image Equalized image NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 9/18
32. Global Transformations (3) Example 2: Adaptive Histogram Equalization (AHE) Input image Equalized image Apply histogram equalization to small regions (tiles) of the image. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 9/18
33. Global Transformations (3) Example 2: Adaptive Histogram Equalization (AHE) • Algorithm Steps 1. Divide the image into tiles (smaller regions, e.g., 8x8 tiles). 2. Compute histograms for each tile. 3. Apply histogram equalization on each tile. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 9/18
34. Global Transformations (3) Example 2: Adaptive Histogram Equalization (AHE) Input image Equalized image (HE) Equalized image (AHE) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 9/18
35. Global Transformations (3) Example 2: Adaptive Histogram Equalization (AHE) Input image Equalized image (AHE) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 9/18
36. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Histogram Equalization and AHE can lead to over-enhancement of noise in homogeneous regions. • → CLAHE (Contrast Limited Adaptive Histogram Equalization) limits the amplification of noise by limiting the contrast of each tile based on a predefined clip limit. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
37. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Algorithm Steps 1. Divide the image into tiles (smaller regions, e.g., 8x8 tiles). 2. Compute histograms for each tile. 3. Clip the histograms, i.e., limit the histogram values to the clip limit to prevent noise amplification. 4. Redistribute the clipped pixels uniformly across the histogram. 5. Apply histogram equalization on each tile. 6. Interpolate the pixel values in the tile borders (for smoothness). NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
38. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Original Image               52 55 61 59 79 61 76 60 85 70 64 59 59 55 61 79 85 70 64 59 59 55 61 79 52 55 61 59 79 61 76 60 85 70 64 59 59 55 61 79 85 70 64 59 59 55 61 79 52 55 61 59 79 61 76 60 85 70 64 59 59 55 61 79               NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
39. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Step 1: Divide the Image into (2 × 2) Tiles Tile 1:      52 55 61 59 85 70 64 59 85 70 64 59 52 55 61 59      Tile 2:      79 61 76 60 59 55 61 79 59 55 61 79 79 61 76 60      Tile 3:      85 70 64 59 85 70 64 59 52 55 61 59 85 70 64 59      Tile 4:      59 55 61 79 59 55 61 79 79 61 76 60 59 55 61 79      NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
40. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Steps 2+3+4 Tile 1: Intensity Original freq. Clipped freq. Redistrib. freq. 52 2 2 2 + 2/7 55 2 2 2 + 2/7 59 4 2 2 + 2/7 61 2 2 2 + 2/7 64 2 2 2 + 2/7 70 2 2 2 + 2/7 85 2 2 2 + 2/7 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
41. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Steps 2+3+4 Tile 2: Intensity Original freq. Clipped freq. Redistrib. freq. 55 2 2 2 + 4/6 59 2 2 2 + 4/6 60 2 2 2 + 4/6 61 4 2 2 + 4/6 76 2 2 2 + 4/6 79 4 2 2 + 4/6 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
42. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Steps 2+3+4 Tile 3: Intensity Original freq. Clipped freq. Redistrib. freq. 52 1 1 1 + 5/7 55 1 1 1 + 5/7 59 4 2 2 + 5/7 61 1 1 1 + 5/7 64 3 2 2 + 5/7 70 3 2 2 + 5/7 85 3 2 2 + 5/7 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
43. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Steps 2+3+4 Tile 4: Intensity Original freq. Clipped freq. Redistrib. freq. 55 3 2 3 59 3 2 3 60 1 1 2 61 4 2 3 76 1 1 2 79 4 2 3 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
44. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Step 5: Apply Histogram Equalization on each tile. Tile 1 Tile 2 Intensity Rela. freq. CDF Equal. val. Intensity Real. freq. CDF Equal. val. 52 0.1429 0.1429 32 55 0.1667 0.1667 43 55 0.1429 0.2857 73 59 0.1667 0.3333 85 59 0.1429 0.4286 109 60 0.1667 0.5 128 61 0.1429 0.5714 146 61 0.1667 0.6667 170 64 0.1429 0.7143 182 76 0.1667 0.8333 213 70 0.1429 0.8571 219 79 0.1667 1 255 85 0.1429 1 255 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
45. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Step 5: Apply Histogram Equalization on each tile. Tile 3 Tile 4 Intensity Rela. freq. CDF Equal. val. Intensity Real. freq. CDF Equal. val. 52 0.1071 0.1071 27 55 0.1875 0.1875 43 55 0.1071 0.2143 55 59 0.1875 0.375 85 59 0.2143 0.4286 109 60 0.125 0.5 128 61 0.1071 0.5357 137 61 0.1875 0.6875 170 64 0.2143 0.7500 191 76 0.125 0.8125 207 70 0.2143 0.9643 246 79 0.1875 1 255 85 0.2143 1 255 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
46. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Resulting Equalized Image (before step 6)               32 73 146 109 255 170 213 128 255 219 182 109 144 43 170 255 255 219 182 109 144 43 170 255 32 73 146 109 255 170 213 128 255 246 191 109 85 43 170 255 255 246 191 109 85 43 170 255 27 55 137 109 255 170 207 128 255 246 191 109 85 43 170 255               NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
47. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Step 6: Interpolate the pixel values in the tile borders. For simplicity, assume the borders are averaged. • Horizontal averaging.               32 73 146 182 255 170 213 128 255 219 182 127 144 43 170 255 255 219 182 182 144 43 170 255 32 73 146 109 255 170 213 128 255 246 191 97 85 43 170 255 255 246 191 97 85 43 170 255 27 55 137 182 255 170 207 128 255 246 191 97 85 43 170 255               NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
48. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Step 6: Interpolate the pixel values in the tile borders. For simplicity, assume the borders are averaged. • Vertical averaging.               32 73 146 182 255 170 213 128 255 219 182 127 144 43 170 255 255 219 182 182 144 43 170 255 144 160 164 109 170 107 192 192 255 246 191 97 85 43 170 255 255 246 191 97 85 43 170 255 27 55 137 182 255 170 207 128 255 246 191 97 85 43 170 255               NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
49. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Step 6: Interpolate the pixel values in the tile borders. For simplicity, assume the borders are averaged. • Corner averaging.               32 73 146 182 255 170 213 128 255 219 182 127 144 43 170 255 255 219 182 182 144 43 170 255 144 160 164 140 170 107 192 192 255 246 191 97 85 43 170 255 255 246 191 97 85 43 170 255 27 55 137 182 255 170 207 128 255 246 191 97 85 43 170 255               NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
50. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Resulting Equalized Image Original image Equalized image NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
51. Global Transformations (4) Example 3: Contrast Limited Adaptive Histogram Equalization • Resulting Equalized Image Original image Equalized image NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 10/18
52. Global Transformations (5) Example 4: Histogram Specification (matching) The histogram of a reference image A is known (hA ), and we would like to modify an image B to produce another image C whose histogram resembles that of A. 1. Compute the cumulative distribution A: ¯ hA = hA . 2. Compute the cumulative distribution B: ¯ hB = hB . 3. For each value of ¯ hB , find the minimum value in ¯ hA that is ≥. The corresponding value of n′ is the new grayscale value in C. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 11/18
53. Global Transformations (5) Example 4: Histogram Specification (matching) 13 14 2 14 10 2 5 9 15 15 3 15 15 8 13 1 11 13 0 13 7 0 2 5 15 15 1 15 15 4 11 0 13 14 2 14 10 2 5 9 15 15 3 15 15 8 13 2 Reference Input Output n 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 hA 0 .0625 .1875 .25 .25 .3125 .3125 .3125 .375 .4375 .5 .5 .5 .625 .75 1 ni 3 1 1 0 1 1 0 1 0 0 0 2 0 2 0 4 hB .1875 .0625 .0625 0 .0625 .0625 0 .0625 0 0 0 .125 0 .125 0 .25 hB .1875 .25 .3125 .3125 .375 .4375 .4375 .5 0.5 0.5 0.5 .625 .625 .75 .75 1 n′ 2 3 5 5 8 9 9 10 10 10 10 13 13 14 14 15 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 11/18
54. Global Transformations (5) Example 4: Histogram Specification (matching) • Left/right Image NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 11/18
55. Global Transformations (5) Example 4: Histogram Specification (matching) • Right image histogram matched to left image NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 11/18
56. Global Transformations (5) Example 4: Histogram Specification (matching) • Right image histogram matched to left image Source Histogram Target Histogram Matched Histogram NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 11/18
57. Linear Filtering NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 12/18
58. Linear Filtering (1) Filtering = Transformation of an image • Definition • Filter an electronic, optical, or acoustic device: blocks signals or radiations of certain frequencies while allowing others to pass. • Filter a signal or an image: transform that signal/image. • Applications • Noise reduction (Smoothing, denoising,...) • Enhancing discontinuities, features • Implementation of differential operators: derivatives, ... • Multiresolution analysis • Linear filters • Continuous linear filter invariant in time = convolution • Separable filter: decomposable into two 1D filters applied successively in horizontal and vertical directions. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
59. Linear Filtering (1) Filtering: Masking process • Using values in the neighborhood of (x,y) to define the new value • A 3 × 3 pixel mask is common • Applying the pixel mask pixel by pixel across the image • To avoid altering the overall brightness of the image, the sum of the coefficients must be equal to 1 1 3 0 2 10 2 4 1 1 ⊗ 1 0 -1 1 0.1 -1 1 0 -1 = 5 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
60. Linear Filtering (1) Filtering: Masking process NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
61. Linear Filtering (1) Filtering: Masking process NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
62. Linear Filtering (1) Filtering: Masking process NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
63. Linear Filtering (1) Filtering: Masking process NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
64. Linear Filtering (1) Example NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
65. Linear Filtering (1) Example NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
66. Linear Filtering (1) Masking process: Convolution NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
67. Linear Filtering (1) Boundary Conditions • Do not process pixels on the edge: Resulting image is smaller! • Partial mask: Process edge pixels with a small number of neighbors. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
68. Linear Filtering (1) Boundary conditions: Zero-padding H = 1 6 0 1 0 1 2 1 0 1 0 H ⊗ 1 8 6 6 6 3 11 8 8 8 9 10 9 10 10 7 → H ⊗ 0 0 0 0 0 0 0 1 8 6 6 0 0 6 3 11 8 0 0 8 8 9 10 0 0 9 10 10 7 0 0 0 0 0 0 0 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
69. Linear Filtering (1) Boundary conditions: Image Mirror H ⊗ 5 6 7 8 0 6 7 8 5 6 15 8 5 6 7 8 → H ⊗ 5 5 6 7 8 8 5 5 6 7 8 8 0 0 6 7 8 8 5 5 6 15 8 8 5 5 6 7 8 8 5 5 6 7 8 8 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
70. Linear Filtering (1) Example 1: Mean Filter • Filter where all coefficients are equal • Replaces each pixel with the average value of its neighbors • Examples: 3 × 3, 5 × 5 H = 1 9 1 1 1 1 1 1 1 1 1 , H = 1 25 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
71. Linear Filtering (1) Example 1: Mean Filter • Smooths the image • Reduces noise and non-essential details • Blur the image (blur edges) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
72. Linear Filtering (1) Example 2: Gaussian Filter • Weighted average filter: gives more importance to the central pixel and its nearby neighbors. • The weights are determined by the values of a Gaussian. • Separable filter: 2D Gaussian = product of two 1D Gaussians. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
73. Linear Filtering (1) Example 2: Gaussian Filter NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
74. Linear Filtering (1) Example 2: Gaussian Filter NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
75. Linear Filtering (1) Example 2: Gaussian Filter NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
76. Linear Filtering (1) Example 2: Gaussian Filter NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
77. Linear Filtering (1) Example 2: Gaussian Filter NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
78. Linear Filtering (1) Example 2: Gaussian Filter NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
79. Linear Filtering (1) Evaluation of the denoising • Mean Squared Error (MSE): MSE = 1 N N i=1 (Ii − ˆ Ii )2 where N is the # of pixels, I, ˆ I are the original and denoised image. • Root Mean Squared Error (RMSE): same unit as the pixel values RMSE = √ MSE • Normalized Mean Squared Error (NMSE): NMSE = MSE Var(I) where Var(I) is the variance of the original image. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
80. Linear Filtering (1) Evaluation of the denoising • Peak Signal-to-Noise Ratio (PSNR): derived from MSE PSNR = 10 · log 10 R2 MSE where R is the maximum possible pixel value (e.g., 255 for 8-bit images). NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
81. Linear Filtering (1) Evaluation of the denoising • Structural Similarity Index (SSIM): measures the perceived quality of the image based on luminance, contrast, and structure: SSIM(I,ˆ I) = (2µI µˆ I + C1 )(2σ Iˆ I + C2 ) (µ2 I + µ2 ˆ I + C1 )(σ2 I + σ2 ˆ I + C2 ) where σ Iˆ I is the covariance between the original and denoised images and C1 and C2 are constants to stabilize the division. High SSIM (close to 1): Indicates better denoising with preserved image quality. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
82. Linear Filtering (1) Evaluation of the denoising • Structural Similarity Index (SSIM): SSIM(I,ˆ I) = (2µI µˆ I + C1 )(2σ Iˆ I + C2 ) (µ2 I + µ2 ˆ I + C1 )(σ2 I + σ2 ˆ I + C2 ) • Luminance 2µI µˆ I +C1 µ2 I +µ2 ˆ I +C1 : evaluates how similar the average brightness levels of the two images are. • Contrast 2σ Iˆ I +C2 σ2 I +σ2 ˆ I +C2 : similarity of the contrast levels of the images. • Structural information σ Iˆ I +C3 σI σˆ I +C3 : measures how well the patterns of the image (the spatial arrangement of pixels) are preserved. These three components are then multiplied together to form the overall SSIM index. Simplifications occur. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 13/18
83. Linear Filtering (2) Example 3: Derivation Filters - Gradient - • The gradient at a point (x,y) is a two-dimensional vector. • The gradient is a vector perpendicular to the contour. • The norm of the gradient is often referred to as the gradient. • Approximation of the first derivative (Taylor’s formula) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 14/18
84. Linear Filtering (2) Example 3: Derivation Filters - Gradient - • Prewitt (k = 1) and Sobel operators (k = 2) -1 -k -1 -1 0 1 0 0 0 -k 0 k 1 k 1 -1 0 1 • Roberts Operators -1 0 0 -1 0 1 1 0 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 14/18
85. Linear Filtering (2) Example 3: Derivation Filters - Gradient - • Gradient representations NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 14/18
86. Linear Filtering (2) Example 4: Derivation Filters - The Laplacian - • The Laplacian is the simplest isotropic derivative operator. It is a scalar linear operator. 0 1 0 1 1 1 1 -4 1 1 -8 1 0 1 0 1 1 1 Isotropic with respect to rotations of 90 or 45 degrees. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 14/18
87. Linear Filtering (3) Example 5: Enhancement Filters - Laplacian - g = f − ∆f , 0 -1 0 -1 5 -1 0 -1 0 NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 15/18
88. Linear Filtering (3) Example 5: Enhancement Filters - Laplacian - NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 15/18
89. Linear Filtering (3) Example 5: Enhancement Filters - Laplacian - g = f + λ(f − fb ) where fb is a regularized version (median, gaussian, ...) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 15/18
90. Linear Filtering (3) Example 5: Enhancement Filters - Laplacian - NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 15/18
91. Linear Filtering (3) Example 5: Enhancement Filters - Laplacian - NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 15/18
92. Non-linear filtering NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 16/18
93. Non-linear filtering Non-linear filtering • Replaces each pixel with a value selected based on a criterion, e.g., statistical • Cannot be implemented as a convolution product • Reduces noise with less blurring NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
94. Non-linear filtering Example 1: Max Filter • Replaces each pixel with the maximum value of its neighbors • Useful for eliminating ”pepper” noise NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
95. Non-linear filtering Example 2: Min Filter • Replaces each pixel with the minimum value of its neighbors • Useful for eliminating ”salt” noise NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
96. Non-linear filtering Example 3: Median Filter • Replaces each pixel with the median value of its neighbors • Effective against salt-and-pepper noise NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
97. Non-linear filtering Example 3: Median Filter • Gaussian/Median Filter NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
98. Non-linear filtering Example 5: Mean Shift Filter • Treat each pixel as a data point in a high-dimensional space. • Iteratively shifting data points towards regions of high data density. • Shift each pixel towards regions of similar pixel intensities. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
99. Non-linear filtering Example 5: Mean Shift Filter • Treat each pixel as a data point in a high-dimensional space. • Iteratively shifting data points towards regions of high data density. • Shift each pixel towards regions of similar pixel intensities. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
100. Non-linear filtering Example 5: Mean Shift Filter • Treat each pixel as a data point in a high-dimensional space. • Iteratively shifting data points towards regions of high data density. • Shift each pixel towards regions of similar pixel intensities. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
101. Non-linear filtering Example 5: Mean Shift Filter • Treat each pixel as a data point in a high-dimensional space. • Iteratively shifting data points towards regions of high data density. • Shift each pixel towards regions of similar pixel intensities. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
102. Non-linear filtering Example 5: Mean Shift Filter • Treat each pixel as a data point in a high-dimensional space. • Iteratively shifting data points towards regions of high data density. • Shift each pixel towards regions of similar pixel intensities. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
103. Non-linear filtering Example 5: Mean Shift Filter • Mean Shift Vector: m(x) = xi ∈N(x) K(xi − x)xi xi ∈N(x) K(xi − x) − x • x is the current position, • N(x) is the neighborhood of x, • K is the kernel function (e.g., Gaussian kernel). • Update Step: xnew = x + m(x) NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
104. Non-linear filtering Example 5: Mean Shift Filter Noisy image NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
105. Non-linear filtering Example 5: Mean Shift Filter Median filtering NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
106. Non-linear filtering Example 5: Mean Shift Filter Mean shift filtering NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 17/18
107. Assignment 2 Write a python code that depends on the OpenCV library and that performs all the methods of this lecture: 1. Point transformations 2. Local Transformations 3. Global Transformations: AHE, CLAHE, Histogram Equalization, Histogram Specification 4. Linear Filtering: Mean, Gaussian, Gradient, Laplacian, with applications. 5. Non-linear filtering: Min, Max, Median, and Mean Shift Filters. Simulations should be conducted to evaluate (qualitative and quantitative) and compare the methods with various parameters. NHSM - 4th year: Digital Image Processing - Spatial Processing (Week 2,3) - M. Hachama (hachamam@gmail.com) 18/18