-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathimage processing with python
249 lines (208 loc) · 19 KB
/
image processing with python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
# 3-Image restoration, Noise, Segmentation and Contours
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2-Noise:
How can an image have noise? Well images are signals and real-world signals usually contain departures from the ideal signal, which is the perfect image, as we observe with our eyes in real life. Such departures are referred to as noise.More specifically, noise is the result of errors in the image acquisition process that result in pixel values that do not reflect the true intensities of the real scene.
--------------------------------
-Apply noise in scikit-image:
>>from skimage.util import random_noise
>>noisy_image = random_noise(image)
-------------------------------
By using the random_noise function, we obtain the original image with a lot of added noise, that is distributed randomly. This type of noise is known as "salt and pepper" because that's in fact what it looks like.
-Reducing noise:
Most of the times we will want to remove or reduce the noise of images instead of adding it in.For that, we can use several algorithms in scikit-image.
"The higher the resolution of the image, the longer it may take to eliminate the noise"
Some types of denoising algorithms are:
1- The total variation filter(TV): This filter tries to minimize the total variation of the image. It tends to produce “cartoon-like” images, that is, piecewise-constant images
2- Bilateral filtering: smooths images while preserving edges. It replaces the intensity of each pixel with a weighted average of intensity values from nearby pixels.
3- The wavelet denoising filter
4- Non-local means denoising.
----------------------------------------
Apply total variation filter(TV) in scikit-image:
>>from skimage.restoration import denoise_tv_chambolle
>>denoised_image = denoise_tv_chambolle(noisy_image, weight =.1 , multichannel = True)
# we can optionally set the denoising weight. The greater the weight, the more denoising but it could also make the image smoother.
------------------------------------------
Apply Bilateral filter denoising in scikit-image:
>>from skimage.restoration import denoise_bilateral
>>denoised_image = denoise_bilateral(noisy_image, multichannel = True)
#The resulting image is less smooth than the one from the total variation filter. And preserves the edges a lot more.
------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
3-Superpixels & segmentation:
Segmentation: The goal is to partition images into regions, or segments, to simplify and/or change the representation into something more meaningful and easier to analyze.
For example, before a tumor is analyzed in a computed tomography, it has to be detected and somehow isolated from the rest of the image. Or before recognizing a face, it has to also be picked out from its background.
Previously we learned about Thresholding, which is the simplest method of segmentation. Separating foreground from background.
Image representation: Consider how we represent images. They are represented as a grid of pixels.
The issue is that they're not a completely natural representation of an image.
If I were to take a single pixel from the image on the left and then showed it to you on the right,
would you be able to tell that the pixel came from a puppy and that this single pixel holds a logical meaning?
A single pixel, standing alone by itself, is not a natural representation.
So, we can explore more logical meanings in an image that's formed by bigger regions or grouped pixels. These are known as superpixels.
superpixel: is a group of connected pixels with similar colors or gray levels. These carry more meaning than their simple pixel grid counterparts.
Superpixel segmentation is dividing an image into superpixels. It has been applied to many computer vision tasks, like visual tracking and image classification.
Some advantages for using them are that You can compute features on more meaningful regions. And that you can reduce an image from thousands of pixels down
to some regions for subsequent algorithms, so you have computational efficiency.
Segmentation:
Two types of segmentation are:
1-Supervised, where some prior knowledge is used to guide the algorithm. Like the kind of thresholding in which we specify the threshold value ourselves.
2-unsupervised where no prior knowledge is required. These algorithms try to subdivide images into meaningful regions automatically. The user may still be able to tweak certain settings to obtain the desired output.
Like the otsu thresholding we used in first chapter.
Unsupervised segmentation technique: based on superpixels, called Simple Linear Iterative Clustering or SLIC.
It segments the image using a machine learning algorithm called K-Means clustering.
It takes in all the pixel values of the image and tries to separate them into a predefined number of sub-regions.
--------------------------------------------------
Apply unsupervised segmentation (slic)
>>from skimage.segmentation import slic
>>from skimage.color import label2rgb
#obtain segments
>>segments =slic(image)
#put segments on top of original image to compare
>>segmented_image = label2rgb(segments, image , kind = 'avg')
#Here we use this function, with default parameters and obtain the segments.
We'll use the label2rgb method from the color module to return an image where the segments obtained from the slic method will be highlighted,
either with random colors or with the average color of the superpixel segment. In this example we'll use the average color. So we pass the segments or labels,
the image, and set the kind parameter to average avg.
#If we want more segments, let's say 300, we can specify this with an optional parameter, n_segments. Its default value is 100 segments.
>>segments =slic(image, n_segments = 300)
---------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
4-Finding contours:
Contour: is a closed shape of points or line segments, representing the boundaries of these objects. Once we find the contours,
we can do things like identifying the total points in domino tokens, we can measure size, classify shapes or determining the number of objects in an image.
Binary images: The input to a contour-finding function should be a binary image, which we can produce by first applying thresholding.
In such binary image, the objects we wish to detect should be white, while the background remains black.
Find contours using scikit-image:First, the image needs to go through some pre-processing steps. We need to turn the image to grayscale to later apply thresholding.
We can do so by using the rgb2gray function from the color module.
To use the find_contours function, we need the image we want to obtain contours from, to be binary. Meaning,
black and white. Here we apply thresholding to do that, so we get a thresholded image.
-----------------------------------------------------
#Find contours
>>from skimage.color import rgb2gray
>>from skimage import filters
>>from skimage import measure
>>image_dice = color.rgb2gray(image_dice)
>>thresh = filters.threshold_otsu(image_dice)
>>binary = image_dice > thresh
>>contours = measure.find_contours(binary, .8)
#This function finds the contour lines or joins points(pixels) of equal elevation or brightness in a 2D array above a given level value.
We import the module "measure" from skimage. And from measure we call the function find_contours.
The function returns a list with all contours of the image. With the coordinates along the contour.
Passing the thresholded image as the first parameter and a constant level value of zero point eight.
The function returns a list with all contours of the image. With the coordinates along the contour.
------------------------------------------------------
Constant level value: The level value varies between 0 and 1, the closer to 1 the more sensitive the method is to detecting contours,
so more complex contours will be detected. We have to find the value that best detects the contours we care for.
A contour's shape:After executing these steps we obtain a list of contours. Each contour is an ndarray of shape (n, 2),
consisting of n row and column coordinates along the contour. In this way, a contour is like an outline formed by multiple points joined together.
The bigger the contour, the more points joined together and the wider the perimeter formed.
------------------------------------------------------
Count the dots in a dice's image:
# Create list with the shape of each contour
shape_contours = [cnt.shape[0] for cnt in contours]
# Set 50 as the maximum size of the dots shape
max_dots_shape = 50
# Count dots in contours excluding bigger than dots size
dots_contours = [cnt for cnt in contours if np.shape(cnt)[0] <max_dots_shape]
# Shows all contours found
show_image_contour(binary, contours)
# Print the dice's number
print("Dice's dots number: {}. ".format(len(dots_contours)))
--------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
# 4-Advanced Operations, Detecting Faces and Features:
--------------------------------------------------------------------------------------------------------------------------------------------------------------
1-Finding the edges with Canny:
Edge detection: is extensively used when we want to divide the image into areas corresponding to different objects.
Most of the shape information of an image is enclosed in edges.Representing an image by its edges has
the advantage that the amount of data is reduced significantly while retaining most of the image information, like the shapes.
- In the previous chapter you have seen how to detect edges using the Sobel filtering technique.
the Canny edge detection: one of the most used edge detection techniques This is widely considered to be the standard edge detection method
in image processing. And produces higher accuracy detecting edges and less execution time compared with Sobel algorithm.
-------------------------------------------------------
#Edge detection
>>from skimage.color import rgb2gray
>>from skimage.feature import canny
>>coins = rgb2gray(coins)
>>canny_edges = canny(coins)
#This function requires the image to be a 2-dimensional array, meaning, a grayscale image. So in this example,
we convert the image from RGB-3 to grayscale, using the rgb2gray method from the color module that we already know from previous chapters.
Then we apply the canny detector on the coin's image and obtain the resulting image. Let's show it and see what it looks like.
We see how the edges are highlighted with thick white lines and that some details are more pronounced than the rest of the image.
We can also spot the boundaries and shapes of coins; by knowing that for each closed circle or ellipse, there's a coin.
-----------------------------------------------------------
-----------------------------------------------------------
#apply canny detector
>>from skimage.color import rgb2gray
>>from skimage.feature import canny
>>coins = rgb2gray(coins)
>>canny_edges = canny(coins, sigma=.5)
#The first step of this algorithm is to apply a gaussian filter in order to remove noise in the image.
The same gaussian filter we have seen previously in the course with the gaussian function from the filters module.
So, in the canny function you can optionally set the intensity of this Gaussian filter to be applied in the image, by using the sigma attribute.
The lower the value of this sigma, the less of gaussian filter effect is applied on the image, so it will spot more edges.
On the other hand, if you set a higher value, more noise will be removed and the result is going to be a less edgy image. The default value of this parameter is 1.
The resulting image has a lot more edges than the previous one and this is because noise was removed before continuing with the rest of the steps in the algorithm.
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2-Right around the corner:
Corner detection: is an approach used to extract certain types of features and infer the contents of an image. It's frequently used in motion detection,
image registration, video tracking, panorama stitching, 3D modelling, and object recognition. We saw in the previous video how to detect
edges with the Canny Edge detector, and before that with Sobel, in chapter 2. Edges are a type of feature in images.
Points of interest: Features are the points of interest which provide rich image content information.
Points of interest are points in the image which are invariant to rotation, translation, intensity, and scale changes.
(Basically, robust and reliable). There are different interest points such as corners and edges.
So corner detection is basically detecting (one type of) interest points in an image.
Corner: can be defined as the intersection of two edges. Intuitively, it can also be a junction of contours.
So by detecting corners as interest points, we can match objects from different perspectives.
Like in this image, where we detect the corners of the original image on the left and then match them in a downscaled image in the right.
Here is another example of corner matching, this time, in a rotated image. We see how the relevant points. are still being matched.
Harris Corner Detector: is a corner detection operator that is widely used in computer vision algorithms
We can access it by importing the corner_harris function from the feature module of scikit-image.
This function requires grayscale images, so we need to first convert the image from rgb to gray.
We can do this with the rgb2gray function we used previously.
This corner_harris function gives us the Harris measure response image, meaning,
the resulting image showing only the possible corners that were measured. So when we show it,
We see how only some black lines are shown. These are the approximated points where the corners candidates are.
To find the corners in the measure response image, we can use the corner_peaks function. Which will return the coordinates of the peaks
of the possible corners. Optionally, we can make sure these peak corners are separated by a minimum distance, in pixels,
using the min_distance parameter.
--------------------------------------------------
#Apply corner harris
>>from skimage.feature import corner_harris
>>image = rgb2gray(image)
>>measure_image = corner_harris(image)
#the coordinates of the peaks of the possible corners
>>
>>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
#3-Face detection:
Several social networks platforms and smart-phones are using face detection to know if there is someone in a picture and if so, apply filters,
add focus in the face area, or recommend you to tag friends. You can even automatically blur faces for privacy protection.
Face detection can be useful in other cases as well. Human faces are able to convey many different emotions such as happiness, sadness and many others.
That's why face detection is the key first step before recognizing emotions.
Detecting faces with scikit-image: With scikit-image, we can detect faces using a machine learning classifier,
with just a couple of lines! In this course, we won't cover machine learning concepts in depth,
but it's important to know that we use a cascade of classifiers, which is like multiple classifiers in one.
You can also use it for other things, like cats, objects, or profile faces, from a side view.
---------------------------------------------------------
-Apply detector
>>from skimage.feature import Cascade
>>trained_file = data.lbp_frontal_face_cascade_filename()
>>detector = cascade(trained_file)
--------------------------------------------------------
Detecting faces: To apply the detector on images, we need to use the detect_multi_scale method, from the same cascade class.
This method searches for the object, in this case a face. It creates a window that will be moving through the image until
it finds something similar to a human face.Searching happens on multiple scales. The window will have a minimum size, to spot the small or far-away faces.
And a maximum size to also find the larger faces in the image.
---------------------------------------------------------
>>detected = detecor.detect_multi_scale(img = image , scale_factor = 1, scale_ratio = 1.2, min_size =(10,10), max_size =(200,200))
>>print(detected)
#detected_face :[{'r' = 115 , 'c'= 210, width = 167 , height = 167}]
#So this method takes the input image as the first parameter, a scale factor, by which the searching window is multiplied in each step, a step ratio,
in which 1 represents an exhaustive search and usually is slow. By setting this parameter to higher
values the results will be worse but the computation will be much faster. Usually, values in the interval 1 to 1.5 give good results.
Then the minimum and maximum window size are defined. These specify the interval for the search windows that are applied to the input image to detect the faces.
The detector will return the coordinates of the box that contains the face. When printing the result, we see that it's a dictionary,
where r represents the row position of the top left corner of the detected window, c is the column position pixel, width is width of detected window,
and height, the height of the detected window.
----------------------------------------------------------
----------------------------------------------------------------------------------------------------------------------------------------------------------------