2-features-in-opencv.md

Step 2: Features in OpenCV

There are several feature methods available in the features2d and xfeatures2d modules in OpenCV. Some of these are detectors, some compute descriptors and some do both. They all extend the abstract cv::Feature2D base class.

First, please take a look at the documentation above to get an overview. Then, lets try it out!

How to construct

In C++ you construct detectors and descriptor extractors by calling their static create() factory method. In Python, this method becomes <feature class>_create(). You can also typically set different parameters using this method (read the documentation).

The following example will create a FAST feature detector and a LATCH descriptor extractor:

detector = cv2.FastFeatureDetector_create()
desc_extractor = cv2.xfeatures2d.LATCH_create()

In order to use the descriptors for matching, we will also need a matcher. The following example constructs a brute force matcher based on the default metric used by the descriptor:

matcher = cv2.BFMatcher_create(desc_extractor.defaultNorm())

How to use

We detect keypoints by calling the detector's detect() method.

keypoints = detector.detect(gray_frame)

The detected keypoints are returned as a tuple of cv2.KeyPoints.

Take a look at cv::KeyPoint. What fields does it contain?

We compute descriptors for each keypoint by calling the descriptor extractor's compute() method:

curr_keypoints, frame_descriptors = desc_extractor.compute(gray_frame, curr_keypoints)

This will return the descriptors (as matrices) and possibly updated keypoints (see compute()).

We will use the matcher to match descriptors between the current frame and a reference frame. In order to apply the ratio test (from the lectures), we will need to extract the two best matches for each keypoint. We can do this with the matcher's knnMatch() method:

matches = matcher.knnMatch(frame_descriptors, ref_descriptors, k=2)

The result is returned as a tuple of tuples, each with two cv2.DMatch objects (the best and the second best matches).

Take a look at cv::DMatch. What fields does it contain?

run_mosaic_lab()

Now, take look at run_mosaic_lab() in lab_mosaic.py, and find where each of the steps above are performed in the code. This is a pretty advanced program, so ask the instructors if you have trouble understanding what is going on in this function.

Then, please continue to the next step.