Learn the trick to using OpenCV groupRectangles() for multiple object detection. This is part 3 in the OpenCV Python tutorial for gaming.

Full tutorial playlist:    • OpenCV Object Detection in Games  

Grab the code on GitHub: https://github.com/learncodebygaming/...
OpenCV Group Rectangles documentation: https://docs.opencv.org/4.2.0/d5/d54/...

0:22 Review of the problem we're solving
1:11 Group rectangles documentation
1:52 Building rectangle list
4:55 Using groupRectangles()
7:17 The TRICK
8:44 Finding the click points
10:26 Wrapping everything we've learned into a convenient function

In this part of the OpenCV tutorial series, I'm going to show you how to take overlapping results from matchTemplate() and turn them into single object detections. We'll do this by using OpenCV's groupRectangles(), and there is a trick I'll show you for using this function successfully. We'll finish up this section by converting our rectangle results into positions where our mouse could click to select the detected object.

So when we use thresholding with matchTemplate() and look at our results visually, we can quickly see that we've detected some number of objects. In the example below, it looks like we've detected 12 cabbages. But when we check our detected locations list, you'll see that we have far more than 12 results. This is because we've found many matching results that are very close to one another. You can also see this visually in the result image as some box lines look thicker. These are actually many different detection results all overlapping each other.

We can solve this problem by using OpenCV's groupRectangles().

The groupRectangles() function expects a list of rectangles that are in the form of [x, y, width, height]. It will then return a new list of rectangles where the rectangles that are near to each other have been grouped together. So the first thing we must do to use this function is to convert our list of (x, y) location results into a list of rectangles.

for loc in locations:
rect = [int(loc[0]), int(loc[1]), needle_w, needle_h]
rectangles.append(rect)

Now that our rectangles list is constructed, we can call groupRectangles():

rectangles, weights = cv.groupRectangles(rectangles, groupThreshold=1, eps=0.5)

The groupThreshold parameter will almost always be 1. If you set it to 0 it's not going to group any rectangles at all. And if you set it to something higher, it's going to require that more rectangles are overlapping each other before creating a grouped result for them.

The eps parameter controls how close together the rectangles need to be before they will be grouped together. Lower values require that rectangles be closer together to be merged, while higher values will group together rectangles that are farther away. This is a good value to play around with to make sure you're getting the results you expect.

Finally, groupRectangles() returns both the new rectangles list and weight information about the grouping process (which we will ignore).

If you run this code, you should find that your overlapping detection results have been discarded. But depending on the threshold that you set, you might also notice that some of your detections have been lost. Why is that?

This is the trick to using groupRectangles() that I mentioned earlier. If you have a lone match result that doesn't have any other nearby or overlapping matches to it, groupRectangles() will discard that result. The best way I've found to correct this issue is to simply add every rectangle to our rectangles list twice. This will give you back your lost results.

Now that we have nice detection results, where each object is detected just once, we can easily convert these rectangles into positions at the center of each rectangle. These represent points we could click on to select the detected object.

Continue with the written tutorial here: https://learncodebygaming.com/blog/gr...