CV2 - rectangular detecting issue

Question

I'm trying to implement an OMR using python's CV2. As part of the code I need to find the corners of the choices box (rectangulars) however I ran into difficulty causes by the grade sheet template.

In the grade sheet template that I'm working on (image 1) there are two verticals black lines that goes through all the rectangular that I'm trying to detect. when I used cv2.findContours after applying all the blur and canny stuff, it recognized all the rectangular as two contours instead of 8 rectangular like I expected (image 2). If someone has an idea of how to get around it I will really appreciate it.

image 1 image 2

Answer 1

The merged contours are wonderfully detected. Now it suffices to traverse them and find the right angles. From this you can easily isolate the individual rectangles.

Answer 2

Here is one way to do that in Python/OpenCV.

• Read the input as grayscale
• Threshold on the white of the page
• Get the largest contour
• Crop the image to remove the dark gray borders
• Apply horizontal close morphology to find the 2 thick black lines
• Threshold on the extracted 2 black line image
• Erode the lines
• Make the two thick black lines similar color to the light gray background in the cropped image
• Apply vertical open morphology on the gray line enhanced cropped image to form dark blocks and invert
• Get contours
• Filter contours on area to keep only the larger blocks, get the bounding boxes dimensions and draw rectangles about each block
• Save results

Input:

import cv2
import numpy as np

# read the input as grayscale
img = cv2.imread('OMR.jpg', cv2.IMREAD_GRAYSCALE)

# threshold on white
thresh = cv2.threshold(img, 200, 255, cv2.THRESH_BINARY)[1]

# get contour of white page
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)

# get bounding box
x,y,w,h = cv2.boundingRect(big_contour)
bias = 50
# crop input to bounding box
crop = img[y+bias:y+h-bias, x+bias:x+w-bias]

# apply horizontal morphology to find the 2 thick black lines
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (200,1))
morph1 = cv2.morphologyEx(crop, cv2.MORPH_CLOSE, kernel)

# threshold on black to clean up morph1 result
thresh1 = cv2.threshold(morph1, 100, 255, cv2.THRESH_BINARY)[1]

# erode lines 
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (50,5))
morph2 = cv2.morphologyEx(thresh1, cv2.MORPH_ERODE, kernel)

# make lines gray in cropped image using thresh1 as mask
crop2 = crop.copy()
crop2[morph2==0] = 220

# apply vertical open morphology to form blocks'
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (7,30))
morph3 = cv2.morphologyEx(crop2, cv2.MORPH_OPEN, kernel)

# threshold and invert
thresh2 = cv2.threshold(morph3, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)[1]

# get large block contours
result = crop2.copy()
result = cv2.merge([result,result,result])
contours = cv2.findContours(thresh2 , cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
    area = cv2.contourArea(cntr)
    if area > 100*250:
        # get bounding box
        x,y,w,h = cv2.boundingRect(cntr)
        cv2.rectangle(result, (x,y), (x+w, y+h), (0,0,255), 2)

# save results
cv2.imwrite('OMR_crop.jpg', crop)
cv2.imwrite('OMR_morph1.jpg', morph1)
cv2.imwrite('OMR_crop2.jpg', crop2)
cv2.imwrite('OMR_result.jpg', result)


# show results
cv2.imshow('thresh', thresh)
cv2.imshow('crop', crop)
cv2.imshow('morph1', morph1)
cv2.imshow('thresh1', thresh1)
cv2.imshow('morph2', morph2)
cv2.imshow('crop2', crop2)
cv2.imshow('morph3', morph3)
cv2.imshow('thresh2', thresh2)
cv2.imshow('result', result)
cv2.waitKey(0)

Cropped Image:

Extract black lines via morphology:

Cropped image with black lines mostly removed:

Resulting blocks drawn on cropped image with lines removed: