Mouth detection not working in certain cases

Question

I'm currently working on a function in Python that aims to detect mouths in images using OpenCV cascades. Here's the code snippet:

def detect_mouth(img):
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, minNeighbors=7)
    for (x, y, w, h) in faces:
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = img[y:y+h, x:x+w]
        mouth = mouth_cascade.detectMultiScale(roi_gray, minNeighbors=10)
        
        if len(mouth) > 1:
            mouth = mouth_cascade.detectMultiScale(roi_gray, minNeighbors=100)
        elif len(mouth) < 1:
            mouth = mouth_cascade.detectMultiScale(roi_gray, minNeighbors=7)
        
        for (mx, my, mw, mh) in mouth:
            cv2.rectangle(roi_color, (mx, my), (mx+mw, my+mh), (255, 0, 255), 2)
            
    return img

It appears that the mouth detection is not consistent across different ethnicities. When testing with images of white women, the mouth is detected properly, as shown in

However, when testing with images of black women, the mouth is not detected in certain instances, as shown in

The mouth is not being detected and instead shows up as the eye! Interestingly, the mouth is successfully detected in some other cases like this one:

I've experimented with adjusting the minNeighbors parameter, but it hasn't improved the results. Is this related to specific skin tones, or is there something else in the setup that needs tweaking?

Answer 1

However, when testing with second images of black women, the mouth is not detected in certain instances, as shown in second image.

The problem can be fixed.

• Reduced index from 7 to 1. Noted, you will have to change depending on lightning condition or make-up face.

Snippet:

mouth = mouth_cascade.detectMultiScale(roi_gray, minNeighbors=7)

To:

mouth = mouth_cascade.detectMultiScale(roi_gray, minNeighbors=1)

Noticed, It will work with any type of darker black women and asian women. By using odd index and not even index.

screenshot: