Applying Computer Vision Techniques for Facial Image Processing with Python and OpenCV

In this blog post, I will be walking you through a project I worked on during my Artificial Intelligence & Machine Learning Master's degree program at Colorado State University. The project's objective was to develop a Python script that processes facial images, performing several operations such as grayscale conversion, face and landmark detection, image rotation, illumination adjustment, cropping, and scaling. This project is a great example of how computer vision techniques can be applied to real-world problems.

Project Overview

The project uses several Python libraries, including OpenCV for image processing, MTCNN for face and landmark detection, NumPy for numerical operations, and Matplotlib for image visualization. The script processes two images, saves the processed images, and displays them using Matplotlib.

Step-by-step Walkthrough

1. Loading the Image

The first step in the process is to load the image. We use the imread function from OpenCV, which loads an image from the specified file.

def load_image(image_path):
    return cv2.imread(image_path)

2. Converting to Grayscale

Next, we convert the image to grayscale using the cvtColor function from OpenCV. This function converts an input image from one color space to another. In our case, we convert from BGR (Blue, Green, Red) to grayscale.

def convert_to_grayscale(image):
    return cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

3. Detecting Faces and Landmarks

We then use the MTCNN (Multi-task Cascaded Convolutional Networks) library to detect faces and facial landmarks in the grayscale image. The detect_faces function returns a list of detected faces, with each face represented as a dictionary containing the bounding box and facial landmarks (left eye, right eye, nose, mouth).

def detect_faces_and_landmarks(gray_image):
    detector = MTCNN()
    faces = detector.detect_faces(gray_image)
    return faces

4. Computing the Angle for Rotation

We compute the angle between the horizontal axis and the line connecting the left and right eyes using the np.arctan2 function. This angle is used to correct the orientation of the face.

def compute_angle(face):
    left_eye = face['keypoints']['left_eye']
    right_eye = face['keypoints']['right_eye']
    dX = right_eye[0] - left_eye[0]
    dY = right_eye[1] - left_eye[1]
    angle = np.degrees(np.arctan2(dY, dX))
    return angle

5. Rotating the Image

We rotate the image using the computed angle to align the face correctly. The getRotationMatrix2D function from OpenCV is used to get the rotation matrix, which is then applied to the image using the warpAffine function.

def rotate_image(gray_image, angle):
    h, w = gray_image.shape[:2]
    center = (w // 2, h // 2)
    M = cv2.getRotationMatrix2D(center, angle, 1.0)
    return cv2.warpAffine(gray_image, M, (w, h), flags=cv2.INTER_CUBIC)

6. Adjusting Illumination

We adjust the illumination of the image using the equalizeHist function from OpenCV, which equalizes the histogram of a grayscale image, thereby improving its contrast.

def adjust_illumination(image):
    return cv2.equalizeHist(image)

7. Cropping and Scaling

We crop the image around the detected face and resize it to a desired size using the resize function from OpenCV.

def crop_and_scale(gray_image, face, desired_size=160):
    x, y, w, h = face['box']
    cropped_image = gray_image[y:y+h, x:x+w]
    return cv2.resize(cropped_image, (desired_size, desired_size))

8. Drawing a Bounding Box

We draw a bounding box around the detected face using the rectangle function from OpenCV.

def draw_bounding_box(image, face, desired_size=160, color=(255, 0, 0), thickness=2):
    x, y, w, h = face['box']
    new_x = max(x - (desired_size - w) // 2, 0)
    new_y = max(y - (desired_size - h) // 2, 0)
    cv2.rectangle(image, (new_x, new_y), (new_x + desired_size, new_y + desired_size), color, thickness)

9. Saving the Processed Image

Finally, we save the processed image using the imwrite function from OpenCV.

def save_processed_image(image, output_path):
    cv2.imwrite(output_path, image)

Conclusion

This project demonstrates the power of Python and OpenCV for image processing tasks. The script can be easily extended to process multiple images, apply different transformations, or use other face detection algorithms. I hope you found this walkthrough helpful and that it inspires you to explore the exciting field of computer vision further.