Edge Detection in Ultrasound Images (Video) Using FFT and Canny Edge Detection

Question

`[Code/Concept Explanation] I've been working with ultrasound imaging probe footage, breaking it down frame by frame and converting each frame into the frequency domain. Following that, I've extracted brightness information in the frequency domain and retained only the brightest values. Subsequently, I've applied inverse FFT to these values, and finally, I've used the Canny Edge detection algorithm to achieve the desired outcome.

Here my code!

# Crop the frame to the desired size
cropped_frame = frame[y1:y2, x1:x2]

# Extract brightness information in the frequency domain
brightness_spectrum = extract_brightness_spectrum(cropped_frame)

# Rotate the frame by 90 degrees
rotated_frame = cv2.rotate(cropped_frame, cv2.ROTATE_90_CLOCKWISE)

# Extract brightness information in the frequency domain (rotated image)
brightness_spectrum_rotated = extract_brightness_spectrum(rotated_frame)






# Function: Extract the brightest value from the frequency spectrum

def keep_brightest_value(spectrum): max_value = np.max(spectrum) brightest_spectrum = np.zeros_like(spectrum) brightest_spectrum[spectrum == max_value] = max_value return brightest_spectrum

import cv2
import numpy as np
from scipy.fft import fft2, ifft2

# Open the video file
video_capture = cv2.VideoCapture('path_to_video/input.mp4')

# Handle if the video file fails to open
if not video_capture.isOpened():
    print("Unable to open the video file.")
    exit()

# Define the desired cropping coordinates and size
x1, y1, x2, y2 = 558, 120, 1183, 642  # Example coordinates (modify to your desired coordinates)

# Set output video parameters
frame_width = int(x2 - x1)
frame_height = int(y2 - y1)
output_video = cv2.VideoWriter('desired_path/output_video2.mp4',
                               cv2.VideoWriter_fourcc(*'mp4v'), 30, (x2 - x1, y2 - y1), isColor=False)

# Function: Extract brightness information in the frequency domain
def extract_brightness_spectrum(frame):
    frame_fft = fft2(frame)
    brightness_spectrum = np.abs(frame_fft)
    return brightness_spectrum

# Function: Extract the brightest value from the frequency spectrum
def keep_brightest_value(spectrum):
    max_value = np.max(spectrum)
    brightest_spectrum = np.zeros_like(spectrum)
    brightest_spectrum[spectrum == max_value] = max_value
    return brightest_spectrum

# Read and process frames from the video
while True:
    ret, frame = video_capture.read()
    if not ret:
        break

    # Crop the frame to the desired size
    cropped_frame = frame[y1:y2, x1:x2]

    # Extract brightness information in the frequency domain
    brightness_spectrum = extract_brightness_spectrum(cropped_frame)

    # Rotate the frame by 90 degrees
    rotated_frame = cv2.rotate(cropped_frame, cv2.ROTATE_90_CLOCKWISE)

    # Extract brightness information in the frequency domain (rotated image)
    brightness_spectrum_rotated = extract_brightness_spectrum(rotated_frame)

    # Match the shapes of brightness_spectrum and brightness_spectrum_rotated
    min_height = min(brightness_spectrum.shape[0], brightness_spectrum_rotated.shape[0])
    min_width = min(brightness_spectrum.shape[1], brightness_spectrum_rotated.shape[1])
    brightness_spectrum = brightness_spectrum[:min_height, :min_width]
    brightness_spectrum_rotated = brightness_spectrum_rotated[:min_height, :min_width]

    # Extract the brightest value from the frequency spectrum
    brightest_spectrum = keep_brightest_value(brightness_spectrum)

    # Extract the brightest value from the frequency spectrum (rotated image)
    brightest_spectrum_rotated = keep_brightest_value(brightness_spectrum_rotated)

    # Rotate the 90-degree rotated frame back to the original orientation
    rotated_frame_original = cv2.rotate(rotated_frame, cv2.ROTATE_90_COUNTERCLOCKWISE)

    # Combine the brightness information from both frames
    combined_spectrum = brightest_spectrum + brightest_spectrum_rotated

    # Apply inverse FFT to obtain the filtered frame (combined image)
    filtered_frame_combined = np.abs(ifft2(combined_spectrum)).astype(np.uint8)

    # Apply Canny edge detection
    edges = cv2.Canny(filtered_frame_combined, threshold1=30, threshold2=255)

    # Write the combined frame to the output video
    output_video.write(filtered_frame_combined)

    # Display the processed frame in a window
    cv2.imshow('Combined Video', filtered_frame_combined)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Close the video files and windows
video_capture.release()
output_video.release()
cv2.destroyAllWindows()

Some frame images from actual ultrasound probe footage

I initially anticipated that the final result would consist of only the white band visible in the picture, detectable through the Canny Edge algorithm. However, the actual results turned out differently, as shown in the image below:

The coding result

Considering this, I thought about a potential solution. Since I performed FFT processing only at one angle, I considered rotating the frame by 90 degrees and applying the same operations. Specifically, I aimed to extract brightness information after rotating the frame by 90 degrees and subsequently extracting the brightest values.

I'm unsure if my understanding is flawed or if there's something incorrect in the code, but the results haven't aligned with my expectations. My objective is to eliminate the straight line and extract only the white band from the original frame, but I'm currently facing some challenges.

Any tips or guidance you can provide would be greatly appreciated. `