How do I convert STL file to numpy array and format it to overlay with Dicom data?

Question

I have STL files of 3D objects and I have converted them into both mesh objects and numpy arrays using the numpy-stl package. I have also converted DCM data into numpy arrays with the pydicom package, but the numpy array versions of the STL files have a size (numpyarray.shape) of 4280, which doesn't matchup up with the 256*256 dimensions of the DCM data. Also, the data type for the numpy array of each STl file includes vectors and normal values, which I do not know how to resolve. Any insight into the dimensional mismatch or an explanation of the STL numpy array's data type would be greatly appreciated, and I have attached a picture of the DCM and STL data in numpy array form.

DCM to numpy array STL to numpy array

Answer 1

I'm going to recommend you take a slightly different approach here- rather than try to accomplish this all in numpy dealing with the raw dicom and STL data, convert the dicom into a PNG and then overlay the STL on top of it.

So first to create the DICOM PNG-

import numpy as np
import png
import pydicom

ds = pydicom.dcmread(path)

shape = ds.pixel_array.shape

# Convert to float to avoid overflow or underflow losses.
image_2d = ds.pixel_array.astype(float)

# Rescaling grey scale between 0-255
image_2d_scaled = (np.maximum(image_2d,0) / image_2d.max()) * 255.0

# Convert to uint
image_2d_scaled = np.uint8(image_2d_scaled)

# Write the PNG file
with open(destination, 'wb') as png_file:
    w = png.Writer(shape[1], shape[0], greyscale=True)
    w.write(png_file, image_2d_scaled)

I borrowed this code from a previous comment I made. One thing I want to explicitly call out is the conversion into a Float- looking at your example images it looks like you have integers, which means you're losing a lot of the data from the DICOM and it will make your output worse.

The next step is to get a flat version of the STL file that you can shove on top of your image. I'm not going to go in depth on that, and instead am going to recommend you use the numpy-stl library which has a "flatten" function for this purpose. The examples on that package should be really helpful for finishing this off.

Answer 2

If you have ever done 3D printing you will know you want to slice your STL object into slices, as Robert Hafner suggests.

First you need to make sure your STL object is aligned with your Dicom volume, this may or may not be difficult, but necessary. Then slice the STL into 256 slices, as if you are going to 3D printing it, output the slices as 256 x 256 images, now you get 256 images of 256x256 pixels. Since it's aligned before your slicing, you may use these images as your masks, threshold them if necessary.