How do I add dependencies that can't be installed with pip to my (django) project on Heroku?

Question

I've come across multiple similar issues to this and I haven't seen a clear answer so I thought it best to ask here. I'm working on a django project, and one of the packages it uses requires PyMuPDF which in turn requires swig, which can't be installed with pip/add to requirements.txt. I'm trying to deploy this project to Heroku and this is causing it to fail.

See the build error from Heroku below.

             PyMuPDF/setup.py: Finished building mupdf.
             PyMuPDF/setup.py: library_dirs=['mupdf-1.20.0-source/build/release']
             PyMuPDF/setup.py: libraries=['mupdf', 'mupdf-third', 'jbig2dec', 'openjp2', 'jpeg', 'freetype', 'gumbo', 'harfbuzz', 'png16', 'mujs']
             PyMuPDF/setup.py: include_dirs=['mupdf-1.20.0-source/include', 'mupdf-1.20.0-source/include/mupdf', 'mupdf-1.20.0-source/thirdparty/freetype/include', '/usr/include/freetype2']
             PyMuPDF/setup.py: extra_link_args=[]
             running bdist_wheel
             running build
             running build_py
             running build_ext
             building 'fitz._fitz' extension
             swigging fitz/fitz.i to fitz/fitz_wrap.c
             swig -python -o fitz/fitz_wrap.c fitz/fitz.i
             error: command 'swig' failed: No such file or directory
             [end of output]
         
         note: This error originates from a subprocess, and is likely not a problem with pip.
         ERROR: Failed building wheel for PyMuPDF
         Running setup.py clean for PyMuPDF
       Successfully built docopt fire langdetect mplcursors
       Failed to build PyMuPDF
       ERROR: Could not build wheels for PyMuPDF, which is required to install pyproject.toml-based projects
 !     Push rejected, failed to compile Python app.
 !     Push failed

I had a similar situation where a package required libffi and cairo which can only be brew install-ed. (I moved away from that package since it also required tensor which is too large from Heroku.

How can I solve this situation? Is there a way to solve this and get these dependencies on Heroku? I tried using a buildpack (I'm not too familiar with) but it didn't work.

Or is there an alternative to Heroku that I can use for this? (I'm a little familiar with Docker but I have never deployed any with Docker)