I have a high-performant function written in Julia, how can I use it from Python?

Question

I have a found a Julia function that nicely does the job I need. How can I quickly integrate it to be able to call it from Python?

Suppose the function is

f(x,y) = 2x.+y

What is the best and most elegent way to use it from Python?

Answer 1

Assuming your Python and Julia are installed you need to take the following steps.

1. Run Julia and install PyCall

   using Pkg
   pkg"add PyCall"
   

2. Put your code into a Julia package

   using Pkg
   Pkg.generate("MyPackage")
   

   In the folder src you will find MyPackage.jl, edit it to look like this:

   module MyPackage
   f(x,y) = 2x.+y
   export f
   end
   

3. Install pyjulia

   python -m pip install julia
   

   (On Linux systems you might want to use python3 instead of python command)

   For this step note that while an external Python can be used with Julia. However, for a convenience it might be worth to consider using a Python that got installed together with Julia as PyCall. In that case for installation use a command such this one:

   %HOMEPATH%\.julia\conda\3\python -m pip install julia
   

   or on Linux

   ~/.julia/conda/3/python -m pip install julia
   

   Note that if you have JULIA_DEPOT_PATH variable defined you can replace %HOMEPATH%\.julia or ~/.julia/ with its value.

4. Run the appropiate Python and tell it to configure the Python-Julia integration:

   import julia
   julia.install()
   

5. Now you are ready to call your Julia code:

   >>> from julia import Pkg
   >>> Pkg.activate(".\\MyPackage") #use the correct path
       Activating environment at `MyPackage\Project.toml`
   >>> from julia import MyPackage
   >>> MyPackage.f([1,2],5)
       [7,9]
   

It is worth noting that the proposed approach in this answer has several advantages over a standalone Julia file which would be possible, although is less recommended. The advantages include:

1. Packages get precompiled (so they are faster in subsequent runs) and can be loaded as a package in Python.
2. Packages come with their own virtual environment via 1Project.toml` which makes production deployments much comfortable.
3. A Julia package can be statically compiled into Julia's system image which can slash itsloading time --- see https://github.com/JuliaLang/PackageCompiler.jl .

EDIT

In February 2022 juliacall was announced, as of December 2022 juliacall might be an easier option for some users - have a look at: How to load a custom Julia package in Python using Python's juliacall

Answer 2

I found that the python package PyCall did not work for me (segaults in unexplicable ways) however the more recent library JuliaCall worked. The differences are listed in the above link to the official pip-package. There is also an example on how to get started but I will describe the process here as well.

Advantages of using juliacall

The advantages of using juliacall over pycall are

• it works more reliably for me (python 3.9/3.10, julia 1.6)
• I can use my existing Julia environment (in production one might want to create a seperate julia-env which is also possible, see the default way in the docs)
• just need a single julia script file (dont need to create a package)
• Furthermore the documentation for juliacall is better: https://cjdoris.github.io/PythonCall.jl/stable/juliacall/

---

Instructions

1. Install PythonCall into Julia either by entering package mode via ] and writing add PythonCall or execute in normal julia mode:

julia> using Pkg
julia> Pkg.add("PythonCall")

2. Install the python package juliacall into you python environment:

$ python -m pip install juliacall

3. note the path of your julia executable. Enter the shell mode in julia with the key ; and execute:

shell> which julia
/home/user/.juliaup/bin/julia

4. note the path to your julia environment by first importing Pkg and then entering shell mode again to execute:

julia> using Pkg
shell> pwd $(Pkg.envdir())
/home/user/.julia/environments
shell> ls -lah $(Pkg.envdir())
v1.6

So my julia-env path is: /home/user/.julia/environments/v1.6/

5. Execute the following python script:

import os
# setting env variables so that juliacall finds the correct julia env
# see the github page for docs about env variables https://github.com/cjdoris/PyJuliaPkg
# juliacall makes use of the package pyjuliapkg to find the correct julia version, so you can read up on that here: https://github.com/cjdoris/pyjuliapkg/blob/main/src/juliapkg/find_julia.py
os.environ['PYTHON_JULIAPKG_EXE'] = "/home/user/.juliaup/bin/julia"
os.environ['PYTHON_JULIAPKG_OFFLINE']= 'yes' # such that nothing new is downloaded
os.environ['PYTHON_JULIAPKG_PROJECT'] = '/home/user/.julia/environments/v1.6/'

from juliacall import Main as jl


def main():
    print('in main...')
    
    # show installed julia packages to make sure you are in the right env 
    jl.seval('using Pkg')
    jl.seval('Pkg.status()') 

    # import your script and execute
    jl.seval('include("test_julia_simple.jl")')
    result = jl.seval('f([2,3], [4,5])')
    print(result)

    print('...done')
    return


if __name__ == "__main__":
    main()

The accompanying julia script test_julia_simple.jl contains your function and in this case looks as follows:

function f(x,y)
    return 2x.+y
end