Estimate homography from ellipse parameters and 2 lines

Question

I want to estimate homography between 2 images:

import numpy as np
import skimage as ski
import matplotlib.pyplot as plt

my_image = np.zeros(shape=(400, 400))
rr, cc = ski.draw.ellipse_perimeter(150, 150, r_radius=45, c_radius=30, shape=my_image.shape)
my_image[rr, cc] = 1

rr, cc = ski.draw.line(50, 220, 300, 220)
my_image[rr, cc] = 1

rr, cc = ski.draw.line(60, 100, 60, 300)
my_image[rr, cc] = 1

plt.imshow(my_image, 'gray')
plt.show()

H = np.array([[1, -0.5, 100],
                   [0.15, 0.6, 50],
                   [0.0005, 0.0007, 1]])

proj = ski.transform.ProjectiveTransform(matrix=H)
proj_image = ski.transform.warp(my_image, proj.inverse, output_shape=my_image.shape)

plt.imshow(proj_image, 'gray')
plt.show()

Instead of pair of points I have the parameterization of the ellipse on the 2 images and the parameterization of the 2 lines on both images.

How can I estimate the homography from the data?

I couldn't find any model for abstract objects like lines and ellipses to estimate homography.
No function on opencv or scikit-image.