How to solve a Weighting Problem with pyspark

Question

I have matrix A of dimension [nXm] like

w1 w2 0
w3 0  w4
0  w5 0

a vector b with known value of size [mX1]:

10
 5
 3

and a vector c of size [nX1]

0
0
0

I want now to find on solution (finding values of w1, ..., w5) that fulfills the Equation:

A*b = c

using pyspark.

Answer 1

Since your set of equations are underdetermined. i.e. number of unknown variables are more than the number of equations, you will get parametrized solutions.

You can solve system of linear equations (underdetermined, overdetermined or unique) using sympy library.

I have adapted the following stackoverflow solution to given an example of how you can solve your equation.

https://stackoverflow.com/a/50048060/3238085

from sympy import *
from sympy import linear_eq_to_matrix, symbols, simplify, sin, cos, Eq, pprint


"""
w1 w2 0
w3 0  w4
0  w5 0
"""

b1, b2, b3 = 10, 5, 3  # known coefficients, they could be symbols too
c1, c2, c3 = 0, 0, 0  # known coefficients, they could be symbols too
w1, w2, w3, w4, w5 = symbols('w1:6')

eq1 = Eq(w1*b1 + w2*b2 + 0*b3, c1)  # lhs, rhs
eq2 = Eq(w3*b1 + 0*b2 + w4*b3, c2)  # lhs, rhs
eq3 = Eq(0*b1 + w5*b2 + 0*b3, c3)  # lhs, rhs

eqns = [eq1, eq2, eq3]
solution = linsolve(eqns, [w1, w2, w3, w4, w5])

print("Solution in symbolic form")
print(solution)

substituted = solution.subs({w2: 1, w4: 1})
print("Solution with independent variables substituted")
print(substituted)

Output :

Solution in symbolic form
{(-w2/2, w2, -3*w4/10, w4, 0)}
Solution with independent variables substituted
{(-1/2, 1, -3/10, 1, 0)} ## corresponding to {w1, w2, w3, w4, w5}

Verification :

eq1 = -1/2*10 + 1*5 + 0 (lhs) = 0 = c1 (rhs)
eq2 = -3/10*10 + 0 + 1*3 (lhs) = 0 = c2 (rhs)
eq3 = 0 + 0 * 5 + 0 (lhs) = 0 = c3 (rhs)