Numpy inverse function does not work for version 1.16.4 and 1.24.x but works for 1.21.6

Question

For a matrix A, when using numpy version 1.16.4 to run numpy.linalg.inv(A.T.dot(A)) will result in: numpy.linalg.LinAlgError: Singular matrix. When using numpy version 1.21.6, no error raised. So I thought all I need is a newer numpy version. However, when using numpy version 1.24.x on the same matrix, the same error returns numpy.linalg.LinAlgError: Singular matrix.

My three questions are:

1. Which/what updates are included between 1.16.4 and 1.21.6 that made numpy functional?
2. numpy does not support backward capability? It seems that for this general library it should.
3. When developing a project, which steps I should take to ensure I have a stable version?

If A.T.dot(A) is indeed a singular matrix, all I can think of is there might be a problem with decimal calculation precision. I am clueless here. Please help.