How to visualize a graph with a high amount of nodes and edges with python?

Question

I want to visualize a directed graph with huge amount of nodes and edges. The graph has about 2000 nodes and nearly the same amount of directed edges between those nodes. Edges may connect two nodes. For example, node_1 and node_2 could be connected by edge node_1_node_2 and vice versa, node_2_node_1. There may exist more than one node that connect those nodes or other nodes to that nodes. I want to visualize all the nodes and all existing edges. Currently, the nodes and edges are in CSV tables, but they can be loaded into a dict or a json file. I implemented the code with python and got a nice and good picture of the graph:

For that I used the following code:

from matplotlib import pyplot as plt
import networkx as nx

G = nx.MultiDiGraph([
    ('BAY_530', 'BAY_530_E'),
    ('BAY_530_E', 'BAY_530_W'),
    ('BAY_530_E', 'SSX_W'),
    ('BAY_530', 'INTR_530'),
    ('BAY_530_W', 'BAY_530'),
    ('INTR_530', 'BAY_530'),
    ('INTR_530', 'M_502_5'),
    ('M_502_1', 'M_502_2'),
    ('M_502_2', 'M_502_1'),
    ('M_502_2', 'M_502_3'),
    ('M_502_3', 'M_502_2'),
    ('M_502_3', 'M_502_4'),
    ('M_502_4', 'M_502_3'),
    ('M_502_4', 'M_502_5'),
    ('M_502_5', 'M_502_4'),
    ('M_502_5', 'SSX_S'),
    ('SSX_N', 'SSX_S'),
    ('SSX_N', 'SSX_W'),
    ('SSX_S', 'INTR_530'),
    ('SSX_S', 'SSX_N'),
    ('SSX_W', 'BAY_530_E'),
    ('SSX_W', 'SSX_N')
])

pos = nx.circular_layout(G)
#pos = nx.spring_layout(G, seed=3113794652)
names = {name: name for name in G.nodes}
options = {"edgecolors": "tab:gray", "node_size": 800, "alpha": 0.9}

nx.draw_networkx_nodes(G, pos, node_color='b', node_size=1500, alpha=1)
nx.draw_networkx_labels(G, pos, names, font_size=8, font_color='w')

ax = plt.gca()
for e in G.edges:
    ax.annotate("",
                xy=pos[e[1]], xycoords='data',
                xytext=pos[e[0]], textcoords='data',
                arrowprops=dict(arrowstyle="->", color="0",
                                shrinkA=10, shrinkB=10,
                                patchA=None, patchB=None,
                                connectionstyle="arc3,rad={}".format(0.3 * e[2])
                                )
                )

plt.tight_layout()
plt.axis('off')
plt.show()

It works for me on small amount of edges and nodes, but if I increase the amount of nodes, it becomes hard to arrange those nodes and edges in nice looking way. Because they overlap each over, it is impossible to understand what is connected to what.

I also tried to use mouseover to select a node and then highlight the edges that comes in and out the node. I have to indicate here that for better understanding the connection/relation between 2 nodes, the edges must be separate for each connection. I also tried to use a circular layout, but then it becomes to big and not all connections are shown.

I know that there is a D3 library in javascript that may work for me, but I would like to implement it in python.

Does there exist a better way to implement this in python or a better library that can deal with a high amount of edges and nodes? I also want support for dynamic elements such as onclick or mouseover in order to be able to implement selecting nodes and then highlighting connected nodes and edges.

Answer 1

I am unfamiliar with any Python libraries, if you found one, then you're all good. To do this by yourself without any libraries would require using an algorithm to display the graph in a "pretty" way.

This is determined by a few factors, generally it is considered that equal spacing and as much symmetry and possible, whilst having as few crossing of edges is best. This is proven to be NP-Complete and would not be fun to solve perfectly, but is possible to get close by using a range of already made iterative algorithms.

Here is a paper collecting some of the more well known force directed: Force Directed Drawing