The power law fitted curve doesn't overlap with the data points

Question

I have been trying to fit a powerlaw to my degree distribution data using the powerlaw package by Alstott et al. Following is my code.

import networkx as nx
import powerlaw
import matplotlib.pyplot as plt

g = nx.powerlaw_cluster_graph(1000, 25, 0.9)

degrees = list(dict(g.degree).values())

fit = powerlaw.Fit(degrees, discrete=True)

fig, ax = plt.subplots(figsize = (10,10))

powerlaw.plot_pdf(degrees, color='r', marker = 'o', linestyle='', ax=ax) #plotting binned data points
fit.power_law.plot_pdf(color = 'darkblue', ax=ax) #plotting fitted curve

ax.set_xlabel('k', fontsize = 16)
ax.set_ylabel('P(k)', fontsize = 16)
ax.set_title('Degree Distribution of a Network', fontsize = 20)
ax.tick_params(labelsize = 12)
plt.text(0.05, 0.6, f'γ =  {round(fit.power_law.alpha, 2)}', transform=plt.gca().transAxes, fontsize=20, color='darkblue')
plt.text(0.05, 0.55, '$x_{min}$ = ' + f'{fit.power_law.xmin}', transform=plt.gca().transAxes, fontsize=20, color='darkblue')
plt.text(0.05, 0.50, f'KS Distance: {round(fit.power_law.D, 2)}', transform=plt.gca().transAxes, fontsize=20, color='darkblue')

As you see, the blue fitted curve is distant from the red binned data points.

The thing that's left me scratching my head is how the fitted curve is positioned away from all the points. Why is this the case? Does it mean the fit can't be trusted? The kolmogorov-smirnov distance is also low. The fit.noise_flag is false meaning the fit is not invalid. So, what gives?

Answer 1

I had figured this out couple of days after I posted it.

The fitting is done by finding the xmin value for which the KS distance of a powerlaw fit is least. This is the optimum xmin value. So, the data used for the fitting is optimum xmin onwards. Therefore, the fitted curve is plotted along the pdf of the data used for the fitting rather than the entire empirical data. The data used for the fitting is shifted to the right 'cause the pdf changes when we remove the x values that are lesser than optimum xmin since the y values (the frequency/probability) increases.

Here's a plot displaying that.

This plot is for a different graph since I didn't save the one I used in the question. Regardless, the point stands.

Here's the code:

import networkx as nx
import powerlaw
import matplotlib.pyplot as plt
from collections import Counter
import pickle
import numpy as np

g = nx.powerlaw_cluster_graph(1000, 25, 0.9)

degrees = list(dict(g.degree).values())

fit = powerlaw.Fit(degrees, discrete=True)


fig, ax = plt.subplots(figsize = (10,10))

fit.plot_pdf(color='red', marker = 'o', linestyle='', label = 'data used     for the fit', original_data = False, ax = ax)
fit.plot_pdf(color='gold', marker = 'o', linestyle='', label = 'empirical data', original_data = True, ax = ax) #plotting original data points
fit.power_law.plot_pdf(color = 'darkblue', label = 'fitted curve') #plotting fitted curve


ax.set_xlabel('k', fontsize = 16)
ax.set_ylabel('P(k)', fontsize = 16)
ax.set_title('Degree Distribution of a Network', fontsize = 20)
ax.tick_params(labelsize = 12)
plt.text(0.05, 0.3, f'γ =  {round(fit.power_law.alpha, 2)}',     transform=plt.gca().transAxes, fontsize=20, color='darkblue')
plt.text(0.05, 0.25, '$x_{min}$ = ' + f'{fit.power_law.xmin}', transform=plt.gca().transAxes, fontsize=20, color='darkblue')
plt.text(0.05, 0.20, f'KS Distance: {round(fit.power_law.D, 2)}', transform=plt.gca().transAxes, fontsize=20, color='darkblue')
plt.legend(fontsize = 16)`