How to copy unique keys and values from another dictionary in Python

Question

I have a dataframe df with transactions where the values in the column Col can be repeated. I use Counter dictionary1 to count the frequency for each Col value, then I would like to run a for loop on a subset of the data and obtain a value pit. I want to create a new dictionary dict1 where the key is the key from dictionary1 and the value is the value of pit. This is the code I have so far:

dictionary1 = Counter(df['Col'])
dict1 = defaultdict(int)

for i in range(len(dictionary1)):       
    temp = df[df['Col'] == dictionary1.keys()[i]]
    b = temp['IsBuy'].sum()
    n = temp['IsBuy'].count()
    pit = b/n
    dict1[dictionary1.keys()[i]] = pit

My question is, how can i assign the key and value for dict1 based on the key of dictionary1 and the value obtained from the calculation of pit. In other words, what is the correct way to write the last line of code in the above script.

Thank you.

Answer 1

Since you're using pandas, I should point out that the problem you're facing is common enough that there's a built-in way to do it. We call collecting "similar" data into groups and then performing operations on them a groupby operation. It's probably wortwhile reading the tutorial section on the groupby split-apply-combine idiom -- there are lots of neat things you can do!

The pandorable way to compute the pit values would be something like

df.groupby("Col")["IsBuy"].mean()

For example:

>>> # make dummy data
>>> N = 10**4
>>> df = pd.DataFrame({"Col": np.random.randint(1, 10, N), "IsBuy": np.random.choice([True, False], N)})
>>> df.head()
   Col  IsBuy
0    3  False
1    6   True
2    6   True
3    1   True
4    5   True
>>> df.groupby("Col")["IsBuy"].mean()
Col
1      0.511709
2      0.495697
3      0.489796
4      0.510658
5      0.507491
6      0.513183
7      0.522936
8      0.488688
9      0.490498
Name: IsBuy, dtype: float64

which you could turn into a dictionary from a Series if you insisted:

>>> df.groupby("Col")["IsBuy"].mean().to_dict()
{1: 0.51170858629661753, 2: 0.49569707401032703, 3: 0.48979591836734693, 4: 0.51065801668211308, 5: 0.50749063670411987, 6: 0.51318267419962338, 7: 0.52293577981651373, 8: 0.48868778280542985, 9: 0.49049773755656106}