Python: How do I remove a key from a list of dictionaries based on it's value in all dictionaries in that list

Question

I've a list of dictionaries like

my_list = [{'a':0, 'b':2, 'c':1, 'd': 0}, 
           {'a':0, 'b':0, 'c':0, 'd': 0},
           {'a':0, 'b':3, 'c':0, 'd': 0}
]

i want to remove all keys which are 0 in ALL dictionaries. e.g. the output should be

my_list = [{'b':2, 'c':1}, 
           {'b':0, 'c':0},
           {'b':3, 'c':0}]

There could be multiple keys that needs to be removed. But the keys will be same for all the dictionaries present in the list. So far i've tried nested loops for this e.g.

keys = list(stats[0].keys())
keys_to_remove = []
for key in keys:
    key_total = 0
    for stat in stats:
        key_total += stat[key]
    if key_total == 0:
        keys_to_remove.append(key)

return [{k: v for k, v in entry.items() if k not in keys_to_remove} for entry in stats]

But I need a optimised solution for this. The one I'm currently using seems really odd to me, is there any shorter way of doing it ? e.g list comprehension or one liner loops etc ?

Answer 1

With nested zip (to combine same key entries) and any functions:

res = list(map(dict, zip(*[t for t in zip(*[d.items() for d in my_list]) 
                           if any(v[1] != 0 for v in t)])))

---

[{'b': 2, 'c': 1}, {'b': 0, 'c': 0}, {'b': 3, 'c': 0}]

Answer 2

pandas can do this with very little code and should offer some optimizations:

import pandas as pd

my_list = [{'a':0, 'b':2, 'c':1, 'd': 0}, 
           {'a':0, 'b':0, 'c':0, 'd': 0},
           {'a':0, 'b':3, 'c':0, 'd': 0}
]
df = pd.DataFrame(my_list)
print(df.loc[:, (df!=0).any(axis=0)].to_dict(orient='records'))

Output:

[{'b': 2, 'c': 1}, {'b': 0, 'c': 0}, {'b': 3, 'c': 0}]