How to check for exception values like nan,None and infinity

67 Views Asked by At

I'm coding a validator for a data pipeline and I want to assert that the numbers in my data are all "good" numbers, so I want to check for NaN, Inf or null. Are some kind of function that can test for that and there is more exception values that I should check ?

I think that a test to check if the number is between a range for example [-10,10] will guarantee good numbers, but i want to know if there is a better method

2

There are 2 best solutions below

1
Cem Koçak On

To check if all the numbers in the data are good numbers:

import math
import numpy as np

def good_number_checker(number):
    return all(isinstance(number, (int,float)) and math.isfinite(number) and not math.isnan(number) for number in data)

data = [1,2,3]
print(good_number_checker(data)) #Prints True
data = [1,2,3,float("inf")]
print(good_number_checker(data)) #Prints False

To check individually:

import math
import numpy as np

def good_number_checker(number):
    return isinstance(number, (int,float)) and math.isfinite(number) and not math.isnan(number)

data = [1,2,3,float("inf")]

for number in data:
    if good_number_checker(number)==False:
        print("Bad number " + str(number) +" exists") #Prints Bad number inf exists
0
cyril On

If your data are in a numpy array x, np.isfinite(x) returns a boolean array of the same shape than, filled with True where x has a finite value.