How do I normalize biased reviews from raters?

139 Views Asked by At

I have a dataset with five different reviewers and their scores of 50 job candidates. The scores are all on the same scale, 0-20 points. However, I create histograms of the frequency of the scores for each reviewer and there are some noticeable biases, such as one reviewer who did not provide a score below 16. None of them are normal distributions. I think I should try to address these biases and adjust scores for the candidates that are more fair, which I think means I need to normalize the reviewers' scores. Any recommendations on which methods I should consider or how I should go about this?

I looked at standardization, specifically z-standardizing the scores, but realized this just changes the mean and distribution but does nothing to address bias. I've also searched the internet for method specifically addressing bias has not been fruitful. I recently learned about min-max normalizing, but I'm not confident this is the right method for my data. I know there's a difference between standardization and normalization. I think I need normalization since the scale is the same.

0

There are 0 best solutions below