Detecting biases in Word-Emdeddings

Important links:

github link- https://github.com/dccuchile/wefe ( check requirement.txt for dependency)

WEFE will not work if there is any conflict of dependency.

metrics available with WEFE- Word Embedding Association Test (WEAT) Relative Norm Distance (RND) Relative Negative Sentiment Bias (RNSB) Mean Average Cosine (MAC)

know more about the WEFE- https://www.kdnuggets.com/2020/08/word-embedding-fairness-evaluation.html

Step 1 -( load the word embedding model)

Here user has to load their embedding ( pre-defined mapping)

Step 2- ( Create a query)

Create the Query with a loaded, fetched or custom target and attribute word sets. In this case, we manually set both target words and attribute words.

Step 3 -Run the WEAT metric from Wefe

1. model checks the dissemilarity of female words between both type of attribues. 2. model chekcs the dissemilarity of male words with both set type of attributes 3. dissemilarity for both the targets( male and female) should be equal.

Step 4- Interpretation of Results

Any postive value represnts that there is indeed a biased relationship between women and the arts with respect to men and science. Any negative value represnts that there is indeed a biased relationship between men and the sciemce with respect to women and arts. Ideal value is 0.