Key takeaways
- Deep learning models often operate as black boxes, limiting transparency in their decision-making processes.
- Human biases influence data collection, which can lead to biased machine learning models despite the expectation of rationality from computers.
- An unbiased model is essential for the widespread acceptance of AI technologies in the future.
- Efforts to mitigate bias include careful data curation, along with innovative approaches like testing with Concept Activation Vectors (TCAV).

Testing With CAV
The concepts in concept activation(CAV) refers to a prediction class with color, gender or race; and other typical high level concepts used by humans to communicate. In traditional methods, the prediction is pivoted towards the weights given to features in an image which are low level like color intensity and other information retrieved by scanning hundreds of pixels. The main objective of the researchers here is to provide ‘ high-dimensional internal state of a neural nets in human-friendly concepts’. TCAV uses directional derivatives to quantify the model prediction’s sensitivity to a hidden high-level concept, learned by a concept activation vector. Firstly a concept of interest is defined by choosing a set of examples that represent that concept. This allows the model to draw more insights other than the usual interpretations.
How Valid Is This Method
To test for the validation of CAVs, the researchers sorted the images of random class for inspection and then patterns of high activations are identified for visual confirmation. As a CAV encodes the direction of a concept in the vector space, cosine similarity between a set of pictures is calculated. This helps in revealing any underlying biases as the sorting is done on similarities. The experiments show that the concept ‘female’ is more relevant to the ‘apron’ class. Biases such as these can be inspected in the model and the quality of a certain concept can be determined. For example, the concept ‘apple’ should be shown more relevance to concepts like ‘red’ or ‘spherical’ and not ‘Newton’ or a ‘doctor’. According to Google, TCAVs can be attributed with the following:- TCAV requires little to no machine learning expertise from the user end.
- The range of concepts are not limited owing to TCAV’s adaptability.
- The machine learning model need not be modified to suit TCAV.
- Can draw insights from just one quantitative measure.
Spotted an error? Tell our editors. Read our editorial standards and corrections.





