01-07-2022

Regional Explanations for XAI

Master Thesis: Regional Explanations for XAI

Concept of thesis

During the four months, I had to write my master's thesis; I wrote a theoretical thesis on the topic of XAI. More specifically, I wrote a thesis exploring the concept of establishing regional explanations for XAI. This post will briefly summarise the thesis, covering the essential findings. The complete thesis is downloadable in the link above.

Explainable Artificial Intelligence

Machine Learning models that are uninterpretable are black boxes. This means that the reasoning for a model creating a decision is unknown. Not understanding why a model has produced an outcome is a massive issue, notably within high-risk domains. In high-risk domains, such as healthcare, the usage of Machine Learning models is as much about automation as the human-machine relationship, enabling practitioners to make fewer oversights and check their beliefs against a less biased assumption to which a computer contributes. Explainable Artificial Intelligence (XAI) contributes to promoting the human-machine relationship. XAI aims to improve the relationship between humans and machines to avoid blind trust and enable an understanding of what a model's algorithm(often a black box) accurately does and how.

Current Explanations

Within XAI, two well-established types of explanations are currently used when extracting information from a given model. The first is local explanations, which provide explanations for a single observation. On the other end of the scale, it is possible to use global explanations. Global explanations aggregate the local values and provide a generalised set of explanations for the whole model. These two types of explanations are either specific to one individual observation or generalised to all observations, leaving little insight into which information may lie between the two types of explanations.

Therefore, granted local and global explanations, one might want to gain insights into regions of explanations. Currently, no formalised method exists to enable insights into regions of explanations for a model. Subsequently, the main focus of this thesis is to explore a new area which positions itself between local and global explanations. From this point forward, regional explanations denote this new area. The focus of the exploration explores areas and considerations that may aid the creation of a sophisticated method, similarly to local and global explanations, for regional explanations.

Visualising the Data & Explanation Space

The exploratory analysis investigated the structural difference in the data space (feature space) and the explanation space, referring to the latent space that is established when applying XAI methods to the data space; in this thesis, the focus was on applying a SOTA method, SHAP.

A Parallel Coordinates Plot (PCP) was chosen to visualise the two spaces. The PCP is an exciting type of visualisation that enables a way to visualise high-dimensional data. The plot maps each observation in the dataset as a vertical line across the x-axis. Each vertical line on the x-axis is a feature, and values are provided on the y-axis. A PCP can work on features of the same numeric value and scale but also with features of different units.

Below are two examples: a PCP of the data space and an explanation space. Here it is evident how the structure and values of the data differ. The data space is more "chaotic" due to the variation in feature values, while in the explanation space, the features follow a scale from -1:1 due to the application of SHAP.

Similarity within and between Spaces

Similarity within Spaces

The method for measuring similarity was taking the squared euclidean distance between the standardized observations, creating a distance matrix. Then it is possible to take a particular observation, such as observation 42 and find the most similar observations in the spaces. When doing this, it becomes apparent how the distances in the data and explanation space differ. The data space generally has greater distances between observations than the explanation space.

The two PCP figures below show observation 42 (the red line) and the ten nearest neighbours (black dotted lines) within each given space.

Similarity between Spaces

Given that the spaces and similarities within are explored, next is to investigate the similarity between these spaces. But why? The interestingness of this comparison is to recognise the shared similar points and the extent to which they are similar. Moreover, it is possible to explore other information from the comparison.

Let us assume that we expect the similarity between spaces to be almost identical. Given this assumption, then moving forward and establishing regional explanations, it would not matter whether which space we utilise for clustering. If the assumption does not hold, then it is possible to look at to which extent it does not hold and what we can conclude from this. The approach for finding shared similar observations was to take different observations and calculate the amount of shared similar observations given four different thresholds. Below is a table showing six observations and the similar shared observations between the data and explanation space for each threshold.

Another approach was to average the global shared similar observations for all observations for each threshold, as seen below in table 4.6. The global average approach enables a way to see, given all observations, how many given the whole dataset share similar observations. Naturally, dilution occurs as the threshold increases, creating a more significant number of shared similar observations.

Given the shared similar observations, it is possible to investigate that, for example, given the top 10 observations to observation 42, how many are non-similar and can we somehow visualise this? Using a PCP, it is possible to map the non-similar points onto a PCP of the explanation spaces' top 10 similar observations to point 42, as seen below.

The figure shows that the observations from the DS have different SHAP values once mapped to the explanation space; therefore, they are not similar to 42 in the explanation space. The SHAP values generate a weight. This weight is not directly transferable from the data space to the explanation space. Therefore, the information gained from the data space is not equal to the information gained from the explanation space. This synergy is explored during clustering and exploring mutual information between the two spaces.

Clustering the Spaces

While global explanations allow for interpreting the whole model, the concept for regional explanations is to interpret smaller groups of observations. By providing a set of observations(a region) and obtaining SHAP values for that set, it is possible to establish an interpretation of the average feature importance for this region. Establishing regions and explanations for a region means shifting toward more specific explanations instead of global ones, which are more generalised.

For the thesis, two clustering methods were explored: K-Means and HDBSCAN. Both the data and explanation space was clustered to compare and investigate similarities and differences between the two spaces. For each clustering method, parameters were required. For K-means this meant the number of clusters (K), while for HDBSCAN, this meant the minimum cluster size and the minimum number of samples.

K-Means

Applying K-means to the spaces resulted in a K = 2 for the data space and K = 5 for the explanation space. While the silhouette scores are one way to evaluate the clustering, the purity of the clustering also provides excellent insight into how well the clustering performed. The purity is the percentage of correctly clustered observations; ideally, the goal is to aim for purity of 100%.

Data Space:

Contingency, purity table and squared euclidean distances from observation 42 given K = 2 in the data space.

Explanation Space:

Contingency, purity table and squared euclidean distances from observation 42 given K = 2 in the data space.

Based on these results, it is clear that clustering the explanation space creates more clusters (regions) and a much more consistent purity of these clusters. See thesis section "4.2 Concept of Regions" for more insight into how many regions is a desirable outcome.

HDBSCAN

HDBSCAN is the second clustering method explored for generating regional explanations. A key aspect of HDBSCAN is the ability to create noise, eliminating potential outliers. Similarly to K-means, the result of clustering each space is seen below.

Data Space:

Contingency and purity table when clustering the data space with HDBSCAN given the hyperparameters min_cluster_size = 5 and min_samples = 7.

Explanation Space:

Contingency and purity table when clustering the explanation space with HDBSCAN given the hyperparameters min_cluster_size = 11 and min_samples = 4.

Using HDBSCAN in both spaces improves the K-means clustering; in particular, clustering the explanation space results in 100% purity.

Identifying Shared Similar Observations Between Spaces

To compare the clusterings methods and the euclidean distance measure approach, table 4.30 below was established. Here it is evident that HDBSCAN in the data space and similarity approach for the data space resulted in the same amount of shared similar observations given a threshold of 10. Similarly, HDBSCAN in the explanation space shared all similar observations to the similarity approach in the explanation space.

Comparing clustering of spaces

The Normalized Information Gain (NMI) and Adjusted Rand Index (ARI) were used to compare the clustering methods and space. Below is a table showing the result of these scores. These results are not surprising and back up the analysis and results from the previous sections.

Once a clustering method and space are selected, it is possible to establish regional explanations by applying SHAP to each region and obtaining the global explanations for each region, becoming the regional explanations.

Final Words

This post was a small summarisation of my thesis; it received the highest grade in the Danish system, 12, and allowed me to graduate with a master's degree in Data Science; I am now a Data Scientist! To fully understand the theory behind it, please refer to the theoretical background section of the thesis.

Finally, do not hesitate to reach out with any questions or comments. I am always happy to discuss XAI and Machine Learning in general.