Explainable AI with PDP (Partial Dependence Plot)

Introduction

Understanding functional relations between predictive and predictor variables can be difficult on a regular basis when using a black-box model. Partial Dependence Plots (PDP) were introduced by Friedman in 2001 who was facing a challenge in understanding the gradient-boosted machine he was working on. Usually, it is easy to calculate the importance of a variable but tough to know and understand its individual impact on the predictor variable. PDPs help us solve this problem by providing a way to functionally calculate and understand a variable’s importance.

What is PDP?

PDP’s visualize the marginal effect of a predictor variable on the predictive variable by plotting the average model outcome at different levels of the predictor variable. It gives an idea of the effect that a predictor variable has on the predictive variable on an average.

Pros

  • PDPs are simple, easy to understand and can be explained to a non-technical person without any difficulties
  • They can be used to compare models to decide which model works best for a use case
  • They are intuitive and easy to implement

Cons

  • PDP assumes on the default that the features are uncorrelated
  • They can only plot averaged marginal dependence function and cannot work on individual points, which can be a huge problem when the dataset has only 2 equal opposite values
  • They also assume that there are no interactions between the variables which is highly unlikely in the real world
  • Though interactions can be plotted, they are only limited to second-order

Implementation

The implementation of this done on Gradient Boosting Machine

What is GBM?

  • Gradient Boosting Machine is a machine learning algorithm that forms an ensemble of weakly predicted decision trees
  • It constructs a forward stage-wise additive model by implementing gradient descent in function space
  • Also known as MART (Multiple Additive Regression Trees) and GBRT (Gradient Boosted Regression Trees)

Visualizations

PDP for every feature

  • As Pregnancies increase, the person’s chances of becoming diabetic go up
  • Higher the Glucose, the higher the chances of a person becoming diabetic
  • BMI of more than 25 increases an individual's chances of becoming diabetic

3-D PDPs

PDP interact plot

References

  1. The Elements of Statistical Learning: Trevor Hastie, Robert Tibshirani and Jerome Friedman
  2. Molnar, Christoph. “Interpretable machine learning. A Guide for Making Black Box Models Explainable”, 2019. https://christophm.github.io/interpretable-ml-book/.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store