ML has increasingly been used across various process across various industries, causal inference being one such process. It is nothing but a process of drawing a conclusion from a causal connection which is based on the conditions of the occurrence of an effect. Casualty is what connects one process to other process, where both the processes are related to and dependent on each other. In other words, the process of determining whether a causal relationship does in fact exist at all, is called causal inference.
To be able to use causal interface, Microsoft introduced a software library called DoWhy. It is a Python-based software library and its name has been inspired from Judea Pearl’s do-calculus, a theory which is a part of probabilistic causality, which again is a part of the Bayesian network.
Why Do We Need DoWhy?
Predictive methods have a set of input and it recognises patterns based on that input. Changing the input will give a different set of pattern. Although there are a number of causal inference ML methods that give predictions based on the inputs whose output pattern is not fed to it. The main objective is to draw attention to finding out the assumptions, instead of giving more importance to estimation. Such methods are important in the context of several decision-making situations.
As these methods start to have its use in our daily life as well, the impact of its effect is an important concern and is gaining importance.
How DoWhy Works
DoWhy is designed using two guiding principles:
- Making causal assumptions explicit
- Testing robustness of the estimate to violation of the assumed assumptions
It performs the following tasks:
- DoWhy distinguishes between identification and estimation problems. It spends the maximum amount of time in identification. Its job here involves making assumptions about data-generation processes. DoWhy uses Bayesian graphical model framework to represent assumptions. In this graphical model framework, users can specify what they are aware of and what they do not know, about the data generating process. On the other hand, estimation involves estimating the target and estimand from the given data. Methods used for estimation are based on the potential outcomes framework, like matching, stratification and instrumental variables.
- DoWhy creates a separate causal graphical model for each problem. This makes each causal assumption very specific. The graph to be provided does not necessarily have to be complete. The graph can be incomplete, expressing information regarding initial conditions of the variables, DoWhy considers the rest of the variables. It first builds a causal model and tests the assumptions using the observed data. It then views and analyses these assumptions, either graphically, or using a set of conditional independence statements.
- Once the assumptions are made, DoWhy provides with sensitivity tests to check the reliability of the obtained estimate. DoWhy checks the validity of the obtained estimate based on the given assumptions in the graphical model. It can check how the estimate changes with the change of certain assumptions. It can also assess robustness of the estimate to violations. It checks how the estimate varies with varying assumptions. These variations can also be input in the graphical model to see how the estimate changes.
How DoWhy Is Different From Existing Softwares For Causal Inference
One of the major advantages is that DoWhy treats identification and estimation separately. Because of this, for a target estimand, it is able to free up the estimator to use any available statistical estimator, thereby being able to focus the causal inference on the process of identification.
The other crucial job of the causal inference is to check the robustness of the estimate to the assumptions that are not verified, and this job is usually neglected. This library can automatically run sensitivity and robustness checks on the obtained estimate.
It uses graphical models as well as potential outcomes for the same. It uses graphical models to estimate the modelling assumption and to identify a non-parametric causal effect. It also uses potential outcome methods for the job of estimation. Identifies a suitable/required causal effect based on the graphical model calculations.
DoWhy supports Python 3+ and requires packages like numpy, scipy, scikit-learn, pandas, pygraphviz, networkx, Matplotlib and sympy. This project plans to add more features to the library and currently invites suggestions and contributions.
