Bayesian Networks and Their Value - An Essay, Part 1

The conditional probability distribution (CPD) at each node depends only on its parents. If node X has no parents, then its local probability distribution is said to be unconditional, otherwise it is conditional. For discrete random variables, this conditional probability is often displayed in a table as in figure 1. Each table lists the local probability that a child takes on each of the feasible values for each combination of values of its parents. In figure 1, each child can take the value of True (T) or False (F) depending on the values of its parents. Furthermore, as it becomes clear from figure 1, a BN provides a compact factorization of the JPDs. Factorization of the joint distribution of all variables by the chain rule results in 25-1=31 model parameters for five random variables. Nevertheless, if a BN is used, as in figure 1, we can reduce the amount of parameters to 10. This simplification provides great benefits from inference, learning, and computational perspectives. It is important to note that BNs have a misleading name. The use of Bayesian statistics in conjunction with BNs provides a method to avoid overfitting of the data. However, using BNs does not imply the usage of Bayesian statistics. Actually, many practitioners follow frequentist approaches to estimate the parameters of a BN (Ben-Gal, 2008).

The backache BN example (Figure 1) considers a person who suffers from a back injury denoted by “Back”. The injury produces pain, denoted by “Ache”. The injury might be the product of a contact sport activity denoted by “Sport”. Perhaps the back injury is actually the product of new, uncomfortable chairs installed at work, denoted by “Chair”. If the chair is the culprit, then it is reasonable to think another coworker is suffering from a similar backache, denoted by “Worker”. If a variable is represented by a node, it is said that it is being “observed”. The beauty of BNs is that each node not only represents a random variable, it can also represent a hypotheses, belief, or a latent variable. Therefore BNs help reasoning on evidence but also on suppositions. In this example, it is very clear the two types of inference support that BNs allow. The first type is called top-down reasoning. It provides predictive support for node X based on evidence nodes connected to X through its parent nodes. If the person being considered in the backache BN example played rugby during the weekend and got hit on his lower back, then it provides predictive support of a back injury. The second type is bottom-up reasoning (Ben-Gal, 2008). It provides diagnostic support for node X through its children nodes. Experiencing back pain is diagnostic support of having a back injury. Once given training data and prior information, the parameter of the JDP in the BN can be estimated. Expert knowledge or causal information are considered prior information.

Sometimes, not only do the parameters need to be calculated, but the structure of the BN also must be learned. This is known as the BN learning problem. It splits into four main categories. For the first category, the goal of learning is to find the parameters that maximize the likelihood of the training dataset. In the second category, the structure is known, but there is partial observability. In this case, an expected maximization algorithm can be used to find a locally optimal maximum-likelihood estimate of the parameters. In the third case, the goal is to learn the DAG that best represents a given dataset. One approach is to assume that the variables are conditionally independent given a class which has a single common parent node to all the variable nodes. Such approach is called naive BN and it brings surprisingly good results to some practical problems (Ben-Gal, 2008). Finally, the fourth case occurs when there is partial observability and unknown graph structure. Depending on the problem, different algorithms can be used.

Check out Part 2!

References:

• Bartlett, J. G. (1997). Management of Respiratory Tract Infections. Blatimore: Williams & Wilkins.
• Ben-Gal, I. (2008). Bayesian Networks.
• Castle, N. (2017, July 13). Supervised vs. Unsupervised Machine Learning. Retrieved May 1, 2018, from https://www.datascience.com/blog/supervised-and-unsupervised-machine-learning-algorithms
• de Bruijn, N., Lucas, P., Schurink, K., Bonten, M., & Hoepelman, A. (2001). Using Temporal Probabilistic Knowledge for Medical Decision Making. In S. Quaglini, P. Barahona, & S. Andreassen (Eds.), Artificial Intelligence in Medicine (Vol. 2101, pp. 231–234). Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/3-540-48229-6_33
• Friedman, N., Geiger, D., & Goldszmidt, M. (1997). Bayesian Network Classifiers, 33.
• Lucas, P. (n.d.). Bayesian Networks in Medicine: a Model-based Approach to Medical Decision Making, 6.
• Lucas, P. J. F., Boot, H., & Taal, B. (1998). A decision-theoretic network approach to treatment management and prognosis. Knowledge-Based Systems, 11(5–6), 321–330. https://doi.org/10.1016/S0950-7051(98)00060-4
• Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. San Francisco: Morgan Kaufmann.
• Pilditch, T. D., Hahn, U., & Lagnado, D. A. (under review). Integrating dependent evidence: naive reasoning in the face of complexity.
• Pilditch, T., & Dewitt, S. (2018, February). Knowledge Learning and Inference Lecture - Bayesian Networks Part 2: Application to Problems. University College London.
• Shachter, R. D. (1986). Evaluating Influence Diagrams. Operations Research, 34(6), 871–882. https://doi.org/10.1287/opre.34.6.871
• Spirtes, P., Glymour, C., & Schienes, R. (1993). Causation Prediction and Search. New York: Springer-Verlag.

Best,

@capatazche