www.cs.ubc.ca/~murphyk/Bayes/bayes.html
A Brief Introduction to Graphical Models and Bayesian Networks By Kevin Murphy, 1998. "Graphical models are a marriage between probability theory and graph the ory. They provide a natural tool for dealing with two problems that occu r throughout applied mathematics and engineering -- uncertainty and comp lexity -- and in particular they are playing an increasingly important r ole in the design and analysis of machine learning algorithms. Fundament al to the idea of a graphical model is the notion of modularity -- a com plex system is built by combining simpler parts. Probability theory prov ides the glue whereby the parts are combined, ensuring that the system a s a whole is consistent, and providing ways to interface models to data. The graph theoretic side of graphical models provides both an intuitive ly appealing interface by which humans can model highly-interacting sets of variables as well as a data structure that lends itself naturally to the design of efficient general-purpose algorithms. Many of the classical multivariate probabalistic systems studied in field s such as statistics, systems engineering, information theory, pattern r ecognition and statistical mechanics are special cases of the general gr aphical model formalism -- examples include mixture models, factor analy sis, hidden Markov models, Kalman filters and Ising models. The graphica l model framework provides a way to view all of these systems as instanc es of a common underlying formalism. This view has many advantages -- in particular, specialized techniques that have been developed in one fiel d can be transferred between research communities and exploited more wid ely. Moreover, the graphical model formalism provides a natural framewor k for the design of new systems."
New York Times article (4/28/01) about Bayesian statistics. Note: Despite the name, Bayesian networks do not necessarily imply a comm itment to Bayesian statistics.
List of other Bayes net tutorials Representation Probabilistic graphical models are graphs in which nodes represent random variables, and the (lack of) arcs represent conditional independence assumptions. Hence they provide a compact representation of joint probability distributions. Undirected graphical models, also called Markov Random Fields(MRFs) or Markov networks, have a simple definition of independence: two (sets of) nodes A and B are conditionally independent given a third set, C, if allpaths between the nodes in A and B are separated by a node in C By contrast, directed graphical models also called Bayesian Networks or Belief Networks (BNs), have a more complicated notion of independence, which takes into account the directionality of the arcs, as we explain below.
In addition, directed models can en code deterministic relationships, and are easier to learn (fit to data). In the rest of this tutorial, we will only discuss directed graphical m odels, ie, Bayesian networks. In addition to the graph structure, it is necessary to specify the parame ters of the model. For a directed model, we must specify the Conditional Probability Distribution (CPD) at each node. If the variables are discr ete, this can be represented as a table (CPT), which lists the probabili ty that the child node takes on each of its different values for each co mbination of values of its parents. Consider the following example, in w hich all nodes are binary, ie, have two possible values, which we will denote by T (true) and F (false).
We see that the event "grass is wet" (W=true) has two possible causes: ei ther the water sprinker is on (S=true) or it is raining (R=true). The st rength of this relationship is shown in the table. For example, we see t hat Pr(W=true | S=true, R=false) = 09 (second row), and hence, Pr(W=fal se | S=true, R=false) = 1 - 09 = 01, since each row must sum to one. S ince the C node has no parents, its CPT specifies the prior probability that it is cloudy (in this case, 05). The simplest conditional independence relationship encoded in a Bayesian network can be stated as follows: a node is independent of its ancestors given its parents, where the ancestor/parent relationship is with respe ct to some fixed topological ordering of the nodes. By the chain rule of probability, the joint probability of all the nodes in the graph above is P(C, S, R, W) = P * P(S|C) * P(R|C,S) * P(W|C,S,R) By using conditional independence relationships, we can rewrite this as P(C, S, R, W) = P * P(S|C) * P(R|C) * P(W|S,R) where we were allowed to simplify the third term because R is independent of S given its parent C, and the last term because W is independent of C given its parents S and R We can see that the conditional independence relationships allow us to re present the joint more compactly. Here the savings are minimal, but in g eneral, if we had n binary nodes, the full joint would require O(2^n) sp ace to represent, but the factored form would require O(n 2^k) space to represent, where k is the maximum fan-in of a node. Inference The most common task we wish to solve using Bayesian networks is probabil istic inference. For example, consider the water sprinkler network, and suppose we observe the fact that the grass is wet. There are two possibl e causes for this: either it is raining, or the sprinkler is on. We can use Bayes' rule to compute the posterior probabili ty of each explanation (where 0==false and 1==true).
is a normalizing constant, equal to the probability (likelihood) of the d ata. Explaining away In the above example, notice that the two causes "compete" to "explain" the observed data. Hence S and R become conditionally dependent given that their common child, W, is observed, even though they are marginally independent. For example, suppose the grass is wet, but that we also know that itis raining. Then the posterior probability that the sprinkler is on goes down: Pr(S=1|W=1,R=1) = 01945 This is called "explaining away". In statistics, this is known as Berkson's paradox, or "selection bias". Let C denote the event that someone is admitted to college, which is made true if they are either brainy or sporty . Suppose in the general population, Band S are independent. We can model our conditional independence assumptions using a graph which is a V structure, with arrows pointing down: B S \ / v C Now look at a population of college students (those for which C is observ ed to be true). It will be found that being brainy makes you less likely to be sporty and vice versa, because either property alone is sufficien t to explain the evidence on C (ie, P(S=1 | C=1, B=1) <= P(S=1 | C=1)) .
This is called diagnostic, or "bott om up", reasoning, since it goes from effects to causes; Bayes nets can also be used for causal, or "top down", reasoning. For example, we can compute the probability that the g rass will be wet given that it is cloudy. Hence Bayes nets are often cal led "generative" models, because they specify how causes generate effect s Causality One of the most exciting things about Bayes nets is that they can beused to put discussions about causality on a solid mathematical basis. One very interesting question is: can we distinguish causation from mere correlation? The answer is "sometimes", but you need to measure the relationships between at least three variables; the intution is that one of the variables acts as a"virtual control" for the relationship between the other two, so we don't always need to do experiments to infer causality.
"Cause and Correlation in Biology", Bill Shipley, 2000, Cambridge University Press. Conditional independence in Bayes Nets In general, the conditional independence relationships encoded by a Bayes Net are best be explained by means of the "Bayes Ball" algorithm (due t o Ross Shachter), which is as follows: Two (sets of) nodes A and B are c onditionally independent (d-separated) given a set C if and only if ther e is no way for a ball to get from A to B in the graph, where the allowa ble movements of the ball are shown below. Hidden nodes are nodes whose values are not known, and are depicted as unshaded;
The most interesting case is the first column, when we have two arrows co nverging on a nod...
|
http://csua.com/feed/