Probabilistic graphical models

A probabilistic graphical model is a powerful framework which represents the conditional dependency among the random variables in a graph structure. It can be used in modelling a large number of random variables having many a non-obvious interdependency. The two approaches of graphical representation of the distribution are Markov and Bayesian Networks. Both of them differ in the set of independence that they can encode.

Bayesian Networks: When the model structure is a Directed Acyclic Graph (DAG), the model represents a factorization of the joint probability of all the random variables. The Bayesian Networks capture the conditional independence between the random variables and reduce the number of parameters required to estimate the joint probability distribution.

Markov Networks: They are used when the underlying network structure is an undirected graph. They follow the Markov process, i.e., given the current state, the future states would be independent of the past states. Markov Networks represent the distribution of the sequence of the nodes. This network could be cyclic, which is never true with Bayesian. The general idea of any Markov Process is that "given the present, future is independent of the past". Visual explanation of Markov chain is here https://setosa.io/blog/2014/07/26/markov-chains/

Difference: The main weakness of Markov networks is their inability to represent induced and non-transitive dependencies; two independent variables will be directly connected by an edge, merely because some other variable depends on both. As a result, many useful independencies go unrepresented in the network. To overcome this deficiency, Bayesian networks use the richer language of directed graphs, where the directions of the arrows permit us to distinguish genuine dependencies from spurious dependencies induced by hypothetical observations.