Bayesian network fundamentals

This page collects worked exercises on Bayesian networks. The value is not only the final probability, but the reasoning process: identifying dependencies, checking conditional independence, reading active paths and applying Bayes' rule without losing the graph structure.

Related reading: dataset preparation, inferential statistics and machine learning risks and potential.

Exercise 1

Indicate which of these relations are true and which are false. For the first and the fifth, show in detail the calculations you performed.

• $I_{p}(A,B)$: In order to determine whether this relation is true or false, we must discover whether there is a relationship between A and B; that is, whether a change in A affects the probability of B ( $p(+b|+a)\neq p(+b|\neg a)$ ).
  With this goal, and using the table with the data provided in the statement, I have calculated the probability of P(A,B) in each possible case.
  $P(+a,+b) =\sum_{c} P(+a,+b,c)= 0.03+0.12=0.15$
  $P(+a,\neg b) =\sum_{c} P(+a,\neg b,c)= 0.07+0.28=0.35$
  $P(\neg a,+b) =\sum_{c} P(\neg a,+b,c)= 0.03+0.27=0.30$
  $P(\neg a,\neg b) =\sum_{c} P(\neg a, \neg b,c)= 0.02+0.18=0.20$
  Obtaining:

  Probability	p(+b)	$p(\neg b)$	Totals
  p(+a)	0,15	0,35	0,5
  $p(\neg a)$	0,30	0,20	0,5
  Totals	0,45	0,55	1

  From this table, I will calculate the conditional probability of both variables:
  $P(+b | +a) = \frac{p(+a,+b)}{p(+a)} = \frac{0.15}{0.50}=0.3$
  $P(+b | \neg a) = \frac{p(\neg a,+b)}{p(\neg a)} = \frac{0.30}{0.50}=0.6$
  $P(\neg b | +a) = \frac{p(+a,\neg b)}{p(+a)} = \frac{0.35}{0.50}=0.7$
  $P(\neg b | \neg a) = \frac{p(+a,\neg b)}{p(\neg a)} = \frac{0.20}{0.50}=0.4$
  From the results shown, I can state that $p(+b|+a)\neq p(+b|\neg a)$ since 0.3 $\neq$ 0.6; that is, depending on the value of variable A, variable B is affected.
• $I_{p}(A,C)$FALSE
• $I_{p}(B,C)$FALSE
• $I_{p}(A,B | C)$TRUE
• $I_{p}(B,C|A)$FALSE. In order to determine whether this relation is true or false, we must discover whether there is a relationship between B and C that blocks A; that is, whether a change in B and C does not affect the conditional probability of A ( $P(B,C|A) = P(B,\neg C|A))$ ).
  To analyze this explanation, it is necessary to begin by calculating all the possibilities of P(B,C), which will be needed for the next steps: $P(+b,+c) =\sum_{a} P(a,+b,+c)= 0.03+0.03=0.06$
  $P(+b,\neg c) =\sum_{a} P(a,+b,\neg c)= 0.12+0.27=0.39$
  $P(\neg b,+c) =\sum_{a} P(a,\neg b,+c)= 0.07+0.02=0.09$
  $P(\neg b,\neg c) =\sum_{a} P( a, \neg b,\neg c)= 0.28+0.18=0.46$
  With this information, I obtained:

  Probability	p(+c)	$p(\neg c)$	Totals
  p(+b)	0,06	0,39	0,45
  $p(\neg b)$	0,09	0,46	0,55
  Totals	0,15	0,85	1

  Using this information and knowing that:
  Where we can observe that:
  $P(+B,+C|+A)\neq P(+B,\neg C|+A)$
  $P(\neg B,+C|+A)\neq P(\neg B,\neg C|+A)$
  $P(+B,\neg C|+A)\neq P(\neg B,+C|+A)$

Therefore, the statement is false.

Exercise 3

For each of the following relations, indicate whether it is true or false; also indicate which paths between the two variables are active and which are inactive.

• Ig(A,B) True, there is no connection. The possible disconnected paths that may exist are A-C-B and A-C-F-D-B
• Ig(A,E) False $\neg Ig(A, E)$The active path is A-C-E
• Ig(A,D) True, there is no connection. The possible paths would be A-C-B-D and A-C-F-D
• Ig(A,F) False $\neg Ig(A, F)$The active path is A-C-F. Another inactive path could be A-C-B-D-F
• Ig(D,E) False $\neg Ig(D, E)$The active path could be B-C-E-D.
• Ig(E,F) False $\neg Ig(E, F)$The active path could be C-E-F or B-D-F-C-E

Indicate which of the following relations are true and which are false. Also indicate for each of them whether any path between the first two variables that was inactive has been activated by the third variable, and vice versa, that is, whether any path that was active has been blocked.

• $Ig(A,B | C) \rightarrow$$\neg Ig(A,B | C)$C activates the path between A and B and, therefore, there is a path between A, B and C.
• $Ig(A,B|E) \rightarrow$$\neg Ig(A,B|E)$Between A and B there is the path A-C-B, which is activated because E is a descendant of C. The path A-C-F-D-B is blocked.
• $Ig(A,B|F) \rightarrow$$\neg Ig(A,B|F)$Between A and B there is the path A-C-B, which is activated because F is a descendant of C. The path A-C-F-D-B would also be activated.
• $Ig(A,D|C) \rightarrow$$\neg Ig(A,D|C)$C activates the path between A and D and, therefore, there is an active path A-C-B-D. Another possible path that is not enabled would be A-C-F-D, because node F is not active.
• $Ig(A,D|F) \rightarrow$$\neg Ig(A,D|F)$F activates the path between F and D and, therefore, there is an active path A-C-F-D. The other possibility, A-C-B-D, is deactivated.
• $Ig(A,F|C) \rightarrow$True. The path between A and F passes through C, which is blocked. Therefore, the path A-C-F.
• $Ig(A,F|E) \rightarrow$$\neg Ig(A,F|E)$There is an active path between A and F that goes through A-C-F.
• $Ig(D,E|B) \rightarrow$True The path D-B-C-E is blocked by B, and E-C-F-D-B also remains blocked.
• $Ig(E,F|C)\rightarrow$True The path E-C-F is blocked by C.

Exercise 5

Let P be a probability distribution that satisfies the following independence property: IP(A,B): Draw all undirected graphs and directed acyclic graphs with two variables that are independence maps (I-maps) of P.

Recall that for a graph G to be an independence map of a probability distribution P, it is sufficient that every separation relation in G corresponds to an independence relation in P; it is not necessary that every independence relation in P be reflected as a separation in G.

Directed graphs.

Undirected graphs.

Exercise 7

In a certain country, the prevalence of typhoid fever is 0'001 and that of tuberculosis is 0'01. Typhoid fever always produces fever, and bradycardia (slow heart rate) in 40 $\%$ of cases. Tuberculosis produces fever in 60 $\%$ of cases and tachycardia (faster than normal heart rate) in 58 $\%$ . The prevalence of fever in patients who do not suffer from either of these two diseases is 1'5 $\%$ , that of bradycardia is 0'05 $\%$ and that of tachycardia is 1'3 $\%$ .

• According to the naive Bayesian method, indicate which variables are involved in this problem and what values each of them can take.
  Typhoid fever is 0'001 = $P(+fiebre)=0'001$
  Tuberculosis 0'01 = $P(+tuberculosis)=0'01$
  typhoid fever $\rightarrow$ always fever = $P(+fiebre|fiebre_tifoidea)=1$
  typhoid fever $\rightarrow$ bradycardia (slow heart rate) in 40 $\%$ of cases = $P(+bradicardia|fiebre_tifoidea)=0.4$
  tuberculosis $\rightarrow$ fever in 60 $\%$ of cases = $P(+fiebre|tuberculosis)=0.6$
  tuberculosis $\rightarrow$ tachycardia in 58 $\%$ of cases = $P(+taquicardia|tuberculosis)=0.58$
  Others $\rightarrow$ fever 1'5 $\%$ = $P(+fiebre|Otros)=0.015$
  Others $\rightarrow$ bradycardia 0'0005 $\%$ = $P(+bradicardia|Otros)=0.0005$
  Others $\rightarrow$ tachycardia 0'013 $\%$ = $P(+taquicardia|Otros)=0.013$
  With the information in the statement, we can conclude that the variables are:
  • Disease: A patient may have:
    • typhoid fever.
    • tuberculosis.
    • something else.
  • Fever: Absent or present.
  • Bradycardia: Absent or present.
  • Tachycardia: Absent or present.
• Draw the corresponding diagram for the naive Bayesian method.

  

  Diagram

• Indicate the conditional probabilities, in table form, that define the model.First of all, I will indicate the probabilities for each of the different diagnoses given in the statement.

  Probability for the different states of patients

  	Typhoid Fever	Tuberculosis	Other
  P(Disease)	0.001	0.01	0.989

  Next comes the probability that a patient with a given disease has fever ( $P(Fiebre|Enfermedad)$ )

  Probability for the different states of patients regarding the symptom fever

  	Typhoid Fever	Tuberculosis	Other
  fever	1	0.6	0.015
  $\neg$ fever	0	0.4	0.985

  Next comes the probability that a patient with a given disease has bradycardia ( $P(braquicardia|Enfermedad)$ )

  Probability for the different states of patients regarding the symptom bradycardia

  	Typhoid Fever	Tuberculosis	Other
  bradycardia	0.4	0	0.0005
  $\neg$ bradycardia	0.6	1	0.9995

  Next comes the probability that a patient with a given disease has tachycardia ( $P(taquicardia|Enfermedad)$ )

  Probability for the different states of patients regarding the symptom tachycardia

  	Typhoid Fever	Tuberculosis	Other
  tachycardia	0	0.58	0.013
  $\neg$ tachycardia	1	0.42	0.987

• State the hypotheses you are using to solve this problem and discuss whether they are reasonable or not, that is, whether they seem to be a good approximation.
  The hypothesis I deduced from the statement is that a patient can have three different types of diseases: typhoid fever, tuberculosis, or another disease that is neither typhoid fever nor tuberculosis.
  The approximation is reasonable, since it indicates the probability that a patient with certain symptoms has one or more diseases, making it a useful approximation to confirm or rule out possible diagnoses.
• What is the diagnosis for each of the possible combinations of findings: fever, no fever, tachycardia, bradycardia, normal rhythm, fever and tachycardia, fever and normal rhythm, etc.? If you wish, you may use the OpenMarkov program to complete the table, but in that case you must work out by hand and show the detailed calculations for two of those combinations.
  To calculate the probabilities that the patient suffers from a disease based on the symptoms, we will use Bayes' theorem through the following formula:
  e = disease
  f = fever
  t = tachycardia
  b = bradycardia
  $p(e|f,b,t)=\frac{p(f,b,t|e)*p(e)}{\sum_{e} {p(f,b,t|e)*p(e)}}$
  From the previous tables, we can generate a table that includes all possible combinations of $p(f,b,t|e)$ , that is, the conditional probability table for each disease.

  Conditional probability table for each disease

  Symptoms|Disease
  +f,+b,+t	0	0	0,0000000975
  +f,+b, $\neg$ t	0,4	0	0,0000074025
  +f, $\neg$ b,+t	0	0.348	0.000195
  +f, $\neg$ ,b, $\neg$ t	0.6	0.252	0.01479
  $\neg$ f,+b, + t	0	0	0,0000064025
  $\neg$ f,+b, $\neg$ t	0	0	0,000486098
  $\neg$ f, $\neg$ b, + t	0	0.232	0.0128
  $\neg$ f, $\neg$ b, $\neg$ t	0	0.168	0.9717

  In this way, using equation 1 we can calculate the different probabilities of a given diagnosis conditioned on the different combinations of symptoms.
  $p(tuberculosis|+f,-b, +t)= \frac{p(+f,-b,+t|tuberculosis)*p(tuberculosis)}{\sum_{e}P(p(+f,+b,+t|e')*p(e')}\\=\frac{0.348*0.01}{0.348*0.01+0*0.001+0.000195*0.989}=\frac{0.348*0.01}{0,003672855 }=0,947\\\\$
  $p(tuberculosis|+f,-b, +t)= \frac{p(+f,-b,+t|tuberculosis)*p(tuberculosis)}{\sum_{e}P(p(+f,+b,+t|e')*p(e')}\\=\frac{0.348*0.01}{0.348*0.01+0*0.001+0.000195*0.989}=\frac{0.348*0.01}{0,003672855 }=0,947\\\\$

  Symptoms|Disease
  +f,+b,+t	0	0	1
  +f,+b, $\neg$ t	0,98	0	0.02
  +f, $\neg$ b,+t	0	0.95	0.05
  +f, $\neg$ ,b, $\neg$ t	0.03	0.12	0.82
  $\neg$ f,+b, + t	0	0	1
  $\neg$ f,+b, $\neg$ t	0	0	1
  $\neg$ f, $\neg$ b, + t	0	0.15	0.085
  $\neg$ f, $\neg$ b, $\neg$ t	0	0.01	0.99

• Experience shows that, when there is tuberculosis, fever and tachycardia are associated in most cases that is, tuberculosis generally produces tachycardia if and only if it produces fever. Does this observation call into question the validity of the results obtained in the previous section?

No, because if tachycardia is present, fever is present and bradycardia is absent, the probability that a patient has tuberculosis is 95%, which seems to indicate that the model could be valid based on experience.