KI Kecerdasan Buatan Materi 13: Learning Probabilistic Models (Bayesian Network)

Transkripsi

1 [AIMA] Russel, Stuart J., Peter Norvig, "Artificial Intelligence, A Modern Approach" 3rd Ed., Prentice Hall, New Jersey, 2010 KI Kecerdasan Buatan Materi 13: Learning Probabilistic Models (Bayesian Network) Teknik Informatika, Fakultas Teknologi Informasi Institut Teknologi Sepuluh Nopember Surabaya 2012 Pengembangan Bahan Ajar sebagai Pendukung Student Centered-Learning (SCL) melalui e-learning : Share ITS

2 Daftar Materi 1. Pengenalan Intelligent Agents, Uninfomed Search (pencarian tanpa informasi tambahan) 2. Informed Search (Heuristics) 3. Local Search & Optimization (pencarian dengan optimasi) 4. Adversarial Search (pencarian dengan melihat status pihak lainnya) 5. Constraint Satisfaction Problems (pencarian dengan batasan kondisi) 6. Reasoning: propositional logic 7. Reasoning: predicate logic 8. Reasoning: first order logic 9. Reasoning: inference in first order logic (forward-backward chaining untuk inference) 10. Learning Probabilistic Models (1+2) 2

3 Latar Belakang Learning Probabilistic Models = solusi yang diambil dari ketidakpastian kondisi Kurangnya informasi di problem nyata Solusi dari problem nyata dapat ditemukan jika ada pembelajaran kemungkinan kejadian Langkah praktis: buat representasi problem nyata menjadi model dugaan berdasarkan probabilitas (probabilistic inference ) Pendekatan dasar: teknik Bayesian 3

4 Ketidakpastian (Uncertainty) Didefinisikan A t = ke bandara t menit sebelum penerbangan Apakah kejadian A t membuat penumpang tidak terlambat? Permasalahan: jalan macet, ban bocor dan semacamnya Kemungkinan dugaan kejadian: A 25 memungkinkan penumpang tidak terlambat penerbangan JIKA tidak ada kecelakaan di jalan, tidak ada kemacetan, tidak ada hujan, tidak terjadi ban bocor, dll. (A 1440 juga memungkinkan tepat waktu tetapi harus bermalam hari sebelumnya di hotel bandara )

5 Keputusan dengan Ketidakpastian Kondisi Semisal dugaan kemungkinan P(A 25 jadi on time ) = 0.04 P(A 90 jadi on time ) = 0.70 P(A 120 jadi on time ) = 0.95 P(A 1440 jadi on time ) = Keputusan yang diambil bergantung pada preferences untuk missing flight vs. waktu tunggu di bandara, etc. Utility theory is used to represent and infer preferences Decision theory = probability theory + utility theory

6 Probabilitas Probabilitas koin (bagian depan, head, dan belakang, tail) P(COIN = tail) = 1 2 P(tail) = 1 2 6

7 Probabilitas Probabilitas penderita kanker P(has cancer) = 0.02 Þ P(Ø has cancer) =

8 Probabilitas Gabung, Joint Probability Variasi kejadian (event): cancer, test result Berdasarkan data di lapangan (misal 1000 pasien) P(has cancer, test positive) Has cancer? Test positive? P(C,TP) yes yes yes no no yes no no org + dari hasil tes 784 org - dari hasil tes 8

9 Probabilitas Bersyarat, Conditional Probability Pertanyaan diagnosa: seberapa mungkin indikasi kanker ditemukan dengan tes? P(has cancer test positive) =? Has cancer? Test positive? P(TP, C) yes yes yes no no yes no no P(C TP) = P(C, TP) / P(TP) = / =

10 Bayes Network (BN) vs Independence MODEL BN: P(cancer) and P(Test positive cancer) PREDICTION BN: Hitung P(Test positive) Bayes Network Cancer P(C, TP) P(C) P(TP) Independence Cancer DIAGNOSTIC REASONING BN: Hitung P(Cancer test positive) versus Test positive Test positive 10

11 Notasi Grafik: Coin N independent coin flips X 1 X 2 X n Tidak ada ketergantungan untuk setiap kejadian pelemparan: absolute independence 11

12 Example: Coin Flips X 1 X 2 X n h 0.5 t 0.5 h 0.5 t 0.5 h 0.5 t 0.5 Only distributions whose variables are absolutely independent can be represented by a Bayes net with no arcs. 12

13 Notasi Grafik: Traffic Variables: R: It rains T: There is traffic Model 1: independence Model 2 (better): rain causes traffic R T Model 2 lebih baik karena lebih sesuai dengan kondisi nyata 13

14 Example: Traffic R +r 1/4 r 3/4 T +r +t 3/4 t 1/4 r +t 1/2 t 1/2 14

15 Independence 2 variabel disebut independent jika: Dengan kata lain: Tertulis : 15

16 Contoh: Independence N pelemparan koin (fair, independent): h 0.5 t 0.5 h 0.5 t 0.5 h 0.5 t

17 Contoh Lain: Independence? T = temperature W = weather T W P warm sun 0.4 warm rain 0.1 cold sun 0.2 cold rain 0.3 T P warm 0.5 cold 0.5 W P sun 0.6 rain 0.4 T W P warm sun 0.3 warm rain 0.2 cold sun 0.3 cold rain

18 Contoh Bayes Network: Sakit Gigi Cavity P(cavity) 1 parameter P(catch cavity) P(toothache cavity) 2 parameters 2 parameters Catch Headache Versus: = 7 parameters P(toothache) = =

19 Conditional Independence P(Toothache, Cavity, Catch) has 23 1 = 7 independent entries If I have a Toothache, a dental probe might be more likely to catch But: if I have a cavity, the probability that the probe catches doesn't depend on whether I have a toothache: P(+catch +toothache, +cavity) = P(+catch +cavity) The same independence holds if I don t have a cavity: P(+catch +toothache, cavity) = P(+catch cavity) Catch is conditionally independent of Toothache given Cavity: P(Catch Toothache, Cavity) = P(Catch Cavity) Equivalent statements: P(Toothache Catch, Cavity) = P(Toothache Cavity) P(Toothache, Catch Cavity) = P(Toothache Cavity) P(Catch Cavity) One can be derived from the other easily Tertulis : 19

20 Contoh Bayes Network: Rumput & Air 20

21 Contoh Bayes Network: Rumput & Air Apabila fakta bahwa kondisi rumput adalah basah, maka bagaimana kemungkinan terjadinya sprinkler menyala, dan kemungkinan terjadinya hujan. normalizing constant kemungkinan

22 Contoh Bayes Network: Car 22

23 Contoh: Alarm Network Variables B: Burglary A: Alarm goes off M: Mary calls J: John calls E: Earthquake! P(J M) = P(J)? No P(A J, M) = P(A J)? P(A J, M) = P(A)? No P(B A, J, M) = P(B A)? Yes P(B A, J, M) = P(B)? No P(E B, A,J, M) = P(E A)? No P(E B, A, J, M) = P(E A, B)? Yes Burglary John calls Alarm Earthquake Mary calls 23

24 Example: Alarm Network B P(B) +b Burglary Earthqk E P(E) +e b e John calls A J P(J A) +a +j 0.9 +a j 0.1 a +j 0.05 a j 0.95 Alarm Mary calls A M P(M A) +a +m 0.7 +a m 0.3 a +m 0.01 a m 0.99 B E A P(A B,E) +b +e +a b +e a b e +a b e a 0.06 b +e +a 0.29 b +e a 0.71 b e +a b e a 0.999

25 Bayes Net Semantics A set of nodes, one per variable X A directed, acyclic graph A conditional distribution for each node A collection of distributions over X, one for each combination of parents values A 1 X A n CPT: conditional probability table Description of a noisy causal process A Bayes net = Topology (graph) + Local Conditional Probabilities 25

26 Probabilities in BNs Bayes nets implicitly encode joint distributions As a product of local conditional distributions To see what probability a BN gives to a full assignment, multiply all the relevant conditionals together: Example: This lets us reconstruct any entry of the full joint Not every BN can represent every joint distribution The topology enforces certain conditional independencies 26

27 Bayes Nets A Bayes net is an efficient encoding of a probabilistic model of a domain Questions we can ask: Inference: given a fixed BN, what is P(X e)? Representation: given a BN graph, what kinds of distributions can it encode? Modeling: what BN is most appropriate for a given domain? 27

28 Causal Chains This configuration is a causal chain X Y Z X: Low pressure Y: Rain Z: Traffic Is X independent of Z given Y? Evidence along the chain blocks the influence Yes! 28

29 Common Cause Another basic configuration: two effects of the same cause Are X and Z independent? Are X and Z independent given Y? X Y Z Y: Alarm X: John calls Yes! Z: Mary calls Observing the cause blocks influence between effects. 29

30 Common Effect Last configuration: two causes of one effect (v-structures) Are X and Z independent? Yes: the ballgame and the rain cause traffic, but they are not correlated Still need to prove they must be (try it!) Are X and Z independent given Y? No: seeing traffic puts the rain and the ballgame in competition as explanation? This is backwards from the other cases Observing an effect activates influence between possible causes. X Z Y X: Raining Z: Ballgame Y: Traffic 30

31 The General Case Any complex example can be analyzed using these three canonical cases General question: in a given BN, are two variables independent (given evidence)? Solution: analyze the graph 31

32 Reachability Recipe: shade evidence nodes Attempt 1: Remove shaded nodes. If two nodes are still connected by an undirected path, they are not conditionally independent L R B Almost works, but not quite Where does it break? Answer: the v-structure at T doesn t count as a link in a path unless active D T 32

33 Reachability (D-Separation) Question: Are X and Y conditionally independent given evidence vars {Z}? Yes, if X and Y separated by Z Look for active paths from X to Y No active paths = independence! A path is active if each triple is active: Causal chain A B C where B is unobserved (either direction) Common cause A B C where B is unobserved Common effect (aka v-structure) A B C where B or one of its descendents is observed All it takes to block a path is a single inactive segment Active Triples Inactive Triples

34 Example Yes R B T T 34

35 Example L Yes Yes R B D T Yes T 35

36 Example Variables: R: Raining R T: Traffic D: Roof drips S: I m sad Questions: T S D Yes 36

37 Causality? When Bayes nets reflect the true causal patterns: Often simpler (nodes have fewer parents) Often easier to think about Often easier to elicit from experts BNs need not actually be causal Sometimes no causal net exists over the domain End up with arrows that reflect correlation, not causation What do the arrows really mean? Topology may happen to encode causal structure Topology only guaranteed to encode conditional independence 37