Estimasi rob. Desity Fuctio dega EM Sumber: -Forsyth & oce Chap. 7 -Stadford Visio & Modelig robability Desity Estimatio arametric Represetatios o-arametric Represetatios Miture Models age
Metode estimasi o-parametric Tapa asumsi apapu tetag distribusi Estimasi sepeuhya bergatug ada DATA cara mudah megguaa: Histogram Histograms Disritisasi, latas ubah dalam betu batag: age
Histograms Butuh omputasi baya, amu sagat umum diguaa Dapat diterapa pada sembarag betu desitas arbitrary desity Histograms ermasalaha: Higher dimesioal Spaces: - umlah batag bis yg. Epoetial - umlah traiig data yg epoetial - Curse of Dimesioality size batag? Terlalu sediit: >> asar Terlalu baya: >> terlalu halus age 3 3
edeata secara prisip: diambil dari uow p probabiliti bahwa ada dalam regio R adalah: p ' d' p V R edeata secara prisip: diambil dari uow p probabiliti bahwa ada dalam regio R adalah: R K p ' d' p V age 4 4
edeata secara prisip: diambil dari uow p probabiliti bahwa ada dalam regio R adalah: R K p ' d' p V p K V edeata secara prisip: p K V Dega Fi V Tetua K Metoda Kerel-Based K-earest eighbor Dega Fi K Tetua V age 5 5
Metoda Kerel-Based: p K V arze Widow: u < / H u 0 otherwise Metoda Kerel-Based: p K V arze Widow: u < / H u 0 otherwise K H age 6 6
Metoda Kerel-Based: p K V arze Widow: u < / H u 0 otherwise K H p h H d Metoda Kerel-Based: p K V Gaussia Widow: p πh d / ep h age 7 7
Metoda Kerel-Based: K-earest-eighbor: eighbor: p K V Kembaa V sampai dia mecapai K poits. age 8 8
K-earest-eighbor: eighbor: K-earest-eighbor: eighbor: Klasifiasi secara Bayesia : p C p p C K V K V age 9 9
K-earest-eighbor: eighbor: Klasifiasi secara Bayesia : p C p p C K V K V p C K K atura lasifiasi -earest-eighbour robability Desity Estimatio arametric Represetatios o-arametric Represetatios Miture Models Model Gabuga age 0 0
Miture-Models Models Model Gabuga: Gaussias: - Mudah -Low Memory - Cepat - Good roperties o-arametric: -Umum - Memory Itesive -Slow Miture Models Campura fugsi Gaussia miture of Gaussias: p Jumlah dari Gaussias tuggal age
Campura fugsi Gaussia: p Jumlah dari Gaussias tuggal Keuggula: Dapat medeati betu desitas sembarag Arbitrary Shape Campura fugsi Gaussia: p Geerative Model: z 3 p age
Campura fugsi Gaussia: p p M p p ep σ d / πσ Campura fugsi Gaussia: Maimum Lielihood: E l L l p age 3 3
Campura fugsi Gaussia: Maimum Lielihood: E l L l p E 0 E Campura fugsi Gaussia: Maimum Lielihood: E l L l p E 0 age 4 4
5 age 5 Campura Campura fugsi fugsi Gaussia: Gaussia: Campura Campura fugsi fugsi Gaussia: Gaussia: M p p
6 age 6 Campura Campura fugsi fugsi Gaussia: Gaussia: / ep d p σ πσ M p p Campura Campura fugsi fugsi Gaussia: Gaussia: / ep d p σ πσ M p p
Campura fugsi Gaussia: Maimum Lielihood: E l L l p Tida ada solusi pede! E 0 E Campura fugsi Gaussia: Maimum Lielihood: E l L l p E Gradiet Descet age 7 7
Campura fugsi Gaussia: Maimum Lielihood: E l L l p E f,..., M, σ,..., σ M, α,..., α M Campura fugsi Gaussia: Optimasi secara Gradiet Descet: Comple Gradiet Fuctio highly oliear coupled equatios Optimasi sebuah Gaussia tergatug dari seluruh campura laiya. age 8 8
Campura fugsi Gaussia: -> Dega strategi berbeda: p Observed Data: Campura fugsi Gaussia: p Desitas yg dihasila Observed Data: age 9 9
Campura fugsi Gaussia: p Variabel Hidde y Observed Data: Campura fugsi Gaussia: p Variabel Hidde y Observed Data: Uobserved: y age 0 0
Cotoh populer ttg.. Chice ad Egg roblem: p Ma.Lielihood Ut. Gaussia # Ma.Lielihood Ut. Gaussia # Aggap ita tahu y Chice+Egg roblem: Aggap ita tahu p y y y age
Chice+Egg roblem: p Tapi yg ii ita tida tau sama seali?? y Chice+Egg roblem: p Coba pura tahu y age
Clusterig: Tebaa bear? y K-mea clusterig / Basic Isodata egelompoa Clusterig: rocedure: Basic Isodata,...,. Choose some iitial values for the meas M Loop:. Classify the samples by assigig them to the class of the closest mea. 3. Recompute the meas as the average of the samples i their class. 4. If ay mea chaged value, go to Loop; otherwise, stop. age 3 3
Isodata: Iisialisasi Isodata: Meyatu Covergece age 4 4
Isodata: Beberapa permasalaha Diteba Eggs / Terhitug Chice p Ma.Lielihood Ut. Gaussia # Ma.Lielihood Ut. Gaussia # Disii ita berada y age 5 5
GaussiaAproimasi yg. bai p amu tida optimal! ermasalaha: Highly overlappig Gaussias Epectatio Maimizatio EM EM adalah formula umum dari problem seperti Chice+Egg Mi.Gaussias, Mi.Eperts, eural ets, HMMs, Bayes-ets, Isodata: adalah cotoh spesifi dari EM Geeral EM for mi.gaussia: disebut Soft-Clusterig Dapat overge meadi Maimum Lielihood age 6 6
7 age 7 Igat Igat rumusa rumusa ii ii?:?: / ep d p σ πσ M p p Soft Chice ad Egg roblem: Soft Chice ad Egg roblem: p 0. 0.3 0.7 0. 0.0 0.000 0.99 0.99 0.99 0.5 0.00 0.0000
Soft Chice ad Egg roblem: p Aggap ita tahu: Weighted Mea of Data 0. 0.3 0.7 0. 0.0 0.000 0.99 0.99 0.99 0.5 0.00 0.0000 Soft Chice ad Egg roblem: p Step-: Hitug ulag posteriors 0. 0.3 0.7 0. 0.0 0.000 0.99 0.99 0.99 0.5 0.00 0.0000 age 8 8
Lagah prosedur EM: rocedure: EM. Choose some iitial values for the meas,..., M E-Step:. Compute the posteriors for each class ad each sample: M-Step: 3. Re-compute the meas as the weighted average of their class: 4. If ay mea chaged value, go to Loop; otherwise, stop. EM da Gaussia miture θ arg ma Q θ, θ i i θ i p, θ p, θ i i age 9 9
30 age 30 EM EM da da Gaussia miture Gaussia miture, arg ma i i Q θ θ θ θ i T i i i i p p,, θ θ EM EM da da Gaussia miture Gaussia miture, arg ma i i Q θ θ θ θ i i p, θ α
Cotoh-cotoh EM: Traiig Samples Cotoh-cotoh EM: Traiig Samples Iitializatio age 3 3
Cotoh-cotoh EM: Traiig Samples Ed Result of EM Cotoh-cotoh EM: Traiig Samples Desity Isocotours age 3 3
Cotoh-cotoh EM: Color Segmetatio Cotoh-cotoh EM: Yair Weiss Layered Motio age 33 33