Core Team. IBA Vice Manager : Utmi Arma. Asset Manager : Kosterman Usri 24/10/2012

Transkripsi

1 PREPARATION FOR IMPLEMENTATION OF THE NATIONAL COMPETENCY EXAMINATION WITH CBT AND OSCE MELANIE SADONO DJAMIL Component 2 HPEQ Project INDONESIAN COLLEGE OF DENTIST DEAN FACULTY OF DENTISTRY TRISAKTI UNIVERSITY 28 October 2011 Core Team Chairman : Melanie Sadono Djamil Secretary : Wiwiek Poedjiastoeti OSCE Exam Manager : Mei Syafriadi Vice Manager OSCE Exam : Haslinda Tamin Member OSCE Exam : Anna Muryani IBA Manager : Indri Kurniasih IBA Vice Manager : Utmi Arma CBT Manager : Iwan Dewanto Asset Manager : Kosterman Usri R&D Manager : Gilang Jubiliana Vice Manager : Mia Damiyanti 1

2 Presentation Outline Background Process Output 1. Problem in health services 2. Legal aspect 1. Blue print development 2. Development & Review 3. Implementation of NCE 1. Result 2. Impact 3. Future direction 2

3 BACKGROUND PRACTICE OF MEDICINE LAWS COLLEGE PUBLISH CERTIFICATE OF COMPETENCY HPEQ Project INDONESIAN COLLEGE OF DENTIST (ICD) RUNNING THE INDONESIAN DENTIST COMPETENCY EXAMINATION (IDCE) Since, 3 April 2007 Implementation of NACE/ IDCE Ensure the quality of graduates ICD With CBT & OSCE 3

4 NCE as integral part of assessment system National Competence Examination Benchmarking Regional Assessment System in Institution - CBC implementation - 80% of curriculum content - CBC implementation - Academic phase (prediagnostic/treatment) - Formative - CBC implementation - 100% of curriculum - Requirement for graduation FACULTY OF DENTISTRY IN INDONESIA 4

5 HPEQ Project Development Review NACE Development WORKSHOPS 5

6 22-23 Mar Workshop National Development for PBT-IDCE May Workshop Standard Setting for Dentistry May Workshop of Regional Development for OSCE of Dentistry in West Region May Workshop of Regional Development for OSCE of Dentistry in East Region May Workshop of National Development of OSCE in Dentistry 9-10 June Workshop of OSCE Examiner in Dentistry of West Region June Workshop of OSCE Examiner in Dentistry of East Region June Workshop of National OSCE Examiner in Dentistry June Workshop of National Review for National Competency Examination July Workshop of Panel Expert part July Workshop of Regional SP trainer in East Region July Workshop of Regional SP trainer in West Region July Workshop of National SP Trainers August Workshop of Regional Dev OSCE in West Region (part 2) August Workshop of Regional Dev OSCE in East Region (part 2) October Workshop of Review part 2 2 Review ITEM BANK ADMINISTRATOR (IBA) 6

7 JANUARI APRIL JULI OKTOBER JANUARI UKDGI UKDGI UKDGI UKDGI UKDGI 24/10/2012 Workflow of Bank Administration Kedokteran G Institution writer review National Bank Administrator National Review item bank 2 Panel expert bank 1 National Bank Administrator selection process Question book (UKDGI) Bank 3 Time schedule IBA Yearly IBA IBA writer review inst review writer review IBA review writer review IBA review writer review inst review inst inst nas inst inst inst nas inst inst inst nas inst inst nas jan feb feb mar apr mei mei juni Juli agt agt sep okt nov nov des 7

8 Competence Base Curriculum In Dentistry kdgi-msd 10/24/2012 Domain-1. Professionalism Main competence 1. Ethics and jurisprudence: ethics and law 2. Analysis of critical health information, scientifically, and effectively 3. Communication skills 4. Socio-cultural relations in the field oral health care kdgi-msd 10/24/2012 8

9 Domain-2 Aplication of Basic Medical and Dental Sciences Main competence 5. Basic Medical Sciences 6. Clinical Medicine 7. Dental Basic Science 8. Clinical Dentistry kdgi-msd 10/24/2012 Domain-3 General physical examination and stomatognatic system Main competence 9. Examination of patients: physical, clinical, laboratory, radiological, psychological, and social; manage patient behavior, and using medical record Diagnosis Care Plan 10. Diagnosis 11. Care plan kdgi-msd 10/24/2012 9

10 Domain-4 Management the function of stomatognatic system Main competence 12. Management of pain and anxiety 13. Dentistry medical action: conservation treatment and restoration of deciduous teeth and adult; periodontal, orthodontic children and adults; Minor surgery in soft and hard tissue, non-surgical lesions, TMJ disorders and occlusion; Prosthodontics for children, adults, and elderly; manage emergency; Team work. kdgi-msd 10/24/2012 Domain-5 dental and oral health community Main competence 14. Doing community services in oral health: Diagnose problem oral health community. Promotive and preventive dentistry Information technologies Working in teams. 15. Management of behaviour kdgi-msd 10/24/

11 Domain-6 Management Practice in Dentistry Main competence 16. Management practices and work environment Restructure management practices and management practices the work environment in dentistry Organizing an ergonomic environment in dentistry with regard to the principles of work safety Applying the basic principles of practice management and its relationship with the social aspects kdgi-msd 10/24/2012 GENERAL BLUE-PRINT for DENTISTRY DOMAIN COMPETENCY MCQ (%) OSCE (%) 1. Professionalism Application of basic medical and dental sciences 3. General physical examination and stomatognatic system 4. Management of stomatognatic functions System Oral and dental health Management of dental practices

12 Blue print ID Fill in by Bank Administrator Range Percentage Ideal Percentage 1. Professionalism Application of basic medical and dental sciences Review General physical examination and stomatognatic system Management of stomatognatic functions System Oral and dental health Management of dental practices

13 13

14 Out Put 14

15 Jumlah Peserta dan Kelulusan Uji Kompetensi Examination Period Number of examinee Pass Mean Fail April 2007 (Uji coba) (100,00%) 0 (0%) July (92,55%) 26 (7,45%) October (91,12%) 43 (9,50%) January (91,12%) 27 (8,88%) April (87,60%) 38 (12,40%) 89,93 July (88,57%) 32 (11,43%) October (92,42%) 32 (8,58%) January (75,80%) 94 (24,20%) April (90,61%) 40 (9,39%) 78,77 July (79,68%) 76 (20,32%) October (68,97%) 153 (31,03%) T o t a l (81,61%) 621 (18,39%) Jumlah Peserta dan Kelulusan Uji Kompetensi Examination Period Number of examinee Pass Mean Fail January (87,21%) 60 (12,79%) April (51,14%) 278 (48,86%) 80,15 July (91,12%) 43 (9,50%) October (91,12%) 27 (8,88%) January (87,60%) 38 (12,40%) April (88,57%) 32 (11,43%) 83,98 July (92,42%) 32 (8,58%) October (75,80%) 94 (24,20%) January (76,38%) 81(23,62%) T o t a l (83,25%) 621 (13,75%) * Uji coba tidak ikut dihitung 15

16 s total from all Institution in Indonesia (2010) NO Institutions Name I (July) II (NOV) III (DES) Number of (format) NO INSTITUTIONS NAME I (July) 1 USU * UNSYIAH 93* 20* II (NOV) III (DES) Number of item (format) 2 BAITURAHMAH UNAND 1 20* 1 3 UI UNSRI 100* USAKTI Maranatha MOESTOPO * UNJANI UNPAD UNSOED - 10* - 7 UGM 30* UMY UNAIR UNISULA UHT Bhakti Wiyata UNEJ UB - 21* - 11 UNHAS UNLAM UNMAS UNSRAT UNPRIMA TOTAL Total s from each Institution (2011) N O NAMA INSTITUSI I JUNI- JULI NAMA INSTITUSI 1 USU 21 UNSYIAH 20 2 BAITURAHMAH 20 UNAND 20 3 UI 25 UNSRI 40 4 USAKTI 24 Maranatha 20 5 MOESTOPO 20 UNJANI 20 6 UNPAD 27 UNSOED 20 7 UGM 120 UMY 35 8 UNAIR 65 UNISULA 20 9 UHT 42 Bhakti Wiyata UNEJ 20 UB UNHAS 15 UNLAM UNMAS 20 UNSRAT 20 UNPRIMA 20 I (JUNI- JULI) ITEMS IN TOTAL =

17 Successful rate SUCCESSFUL RATE & MEAN IN EACH EXAMINATION PLACES 17

18 PREPARATION IN THE INSTITUTIONS Computer Based Test (CBT) development review (expertise in every subject) CBT coordinator Trained in every institution OSCE development review (expertise in every subject) OSCE Examiner & standard setting Standardize Patient trainers OSCE coordinator Trained in every institution 3 National Competency Examination CBT 18

19 WS 1 CBT Software processed by KDGI NACE s from IPDG review IBA system bankl UKDGI Panel expert Software Try out NCE Try Out UKDGI CBT Software System Koord. CBT Local IT CBT CBT center supervisor CBT UKDGI 2012 Standard setting Analisis Software by UKDGI OSCE 3 Penyelengaraan Uji OSCE 19

20 BLUE-PRINT AGREEMENT Security of s St. 1. 5x5 + 1x1 = 26 St. 2. 5x5 + 1x1 = 26 St. 3. 5x5 + 1x1 = 26 St. 4. 5x5 + 1x1 = 26 St. 5. 5x6 = 30 St. 6. 5x6 = 30 St. 7. 5x3 = 15 St. 8. 5x4 = 20 + TOTAL = 199 (items collected) For security prepare 3 items in each station 597 items 20

21 OSCE COORDINATOR (trained for every institution) Responsible to: Preparation Coordinating the examiner, standardize patients, SP trainer, item OSCE and infrastructures Facilities including all stations Examination rules Implementation Evaluation Documentation OSCE Workshop in Dentistry and output Date Place Activities Output Mei 2010 Palembang WS item writer Dev OSCE West region. Collected 9 items OSCE Mei 2010 Yogya WS item writer Dev OSCE East region Mei 2010 Surabaya WS National item writer Dev OSCE Collected 12 items OSCE Review: 9 Blue Print OSCE KG Eligible item OSCE collected 18 21

22 OSCE Workshop in Dentistry and output Date Place Activities Output 9-10 Juni 2010 Yogya-UMY WS Standard Setting & Examiner OSCE west region OSCE still not suit to the standard e.g in the check list Juni 2010 Bali-UNMAS WS Standard Setting & Examiner OSCE East region Juni 2010 Surabaya WS National Examiner OSCE Need to completed the item OSCE Kesepakatan PS yang digunakan di OSCE KG OSCE Workshop OSCE KG and output Date Place Activities Output Juli 2010 Medan-USU Makasar-UNHAS WS SP trainers OSCE West region WS SP trainers OSCE East region Perlu adanya pertemuan antara pelatih SP & item Dev Juli 2010 Batam WS Nasional Pelatih SP OSCE Agustus Padang Surabaya WS item writer Dev OSCE Gel. 2 Wil. Barat WS item writer Dev OSCE Gel. 2 Wil. Timur Kesepakatan- Kesepakatan &Koordinator Nasional Pelatih SP Terlampir (18 soal Tereview) 21 soal Tereview 22

23 OSCE Workshop OSCE KG and output Date Place Activities Output Agustus 2010 Yogya WS Koordinator OSCE 12 FK 13 FKG Tersusunnya konsep 1. Dokumen Pedoman Tatalaksana Penyelenggaraan OSCE Nasional 2. Dokumen Standar / Protokol kerja penyelenggaraan OSCE di tempat ujian 3. Dokumen Standar Fasilitas & SDM Untuk Penyelenggaraan OSCE Nasional OSCE Workshop OSCE KG and output Date Place Activities Output September 2010 Yogyakarta WS Nasional Koordinator OSCE dan CBT - Tersusunnya konsep konsep penyelenggaran OSCE Ked dan KG nasional 2-3 Oktober 2010 Medan WS Nasional item Dev OSCE Gelombang Oktober 2010 Surabaya WS Nasional item Dev OSCE Gelombang 3 Jumlah soal tereview: 16 dari kumpulan soal OSCE gel. 2 wil. Barat (3 set blue print ujian osce ) Jumlah soal tereview: 20 dari kumpulan soal OSCE gel. 2 wil. Barat (4 set blue print ujian osce TOTAL SOAL 32) -HPEQ Project

24 OSCE Workshop OSCE KG and output Date Place Activities Output Oktober 2010 Bandung WS Nasional Sinkronisasi Dev, examiner, SP Trainer dan Koordinator OSCE November 2010 Wilayah Barat: Jakarta- Usakti Wilayah Timur: Makasar- Unhas WS examiner ke 2 menggunakan 1 set paket soal yang telah tersedia Tereview 2 paket soal osce Total paket soal osce yang tersedia 4 paket soal Masih ditemukan kekurangan dalam 1.skenario soal soal Konser dan Perio, masih perlu direview ulang 2.Waktu yang dibutuhkan kandidat di setiap stasion masih belum konsisten Mei-Juni 2011 dan 30 Juni institusi KG yaitu: USU, Unbrah, UNSRI, UNPAD, UI, Moestopo, Trisakti, UMY, UGM, UHT, Unair, UNEJ, UNMAS, Survei kesiapan IPDG sebagai penyelenggaraan OSCE dan laporan hasil survey tahap I 13 IPDG menyatakan siap menjadi pusat penyelenggara OSCE mulai tahun 2011 kecuali UNSRI menyatakan siap OSCE Workshop OSCE KG and output Date Place Activities Output Juli institusi KG yaitu: Unsyiah, Unprim, Unand, UKM, Unjani, Unsoed, Unissula, IIK Bakti Wiyata, UB, Unsrat, Unlam, Survei kesiapan IPDG sebagai penyelenggaraan OSCE 11 institusi KG yaitu: Unsyiah, Unprim, Unand, UKM, Unjani, Unsoed, Unissula, IIK Bakti Wiyata, UB, Unsrat, Unlam, telah di survey Juli 2011 Hotel grand Sahid, Jakarta review soal CBT dan OSCE Soal OSCE terkumpul 35 Soal tetapi belum sempat di review 8-9 Agustus 2011 Hotel grand Sahid, Jakarta Laporan hasil Survey tahap II dan Grading hasil survey OSCE -HPEQ Project Masih diperlukan perbaikan dan peningkatan sarana dan prasarana serta mengadakan pelatihan 24

25 OSCE Workshop OSCE KG and output Date Place Activities Output September 2010 Yogyakarta WS Nasional Koordinator OSCE dan CBT - Tersusunnya konsep konsep penyelenggaran OSCE Ked dan KG nasional 2-3 Oktober 2010 Medan WS Nasional item Dev OSCE Gelombang 2 Jumlah soal tereview: 16 dari kumpulan soal OSCE gel. 2 wil. Barat (3 set blue print ujian osce ) Oktober 2010 Surabaya WS Nasional item Dev OSCE Gelombang 3 -HPEQ Project Jumlah soal tereview: 20 dari kumpulan soal OSCE gel. 2 wil. Barat (4 set blue print ujian osce TOTAL SOAL 32) OSCE Workshop OSCE KG and output Date Place Activities Output Oktober 2010 Bandung WS Nasional Sinkronisasi Dev, examiner, SP Trainer dan Koordinator OSCE November 2010 Wilayah Barat: Jakarta- Usakti Wilayah Timur: Makasar- Unhas WS examiner ke 2 menggunakan 1 set paket soal yang telah tersedia Tereview 2 paket soal osce Total paket soal osce yang tersedia 4 paket soal Masih ditemukan kekurangan dalam 1.skenario soal soal Konser dan Perio, masih perlu direview ulang 2.Waktu yang dibutuhkan kandidat di setiap stasion masih belum konsisten Mei-Juni 2011 dan 30 Juni institusi KG yaitu: USU, Unbrah, UNSRI, UNPAD, UI, Moestopo, Trisakti, UMY, UGM, Survei kesiapan IPDG sebagai penyelenggaraan OSCE dan laporan hasil survey tahap I 13 IPDG menyatakan siap menjadi pusat penyelenggara OSCE mulai tahun 2011 kecuali 25

26 OSCE Workshop OSCE KG and output Date Place Activities Output Juli institusi KG yaitu: Unsyiah, Unprim, Unand, UKM, Unjani, Unsoed, Unissula, IIK Bakti Wiyata, UB, Unsrat, Unlam, Survei kesiapan IPDG sebagai penyelenggaraan OSCE 11 institusi KG yaitu: Unsyiah, Unprim, Unand, UKM, Unjani, Unsoed, Unissula, IIK Bakti Wiyata, UB, Unsrat, Unlam, telah di survey Juli 2011 Hotel grand Sahid, Jakarta review soal CBT dan OSCE Soal OSCE terkumpul. Soal tetapi belum sempat di review 8-9 Agustus 2011 Hotel grand Sahid, Jakarta Laporan hasil Survey tahap II dan Grading hasil -HPEQ survey OSCE Project Masih diperlukan perbaikan dan peningkatan sarana dan prasarana serta Survey for OSCE Center in Dentistry 26

27 Evaluasi per station Station 4 Penguji tidak mengisi Global Rating Scale dengan lengkap Penguji tidak mengisi ceklis Station 3 Penguji tidak mengisi Global Rating Scale dengan lengkap Station 2 Penguji tidak mengisi Global Rating Scale dengan lengkap Penguji tidak mengisi ceklis Station 1 Penguji tidak mengisi ceklis Evaluasi per station Station 8 Penguji tidak mengisi Global Rating Scale dengan lengkap Station 7 Penguji tidak mengisi ceklis Penguji tidak memahami Global Rating Scale dengan baik Tugas dan ceklis tidak sejalan (resep) Station 6 Penguji tidak mengisi Global Rating Scale dengan lengkap Penguji tidak mengisi ceklis dengan lengkap Station 5 Penguji tidak memahami Global Rating Scale dengan baik 27

28 Penguji Tidak mengisi CEKLIST Tidak mengisi Global Performance Scale (GPS) Tidak memahami defenisi BORDERLINE Mempengaruhi standard setting Metode Standard Setting Borderline Regression Method 28

29 Draft-Standar Setting Peny. Akibat Trauma & Kecelakaan Peny. Infeksi dan imunologi Peny. Pertumbuhan & perkembangan Peny. Neoplastik & non neoplastik Penyakit degeneratif & compromised medis Lulus Station 1 atau 2 Lulus station 3 atau 4 Lulus Station 5 atau 6 Lulus station 7 Lulus station 8 29

30 Grafik Persentase Kelulusan OSCE UKDGI Periode Januari 2012 Tabel 1: Rekapitulasi Persentase Kelulusan OSCE UKDGI Periode 2012 Per Station berdasarkan Institusi No Institusi Station 1 Station 2 Station 3 Station 4 Station 5 Station 6 Station 7 Station 8 NBL= 22 NBL= 19 NBL= 7 NBL= 7 NBL= 6 NBL= 15 NBL= 6 NBL= 10 Jml L % Jml L % Jml L % Jml L % Jml L % Jml L % Jml L % Jml L % 1 UGM % % % % % % % % 2 UHT % % % % % % % % UI % % % % % % % % 3 4 UMY % % 3 0 0% % % % % % 5 UNAIR % % % % % % % % 6 UNBRAH % % % % % % % % UNEJ % % % % % % % % 7 8 UNHAS % % % % % % % % 9 UNMAS % % % % % % % % 10 UNPAD % % % % % % % % 11 UPDM(B) + UNSRI % % % % % % % % 12 USAKTI % % % % % % % % 13 USU % % % % % % % % T o t a l % % % % % % % % 30

31 120% Station 1 120% Station 2 100% 100% 80% 80% 60% 60% 40% 40% 20% 20% 0% % % Station 3 120% Station 4 100% 100% 80% 80% 60% 60% 40% 40% 20% 20% 0% %

32 120% Station 5 120% Station 6 100% 100% 80% 80% 60% 60% 40% 40% 20% 20% 0% % % Station 7 120% Station 8 100% 100% 80% 80% 60% 60% 40% 40% 20% 20% 0% %

33 Rencana Tindak Lanjut Workshop Review OSCE Institusi & Nasional Pelatihan Penguji Tingkat Nasional dan Institusi Pelatihan Standard Setting OSCE THANK YOU PDGI (INDONESIAN DENTAL ASSOCIATION) AFDOKGI (ASSOCIATION OF INDONESIA FACULTY OF DENTISTRY) CORE TEAM HPEQ PROJECT IN DENTISTRY 33