24/0/202 Tim teknis pass fail competent incompetent Avoid false positive or false negative
24/0/202 A standard is a statement about whether performance is good enough for a particular purpose The standard defines the boundary between passing and failing; separates candidates who are competent from those who are not competent The numerical answer to: How much is enough? How good is good enough? A Policy Decision The standard is based on judgments about examinees performances. Judgments are an expression of our professional, social and educational values. 2
24/0/202 C H A R A C T E R I S T I C S The method has to be: Defensible Credible Supported by a body of evidence Feasible Acceptable to all stakeholders 6 Langkah Penentuan Batas Kelulusan. Menentukan tipe standar 2. Menentukan metode penentuan nilai batas kelulusan 3. Memilih judges 4. Proses penentuan batas kelulusan oleh judges 5. Menghitung batas kelulusan 6. Menentukan follow up (Norcini, 2003) 3
24/0/202. Menentukan tipe standar Relative standards Norm-referenced Patokan Acuan Normal / PAN Absolute standards Criterion-referenced Patokan Acuan Baku / PAB Relative standardsfeatures Absolute standards Dinyatakan dalam jumlah atau persentase peserta yang lulus (mis: 00 peserta terbaik, 30% terbaik) Performan seorang peserta tergantung pada performan peserta lain Menstimulus kompetisi Lebih mudah dilakukan Tidak tepat untuk ujian berbasis kompetnsi Dinyatakan dalam persentase jawaban benar (mis: 65% jawaban benar) Performan seorang peserta tidak tergantung pada performan peserta lain Menstimulus kolaborasi Lebih sulit dilakukan Tepat untuk ujian berbasis kompetensi 4
24/0/202 2. Menentukan metode penentuan nilai batas kelulusan Relative standards Absolute standards Relative methods Absolute methods. Keputusan didasarkan pada item soal 2. Keputusan didasarkan pada peserta ujian Compromise Methods Contoh Hofstee s method 3. Memilih judges Kredibilitas nilai batas kelulusan sangat tergantung pada kredibilitas panel judges, sehingga perlu diperhatikan:. Kapabilitas, komitmen dan kredibilitas judges 2. Aspek keterwakilan dari judges 5
24/0/202 Step: 4. Proses penentuan batas kelulusan oleh judges 5. Menghitung batas kelulusan 6. Menentukan follow up STANDARD SETTING DALAM OSCE 6
24/0/202 Borderline Regression Method Increasingly popular in OSCE and performance based examination In each station each examinee is assessed using two methods Check list based method Global performance Data from each station is aggregated A regression equation is developed using global performance is independent variable and check list marks as dependent variable Cut-mark (standard) is the boundary between borderline and pass Global mark Profesional Judgement : tentang kompetensi dan penampilan kandidat secara keseluruhan dalam station tertentu Global score sebaiknya tidak berkaitan/ independen dengan marking sheet / checklist Penguji memutuskan apakah seorang kandidat melakukan semua instruksi secara fluently dan melakukan semua ketrampilan secara independent 7
24/0/202 Borderline Regression Method Items Full mark Mark Explain to pt Position pt Inspection of skin 2 Temperature change Femoral Popliteal Post. tibial Dors. pedis Capillary refill Total 0 2 0 9 Put the global rating from the overall performance in OSCE Fail () Borderline (2) Pass (3) Outstanding (4) a Borderline Regression Method Items Full mark Mark Explain to pt Position pt Inspection of skin 2 Temperature change Pulsation Pulsation Femoral Popliteal Post. tibial Dors. pedis Capillary refill Total 0 0 0 0 0 5 Put the global rating from the overall performance in OSCE Fail () Borderline (2) Pass (3) Outstanding (4) a 8
24/0/202 excellent Excellent performance of skill. Outstanding demonstration of technical and non-technical aspects of skill. Air of confidence and fluent Very good pass Very good performance of skill. Majority of the technical aspects of the skill demonstrated. Few minor and non-essential omission/errors. Examiner more than satisfied that candidate has passed station 9
24/0/202 Clear pass Acceptable performance of skill. Despite omission /errors demonstrated in performance of skill-safe to progress. At times can be formulaic in approach. Borderline : pass doubtful Patchy performance of skill. Examiner undecided whether to pass or fail candidate. Demonstrated some aspects of the skills however omissions and inaccuracies occurred in their performance of the skill. Often formulaic in approach and struggled with performing skill 0
24/0/202 Clear fail Performance of skill did not come up to a passing standard. Appeared disorganised. Unsafe and unsuitable to progress. Borderline Regression Methods
24/0/202 Global Rating dalam Checklist OSCE UKDGI 3 = Pass (excellent, very good) 2 = Borderline (doubtful) = Fail (clear fail) Deskripsi global mark OSCE UKDGI Pass/lulus : penampilan kandidat dalam mengerjakan semua instruksi cukup lancar, tidak melakukan kesalahan yang fatal dan menggunakan hampir semua alat atau bahan yang essential dalam satu tindakan Borderline/pass doubtful : penampilan kandidat mengalami banyak keragu-raguan, terdapat beberapa kesalahan dalam melakukan tindakan sehingga penguji mengalami keraguan dalam memberi nilai 2
24/0/202 Fail / tidak lulus : penampilan dalam melakukan suatu tindakan tidak memenuhi persyaratan standar, tidak terorganisir, tidak aman CONTOH SCORE CANDIDAT PADA STATION 7 TRY OUT OSCE OKT 20 NO ITEM FULL MARK Memperkenalkan 2 0 diri 2 Identitas pasien 2 2 3 Keluhan utama 2 2 4 Riwayat penyakit 2 0 saat ini 5 Riwayat gigi tiruan 2 2 6 Riwayat penyakit 2 0 7 Rencana 2 2 perawatan TOTAL SCORE 4 8 MARK Put the global rating from the overall performance in OSCE Fail () Borderline (2) Pass (3) V 3
Total Score 24/0/202 6 STATION 7 4 2 R square : 0.66 0 8 7,96 6 Total Score Predicted Total Score Linear (Predicted Total Score) 4 2 0 0 2 3 Performance Scale 4