Simple model of competence

Transkripsi

1 Professional authenticity 24/08/2012 PEMBUATAN SOAL & INSTRUMEN EVALUASI KETERAMPILAN MEDIK DENGAN OSCE OSCE 1 Simple model of competence Does Shows how Knows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. OSCE 2 1

2 Testing formats Behaviour~ attitude/skills Does Professional practice Cognition~ knowledge Shows how Knows how Knows OSCEs EMQs, SEQs MCQs OSCE 3 Develop a Plan Make decisions about your OSCE - Summative? - Level of training being assessed? - Clinical objectives or mix of patient problems to be assessed? - Length and number of stations? - What competencis are being ass? (history taking, communication skills, problem solving etc) OSCE 4 2

3 First steps in Developing an OSCE case 1. Describe the purpose of the case (i.e. define the competencies you are testing) 2. Develop the case n instruction to the test takers 3. Create the scoring instruments (checklist, follow up questions and answer keys OSCE 5 Step 1: Purpose State the purpose of the station Demonstrate ability to perform an appropriate physical examination for abdominal pain secondary to appendicitis and be able to identify positive findings. Statement of purpose is your guide for what to include / exclude as you develop the case OSCE 6 3

4 Step 2: Instructions Write the case and instructions to the student Include patient s name, age, setting (e.g., emergency, clinic, hospital ward), relevant background information (if any); specify the task and time limit Example: Marie Beckett, 16 years old, has come to the Emergency Room with a 16-hour history of abdominal pain. In the next five minutes, conduct a focused and relevant physical examination. As you proceed, explain what you are doing and describe your findings. OSCE 7 Step 3: Scoring Instruments Checklists are useful when assessing Thoroughness or key elements Student (beginner) levels of ability Procedural tasks Limited time available for training markers and/or for marking Rating scales are useful when assessing Behaviours Emphasis is on how well it is done rather than on done / not done Higher levels of expertise Emphasis is on judgment rather than thoroughness To weight or not to weight -- keep it simple OSCE 8 4

5 How is it done? A Clinical competence to be tested is broken down into specific skills A Specific skill is tested at a time It is organized in the form of several stations Each station tests a particular skill For each skill, a checklist is prepared Checklist contains essential steps and precautions to be observed OSCE 9 How is it done? Each step has its own score proportional to the importance of the step/precaution Each component is tested at one fixed station by the same examiner The students rotate through several such locations Time allocated for each station: 4-7 minutes, minutes Stations 8-20 (UNJANI : 10 sts) OSCE 10 5

6 How is it done? Categories of the station : Procedure or performance station Question or interpretation station The examiner s checklists & student s answer sheets are marked according to predetermined scheme OSCE 11 MENGEMBANGKAN OSCE Menyusun Kisi-kisi Menyusun Soal/Skenario Melakukan Seting Station Koordinasi Dengan Observer Persiapan Pasien Simulasi Menyusun Check List Mengatur Pelaksanaan OSCE 12 6

7 Menyusun Soal/Skenario Kompetensi yang diharapkan (SKDI) Waktu tiap soal (station) Aspek yang terkait History taking skills Clinical examination Procedure skills Formulation of investigations Interpretation of investigations OSCE 13 Soal Topik Instruksi kandidat Instruksi penguji Daftar tilik instrumen penilaian Instruksi pasien simulasi Daftar peralatan Penulis Referensi OSCE 14 7

8 MENYUSUN DAFTAR TILIK Tiap item hanya menyangkut satu issue Gambaran rekaman perilaku mahasiswa ditulis dalam bentuk butir (item) Tiap item dapat diobservasi dan diberi nilai Daftar tilik untuk 10` cukup item Item yang PENTING perlu diberi tanda, dengan pembobotan yang lebih besar Hasil: NILAI / STATION = NILAI YANG DIPEROLEH X 100 TOTAL NILAI OSCE 15 Dokumen yang diperlukan penilai/pelaksana OSCE Daftar mahasiswa Daftar observer/penilai Daftar pasien simulasi/sukarelawan Daftar peralatan Instruksi untuk mahasiswa Instruksi untuk penilai Daftar seluruh stasiun Lembar jawaban Rencana lokasi stasiun dan arah rotasi OSCE 16 8

9 Pass Mark? Clinical procedural skills : 100 % (A) Physical examination : 100 % (A) History taking : 80 % (A) Interpretations : 80 % (A) Investigation : 80 % (A) Management : 80% (A) OSCE 17 Heteroanamnese dari ibu yang mempunyai anak sakit CONTOH UJIAN OSCE Melakukan prosedur aseptik Melakukan pemeriksaan hepar Pemeriksaan reflex Membaca hasil EKG Melakukan pemeriksaan kadar gula darah Evaluasi hasil laboratorium Anamneses penyakit kronis OSCE 19 9

10 Station 1 Station 2 Station 3 Station 4 Rest station Interstation Station 8 Station 7 Station 6 Station 5 I believe that teaching without testing is cooking without tasting (Ian Lang) OSCE 20 BLOOD PRESSURE MEASUREMENT Level of Student: Cardiovascular Block STATION INFORMATION CANDIDATE S INSTRUCTION 28 Des 07 OSCE 21 10

11 Blood Pressure Measurement Rating 28 Des 07 OSCE 22 STATION INFORMATION Domain Discipline The student to be assessed Problem + Diagnosis Purpose of Station Scoring Guidelines History taking Surgery The 2 nd year student Acute Urinary retention Demonstrate ability to perform an appropriate history taking for acute urinary retention secondary to benign prostatic hyperplasia and be able to identify positive findings Checklist 28 Des 07 OSCE 23 11

12 Candidate s Instructions Patient name: Mr Abdullah Age: 65 years Where: in Emergency Room Time allowed: in the next ten minutes Task: history taking Related Information: patient was complaining about inability to void. 28 Des 07 OSCE 24 Instruction to the Students: You are a medical student doing your surgical practice emergency. You are about to see Mr Abdullah, who has a voiding problem. In The first 6 minutes take a focused history of this patient s problem. At 6 minutes, you will hear a knock at the door. The examiner will then verbally ask you some questions regarding this patient. At the end of 8 minutes, the examiner will provide you with feedback on your performance 28 Des 07 OSCE 25 12

13 Checklist and rating Scale for Voiding Problem History Taking Case No Item Item weight 1 Greets and introduces self to patient in a professional manner 1 if performed 2 Asks how patient would like to be addressed 1 3 Defines objective (student explains what he/she is going to do) 1 4 Asks how long patient has had inability to void 1 5 Asks how patient first notice of his complain 1 6 Asks about the changing of his voiding pattern 1 7 Asks if the patient feeling a lower urinary tract symptomps (hesistancy, urgency, intermittency, weak stream, dysuria, post voiding dribbling) 8 Asks if patient has ever had any urinary symptom in his life 1 9 Asks if patient had suffered from hematuria 2 10 Asks what treatment patient has used so far for his hematuria 1 11 Ask if patient has had similar inability to void before 1 12 Asks if anyone in family has inability to void Des 07 OSCE 26 Rating 1=fail, 2= borderline, 3=meet expecations, 4= exceeds expectations Initiation of interview Listening skills Questioning skills Organization of questions Attitudes Non verbal communication Closing Global Rating - overall judgment of studen s performance Des 07 OSCE 27 13

14 Assigment Develop one OSCE case that will use a standardized patient. Identify he level of students to be assessed and purpose of the case(i.e.define the competencies that are being assessed) Prepare the case and the instructions to the student Develop the checklist/rating form, possible follow up questions qnd answer keys 28 Des 07 OSCE 28 14