ITEM ANALYSIS ON THE ENGLISH FINAL TEST IN CURRICULUM 2013 FOR THE SEVENTH GRADE STUDENTS OF JUNIOR HIGH SCHOOLS IN KARANGANYAR YUNI ARIFIANI K2210092 Submitted to the Teacher Training and Education Faculty of Sebelas Maret University to Fulfill One of the Requirements for Obtaining the Undergraduate Degree in English Education ENGLISH EDUCATION DEPARTMENT TEACHER TRAINING AND EDUCATION FACULTY 2014 i
SEBELAS MARET UNIVERSITY ii
ITEM ANALYSIS ON THE ENGLISH FINAL TEST IN CURRICULUM 2013 FOR THE SEVENTH GRADE STUDENTS OF JUNIOR HIGH SCHOOLS IN KARANGANYAR YUNI ARIFIANI K2210092 Thesis Submitted to Teacher Training and Education Faculty of Sebelas Maret University as a Partial Requirement for Getting the Undergraduate Degree of English Education ENGLISH EDUCATION DEPARTMENT TEACHER TRAINING AND EDUCATION FACULTY SEBELAS MARET UNIVERSITY SURAKARTA 2014 iii
iv
v
ABSTRACT Yuni Arifiani. K2210092, Item Analysis on the English Final Test in Curriculum 2013 for the Seventh Grade Students of Junior High Schools in Karanganyar. A thesis. Surakarta: Teacher Training and Education Faculty. Sebelas Maret University, Surakarta, 2014. This research aims to describe the quality of English final test in Curriculum 2013 for the seventh grade students of junior high schools in Karanganyar in the first semester. The quality described in this research includes the qualitative aspects (material, construct, language, and cognitive domain), content validity and quantitative aspects (including the item difficulty, discrimination power, the effectiveness of distracters and reliability). This is descriptive research. In collecting the data the writer uses document as data source. The data are taken from 245 students answer sheets and one set English test items of seventh grade in SMP Jaten 1 and SMP 2 Karanganyar. This research employs expert judgment to review the material, the construct, and the language of the items. The students responses are also quantitatively analyzed by using Iteman 3.00 for multiple-choice item and Anates for essay item. The research shows the following results: (1) the result of qualitative analysis with expert judgment which shows that out of the 55 test items, 51 test items (92.72%) are categorized as good viewed from material, 42 test items (76.36%) are categorized as good viewed from construct, and 49 test items (89.09%) are categorized as good viewed from language, while viewed from the distribution of cognitive domain, the data shows that percentage of each level is as follows: C1 are 60%, C2 are 32.73 %, C3 are 7.27 %, C4 is 0%, C5 is 0 %, and C6 is 0%. It shows that this English final test is considered easy; (2) the result of content validity analysis shows there are 51 test items (92.72%) having the relevance to the basic competence and test indicators indicating this English test has high validity; (3) the quantitative analysis with Iteman 3.00 and Anates shows that there are only 13 test items (23.6%) of the total items that fulfill all criteria which are difficulty index and discrimination power with all effective distracters (for multiple-choice item). Then, there are 37 items (67.3%) that need some revision and are considered to be written back. The remaining items, that are 5 test items (9.1%) of the items belong to the rejected that cannot be used again. Then, the multiple-choice items of this English test have high reliability; that is 0.873 and the essay items have low reliability. It is 0.51. Key words: Item analysis, English final test, Qualitative aspects, Quantitative aspects vi
ABSTRAK Yuni Arifiani. K2210092, "Analis Butir Soal Semester Satu Mata Pelajaran Bahasa Inggris Kurikulum 2013 untuk Siswa Kelas Tujuh Sekolah Menengah Pertama di Karanganyar". Skripsi. Surakarta: Fakultas Keguruan dan Ilmu Pendidikan. Universitas Sebelas Maret, Surakarta, 2014. Penelitian ini bertujuan untuk mendeskripsikan kualitas tes semester satu mata pelajaran Bahasa Inggris Kurikulum 2013 untuk siswa kelas tujuh Sekolah Menengah Pertama di Karanganyar. Kualitas yang diuraikan dalam penelitian ini meliputi aspek yang kualitatif (material, konstruksi, bahasa, dan kognitif domain), validitas isi dan aspek kuantitatif (mencakup tingkat kesukaran, daya pembeda, efektivitas pengecoh dan reliabilitas). Penelitian ini merupakan penelitian deskriptif. Dalam pengumpulan data penulis menggunakan dokumen sebagai sumber data. Data diambil dari 245 lembar jawab siswa dan satu set soal Bahasa Inggris kelas tujuh dari SMP Jaten 1 dan SMP 2 Karanganyar. Penelitian ini menggunakan pertimbangan ahli untuk menganalisis aspek material, konstruksi, dan bahasa. Lembar jawab siswa dianalisis secara kuantitatif dengan menggunakan Iteman versi 3.00 untuk soal pilihan ganda dan Anates untuk soal esay. Penelitian menunjukkan hasil sebagai berikut: (1) Hasil analisis kualitatif dengan menggunakan pertimbangan ahli menunjukkan bahwa dari 55 soal, 51 soal (92,72%) dipandang baik dari aspek materi, 42 soal (76,36%) dipandang baik dari aspek konstruksi, dan 49 soal (89,09%) dipandang baik dari aspek bahasa, dilihat distribusi ranah kogniif, data menunjukkan bahwa persentase dari masing-masing tingkatan adalah sebagai berikut: C1 60%, C2 adalah 32,73%, C3 adalah 7,27%, C4 0%, C5 adalah 0%, dan C6 0%. Hal ini menunjukkan bahwa soal Bahasa Inggris ini dipertimbangkan sebagai low order thinking; (2) Hasil analisis validitas isi menunjukan ada 51 soal (92,72%) yang sesuai dengan kompetensi dasar dan indikator tes yang berarti soal Bahasa Inggris ini mempunyai validitas isi yang tinggi; (3) analisis kuantitatif dengan Iteman 3.00 dan Anates menunjukkan bahwa hanya ada 13 soal (23,6%) dari total soal yang memenuhi semua kriteria soal yang baik yaitu tingkat kesukaran dan daya pembeda dengan semua pengecoh yang efektif (untuk soal pilihan ganda). Kemudian, ada 37 soal (67,3%) memerlukan beberapa revisi dan dianggap sebagai soal yang ditulis kembali. Soal yang lain, yaitu 5 soal (9,1%) yang tergolong dalam soal yang ditolak yang mana tidak bisa digunakan lagi. Kemudian, soal pilihan ganda dalam tes ini mempunyai reliabilitas yang tinggi; yaitu 0.873 dan soal esay mempunyai reliabilitas rendah, yaitu 0.51. Kata kunci: Analisis butir soal, Soal Bahasa Inggris, Aspek-aspek kualitatif, Aspekaspek Kuantitatif. vii
MOTTO Good start will lead you to great end. Never say Never! Allah will never give us a burden beyond our ability viii
DEDICATION This thesis is dedicated to: Allah S.W.T the Almighty. My beloved parents, Mr. Achmad Hadiwiono and Mrs. Sudarni, my super brother, Irfan Purwanto and my dearest sisters, Tri Sulistyowati, Arisma Rahmadani, and Syafrina Nur. S thanks for your support and blessing. My lovely and best friends Timpux Happy Bee thanks for your flowing love. My friends in Giri Ayu boarding house, thanks for helps and supports. All of my awesome friends in English Education Department 2010. ix
ACKNOWLEDGEMENT First and above all, the writer praises Allah SWT for the blessing, mercy and love given to the writer, so the writer has finished the thesis entitled An Item Analysis of The English Final Test in Curriculum 2013 for the Seventh Grade Students of Junior High School in Karanganyar. The writer realizes that the writer would not be able to complete the thesis without support, advice and guidance from many people. Therefore, the writer would like to express the sincerest gratitude to: 1. Prof. Dr. H.M. Furqon Hidayatulloh M.Pd, the Dean of Teacher Training and Education Faculty of Sebelas Maret University; 2. Teguh Sarosa SS.,M.Hum, the Head of English Education Department; 3. Drs. Siswantoro, M.Hum as the first advisor who always gives guidance, excellent advice and idea to the writer; 4. Teguh Sarosa SS.,M.Hum as the second advisor, who has carefully read and made several corrections for the improvement of this research and given valuable advice; 5. All of the lecturers and staffs of English Education Department of Teacher Training and Education Faculty of Sebelas Maret University who have gave their contribution to the writer for completing this research; 6. Dr. Sumardi, M. Hum, a lecturer from English Education Department of post graduate program in Sebelas Maret University and Mr. Bambang W. K, S. Pd, an English teacher in SMP N 16 Surakarta as experts who have given the judgments experts and the helps to the writer; 7. Mrs. Endang as the English teacher of the seventh grade students of SMP Jaten 1 and Mrs. Triswati as the English teacher of the seventh grade students of SMP Karanganyar 2 who have given guidance in giving the data source of this research; x
8. Her beloved parents, Mr. Achmad Hadiwiono and Mrs. Sudarni, her super brother, Irfan Purwanto, and her dearest sisters, Tri Sulistyowati, Arisma Rahmadani, and Syafrina Nur. S thanks for your love, supports, prays and blessing; 9. Her best friends Timpux who has given support, love, care and spirit; 10. Her friends in Giri Ayu Boarding House, who cannot be mentioned one by one; 11. All of her awesome friends in English Education Department 2010 who have shared the joy, spirit, sadness and happiness. 12. Anyone who always encourages and prays her. The writer would like to express her sincerest gratitude to the readers for some critics and suggestions. Hopefully this thesis will be useful for everyone. Surakarta, November 2013 The writer Yuni Arifiani xi
TABLE OF CONTENTS TITLE.... i PRONOUNCEMENT...... ii PROPOSED TITLE...... iii THE APPROVAL OF THE CONSULTANTS...... iv LEGALIZATION OF THE EXAMINERS... v ABSTRACT...... vi MOTTO... viii DEDICATION...... ix ACKNOWLEDGEMENT..... x TABLE OF CONTENTS...... xii LIST OF APPENDICES...... xv LIST OF TABLES... xvi LIST OF FIGURES...... xvii CHAPTER I INTRODUCTION...... 1 A. Background of the Study... 1 B. Problem Statements...... 5 C. Objectives of the Research... 5 D. Benefits of the Research... 6 CHAPTER II THEORETICAL REVIEW... 8 A. Review on Test...... 8 1. The Definition of Test... 8 2. Test Types.... 10 3. Criteria of Good Test...... 14 4. Multiple-Choice Test... 17 5. Essay Test... 21 B. Item Analysis... 22 1. Qualitative Analysis... 24 xii
a. Material, Construct, and Language... 24 b. Cognitive Domain... 25 c. Validity... 26 2. Quantitative Analysis... 28 a. Item Difficulty... 28 b. Item Discrimination... 30 c. The Effectiveness of Distracter... 31 d. Reliability... 32 C. Assessment in Curriculum 2013... 33 D. Review of Related Researches... 39 E. Rationale... 41 CHAPTER III RESEARCH METHODOLOGY... 43 A. The Setting of Research... 43 B. Research Method... 43 C. Data Resources.. 44 D. The Techniques of Collecting Data... 47 E. The Techniques of Data Analysis... 47 CHAPTER IV RESEARCH FINDINGS & DISCUSSION.... 57 A. Research Findings... 57 1. Data Description... 57 2. Data Analysis... 60 1) Qualitative Analysis... 60 a. Material... 60 b. Construct... 62 c. Language... 67 2) Quantitative Analysis... 70 a. Analysis of Multiple-Choice Items... 70 b. Analysis of Essay Items... 107 3. Result of Analysis... 109 xiii
1. Qualitative Analysis Result... 109 2. Quantitative Analysis Result... 112 B. Discussion... 115 1. Qualitative Analysis... 115 a. Material... 115 b. Construct... 116 c. Language... 116 d. Distribution of Cognitive Domain... 117 e. Validity... 118 2. Quantitative Analysis... 118 a. Item Difficulty... 118 b. Item Discrimination... 120 c. The Effectiveness of Distracter... 121 d. Reliability... 122 CHAPTER IV CONCLUSION, IMPLICATION & SUGGESTION... 124 A. Conclusion... 124 B. Implication... 126 C. Suggestion... 126 BIBLIOGRAPHY 128 APPENDICES.. 132 xiv
LIST OF APPENDICES Appendix 1. English Test Item... 132 Appendix 2. Blue-print of English Test Item... 138 Appendix 3. The Analysis Result from Experts... 141 Appendix 4. The Analysis of Multiple-Choice Items by Using Iteman... 158 Appendix 5. The Analysis of Essay Items by Using Anates... 169 Appendix 6. Students Answer Sheet... 179 Appendix 7. Letter of Permission... 189 xv
LIST OF TABLES Table 2.1. Structure of the Cognitive Process: Dimension of the Revised - Taxonomy... 26 Table 3.1 Research Schedule... 43 Table 3.2 Card Analysis for Multiple-Choice Items... 48 Table 3.3 Card Analysis for Essay Items... 49 Table 4.1 Key Answer... 58 Table 4.2. Core Competence and Basic Competence for Seventh Graders for the First Semester... 59 Table 4.3: The Result of Essay Items Analysis Using Anates... 107 Table 4.4: Qualitative Analysis of Multiple-Choice Items... 110 Table 4.5: Qualitative Analysis of Essay Items... 111 Table 4.6: The Result of Material, Construct, and Language Analysis... 111 Table 4.7: Result Analysis of Cognitive Domain... 112 Table 4.8: Quantitative Analysis of Multiple-Choice Items... 113 Table 4.9: Quantitative Analysis of Essay Items... 114 Table 4.10: The Result of Quantitative Analysis... 114 xvi
LIST OF FIGURES Page Figure 2.1: Rationale... 42 Figure 4.1: Diagram of Item Difficulty Analysis... 119 Figure 4.2: Diagram of Discrimination Index Analysis... 121 xvii