Particularised checklists in materials evaluation: Developing contextually relevant criteria for Turkish EFL classes
Abstract
The major trend in ELT materials evaluation has been the measurement of general satisfaction levels against borrowed items from previous checklists. However, their real quality can be accurately assessed through contextually relevant criteria. In this study, 85 English teachers, and 500 seventh-graders evaluated the effectiveness of their locally-produced English coursebook with standardised, particularised, and importance-weighted Likert-type checklists. The extent of consumer dissatisfaction in their primary focus areas was found much greater than the overall evaluation alone had yielded. The teachers’ evaluations were yet unaffected by gender, experience, and length of textbook use. The top teacher-preferred criteria focused on accommodating different proficiency levels, and learner types; and the top student-preferred criteria on supplementary resources, and thematic appeal. The list of coursebook deficiencies might have been topped by the lack of authenticity, and supplementary resources for teacher guidance, and student self- instruction, but further overlaps existed between user views over such serious weaknesses as lack of: support for creative thinking, thematic appeal, grammatical explanations, and cultural diversity. Since these persistent coursebook concerns have been diagnosed in similar EFL contexts, where local materials were preferred over their global (UK/US-produced) counterparts, a context-sensitive, synergistic model was recommended for drafting, editing, and assessing ELT materials.
Keywords
Full Text:
PDFReferences
AbdelWahab, M. M. (2013). Developing an English language textbook evaluative checklist. IOSR Journal of Research & Method in Education, 1(3), 55-70.
Acar, A. (2006). Designing an effective ELT primary school textbook model by means of evaluating the primary school ELT textbooks: A sample in the designated group and context (Doctoral dissertation). İzmir: Institute of Educational Sciences, Dokuz Eylül University.
Aftab, A. (2011). English language textbooks evaluation in Pakistan (Doctoral dissertation). Birmingham: College of Arts and Law, University of Birmingham.
Ahmadi, A., & Derakhshan, A. (2016). EFL teachers’ perceptions towards textbook evaluation. Theory and Practice in Language Studies, 6(2), 260-267.
Ahour, T., Towhidiyan, B., & Saeidi, M. (2014). The evaluation of English Textbook 2 taught in Iranian high schools from teachers’ perspectives. English Language Teaching, 7(3), 150-158.
Alamri, A. (2008). An evaluation of the sixth grade English language textbook for Saudi boys (Master’s thesis). Riyadh: College of Arts, King Saud University.
Alani, A. & Jahangard, A. (2015). Analysis of teachers’ and students’ attitudes toward Iranian senior high school textbook series. Global Journal of Foreign Language Teaching, 5(1), 23-43.
Alfahadi, A. (2012). Saudi teachers’ views on appropriate cultural models for EFL textbooks: Insights into TESOL teachers’ management of global cultural flows and local realities in their teaching worlds (Doctoral dissertation). Exeter: University of Exeter.
Alhamlan, S. (2013). EFL curriculum and needs analysis: An evaluative study (Master’s thesis). Riyadh: King Saud University.
Alharbi, A. (2015). A descriptive-evaluative study of a Saudi EFL textbook series. Cogent Education, 2(1), 1-26.
Al-Momani, N. F. A. (1998). An evaluation of EFL textbooks (AMRA) for the first and second secondary classes in Jordan: Students’, teachers’ and supervisors’ perspectives (Doctoral dissertation). Irbid: TEFL Department, Yarmouk University.
Al-sowat, H. (2012). An evaluation of English language textbook, Say it in English, for first year intermediate grade in Saudi Arabia. Journal of Studies in Curriculum and Supervision, 3(2), 332-413.
Al-Yousef, H. (2007). An evaluation of the third grade intermediate English coursebook in Saudi Arabia (Master’s thesis). Riyadh: College of Arts Department of English, King Saud University.
Angulo, J. E. V., & Miranda, A. G. C. (2014). Evaluating a textbook’s efficiency to enhance students’ English proficiency in a Costa Rica’s rural school. Revista de Lenguas Modernas, 20, 395-425.
Ansary, H., & Babaii, E. (2002). Universal characteristics of EFL/ESL textbooks: A step towards systematic textbook evaluation. The Internet TESL Journal, 8(2). Retrieved on December 18, 2017 from http://iteslj.org/Articles/Ansary-Textbooks/
Aqel, A. R. A. Q. M. (2009). An evaluative study of the Palestinian 11th grade English textbook from the teachers’ perspective in Southern Nablus and Salfit districts (Master’s thesis). Nablus: Faculty of Graduate Studies, An-Najah National University.
Arıkan, G. (2008). Textbook evaluation in foreign language teaching (Master’s thesis). Adana: Institute of Social Sciences, Çukurova University.
Ariebowo, T. (2014). An evaluation study: The effectiveness of English language coursebook entitled, When English Rings the Bell. In The 61st TEFLIN International Conference 2014 Proceedings Book 3 (pp. 377-381). Surakarta: Sebelas Maret University.
Aydoğanlı, S. (2006). An analysis about the evaluation of the English textbooks based on the opinions of the first class of the vocational school of industry students, the English teachers and the administrators of the industrial vocational school (Master’s thesis). Gaziantep: Institute of Social Sciences, Gaziantep University.
Aytuğ, S. (2007). An EFL textbook evaluation study in Anatolian high schools: New Bridge to Success for 9th grade new beginners (Master’s thesis). Ankara: Institute of Educational Sciences, Bilkent University.
Büyüköztürk, Ş. (2011). Sosyal bilimler için veri analizi el kitabı. Ankara: Pegem.
Byrd, P. (2001). Textbooks: Evaluation for selection and analysis for implementation. In M. Celce-Murcia (Ed.), Teaching English as a second or foreign language (pp. 415-427). Boston: Heinle.
Chambers, F. (1997). Seeking consensus in coursebook evaluation. ELT Journal, 51(1), 30-35.
Chao, T. C. (2010). A retrospective evaluation of a local college English textbook from an intercultural perspective. In Y. J. Chen, S. J. Huang, H. C. Liao & S. Lin (Eds.), Studies in English for professional communications and applications (pp. 89-100). Taipei: Crane Publishing Co.
Chow, Y. W. F. (2004). Teacher and learner perceptions of ELT textbooks in Hong Kong secondary schools (Doctoral dissertation). Leicester: School of Education, University of Leicester.
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Boston: Pearson.
Cummins, R. A. (1997). Comprehensive quality of life scale: School version - Grades 7-12. Melbourne: Deakin University.
Cunningsworth, A. (1984). Evaluating and selecting EFL teaching materials. London: Heinemann.
Cunningsworth, A. (1995). Choosing your coursebook. Oxford: Heinemann.
Çelik, Ş. N. (2011). Students’, teachers’ and inspectors’ views about secondary school English coursebook Breeze 9 (Master’s thesis). Ankara: Instiute of Social Sciences, Hacettepe University.
Dahmardeh, M. (2009). Communicative textbooks: English language textbooks in Iranian secondary school. Linguistik Online, 40(4), 45-61.
Danaye-Tous, M., & Haghighi, S. (2014). Evaluation of ESP textbooks: Evidence from ESP textbook of computer engineering major. International Journal of Research Studies in Language Learning, 3(2), 55-68.
Dat, B. (2008). ELT materials used in Southeast Asia. In B. Tomlinson (Ed.), English language learning materials, (pp. 263-280). London: Continuum.
Demir, S. (2008). An evaluation of thinking skills in ELT coursebooks at primary and secondary levels (Master’s thesis). Adana: Institute of Social Sciences, Çukurova University.
Demirci, D., & Tavil, Z. M. (2015). Are the skills really integrated in coursebooks? A sample case - Yes You Can. Educational Research and Reviews, 10(12), 1599- 1632.
Diktaş, M. Y. (2010). Gender representations in EFL coursebooks: A comparative study of textbooks designed by ministry of education of Turkey and non-Turkish publishers (Master’s thesis). Istanbul: Institute of Social Sciences, Maltepe University.
Durukafa, G. (2000). Ders kitabı analizi ve analiz ölçütleri [Coursebook analysis and criteria]. Çanakkale Üniversitesi Eğitim Fakültesi Dergisi, 2, 111–116.
Dülger, O. (2016). Evaluation of EFL coursebooks taught in Turkey based on teachers’ views. Journal of Advances in English Language Teaching, 4(1), 1-11.
Ertürk, H. (2013). An EFL coursebook evaluation: Unique 6 teachers’ and students’ perspectives (Master’s thesis). Konya: Institute of Educational Sciences, Necmettin Erbakan University.
Ezici, I. Ç. (2006). Evaluation of the EFL textbook New Bridge to Success 3 from the perspectives of students and teachers (Master’s thesis). Ankara, Turkey: Middle East Technical University, Institute of Social Sciences.
Faujiah, E., & Floris, F. D. (2015). The evaluation of When English Rings A Bell (Revised edition): Teachers’ perspectives. Kata Kita: Journal of Language, Literature, and Teaching, 3(2), 13-19.
Gearing, K. (1999). Helping less-experienced teachers of English to evaluate teachers’ guides. ELT Journal, 53(2), 122–127.
George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference (Fourth Edition). Boston: Allyn & Bacon.
George, D., & Mallery, P. (2016). IBM SPSS Statistics 23 step by step: A simple guide and reference (Fourteenth Edition). New York: Routledge.
Golpour, F. (2012). Iranian junior high school English book series (Right Path to English) weighted against material evaluation checklists. International Journal of Applied Linguistics and English Literature, 1(7), 170-179.
Guilani, M. A., Yasin, M. S. M., & Hua, T. K. (2011). Authenticity of Iranian English textbooks for schools. English Language and Literature Studies, 1(2), 25-30.
Güreli, N. (2008). A study into Çanakkale science high school students’ perceptions of an ideal English coursebook (Master’s thesis). Çanakkale: Institute of Social Sciences, Çanakkale Onsekiz Mart University.
Hadley, G. (2014). Global textbooks in local contexts: An empirical investigation of effectiveness. In N. Harwood (Ed.), English language teaching textbooks: Content, consumption, production (pp. 205-238). London: Palgrave Macmillan.
Harmer, J. (2001). The practice of English language teaching. New York: Longman.
Harwood, N. (2005). What do we want EAP teaching materials for? Journal of English for Academic Purposes, 4(2), 149-161.
Heiman, G. (2011). Basic statistics for the behavioral sciences. Belmont: Cengage Learning.
Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393-416.
Hooman, Y. (2014). The evaluation of Iranian EFL textbooks from post method principles pedagogy. Theory and Practice in Language Studies, 4(5), 1026-1032.
Huang, S. (2011). Ideal and reality of textbook selection: An interview- and questionnaire-based investigation in the Taiwanese tertiary context (Doctoral dissertation). Coventry: Centre for Applied Linguistics, University of Warwick.
Hutchinson, T., & Torres, E. (1994). The textbook as agent of change. ELT Journal, 48(4), 315-328.
Hutchinson, T., & Waters, A. (1987). English for specific purposes: A learning-centred approach. Cambridge: Cambridge University Press.
Iriskulova, A. (2012). The investigation of the cultural presence in Spot on 8 ELT textbook published in Turkey: Teachers’ and students’ expectations versus real cultural load of the textbook (Master’s thesis). Ankara: Institute of Social Sciences, Middle East Technical University.
Işık, A. (2011). Language education and ELT materials in Turkey from the path dependence perspective. Hacettepe University Journal of Education, 40, 256–266.
Ä°nal, B. (2006). Coursebook selection process and some of the most important criteria to be taken into consideration in foreign language teaching. Journal of Arts and Sciences, 5, 19-29.
Jahangard, A. (2007). Evaluation of EFL materials taught at Iranian public high schools. Asian EFL Journal, 9(2), 130-150.
Jamalvandi, B. (2014). ELT textbook evaluation in Iran: New insights. European Online Journal of Natural and Social Sciences, 3(4), 1068- 1078.
Juan, W. U. (2010). A content analysis of the cultural content in the EFL textbooks. Canadian Social Science, 6(5), 137-144.
Karamifar, M., Barati, H., & Youhanaee, M. (2014). Local evaluation criteria for global textbooks: A case study from Iran. Theory and Practice in Language Studies, 4(5), 923-930.
Karamoozian, F. M., & Riazi, A. (2008). Development of a new checklist for evaluating reading comprehension textbooks. ESP World, 7(19), 19-39.
Karimi, K., & Biria, R. (2016). A multidimensional critical evaluation of English coursebook used for Iranian high school first grader. Modern Journal of Language Teaching Methods, 6(6), 214- 232.
Kayaoğlu, M. N. (2011). A critical appraisal of the language textbook. Ahi Evran University Journal of Kırşehir Education Faculty, 12(4), 341-356.
Kayapınar, U. (2009). Coursebook evaluation by English teachers. Inönü University Journal of the Faculty of Education, 10(1), 69-78.
Kırkgöz, Y. (2009). Globalization and English language policy in Turkey. Educational Policy, 23(5), 663-684.
Kim, H. D. (2004). Learners’ opinions on criteria for ELT materials evaluation. English Teaching, 59(3), 3-28.
Kitao, K., & Kitao, S. (1997). Selecting and developing teaching/learning materials. The Internet TESL Journal, 4(4). Retrieved on January 8, 2018 from http://iteslj.org/Articles/Kitao-Materials.html
Korpela, N. (2007). If I were a textbook writer: Views of EFL textbooks held by Finnish comprehensive school pupils (Master’s thesis). Jyväskylä: Department of Languages, University of Jyväskylä.
Köroğlu, Z. Ç. (2013). Language instructors’ perspectives on textbook content in terms of intercultural communicative competence: Gazi University case (Master’s thesis). Ankara: Institute of Educational Sciences, Gazi University.
Law, W. H. (1995). Teachers’ evaluation of English textbooks: An investigation of teachers’ ideas and current practices and their implications for developing textbook evaluation criteria (Master’s thesis). Hong Kong: University of Hong Kong.
Littlejohn, A. (2012). Language teaching materials and the (very) big picture. Electronic Journal of Foreign Language Teaching, 9(1), 283–297.
Mahboudi, H. R., & Javdani, F. (2012). The teaching of English in Iran: The place of culture. Journal of Language and Culture, 3(5), 87-95.
Mahfoodh, M. I. H. A., & Bhanegaonkar, S. G. (2013). New approach for evaluating EFLM (An eclectic developed checklist). International Journal of Scientific and Research Publications, 3(10), 1-8.
Maleki, A., Mollaee, F., & Khosravi, R. (2014). A content evaluation of Iranian pre-university ELT textbook. Theory and Practice in Language Studies, 4(5), 995-1000.
Mazgon, J., & Stefanc, D. (2012). Importance of the various characteristics of educational materials: Different opinions, different perspectives. The Turkish Online Journal of Educational Technology, 11(3), 174- 188.
McGrath, I. (2002). Materials evaluation and design for language teaching. Edinburgh: Edinburgh University Press.
Miekley, J. (2005). ESL textbook evaluation checklist. The Reading Matrix, 5(2). Retrieved on November 9, 2017 from http://www.readingmatrix.com/reading_projects/miekley/project.pdf
Mukundan, J., & Ahour, T. (2010). A review of textbook evaluation checklists across four decades (1970-2008). In B. Tomlinson & H. Masuhara (Eds.), Research for materials development in language learning: Evidence for best practice (pp. 336-352). London: Continuum.
Mukundan, J., Hajimohammadi, R., & Nimehchisalem, V. (2011). Developing an English language textbook evaluation checklist. Contemporary Issues in Education Research, 4(6), 21- 28.
Mukundan, J., Nimehchisalem, V., & Hajimohammadi, R. (2011). Developing an English language textbook evaluation checklist: A focus group study. International Journal of Humanities and Social Science, 1(12), 100-106.
Mukundan, J. & Nimehchisalem, V. (2012). Evaluative criteria of an English language textbook evaluation checklist. Journal of Language Teaching & Research, 3(6), 1128-1134.
Naumann, E., & Giel, K. (1995). Customer satisfaction measurement and management: Using the voice of the customer. Ohio: Thomson Executive Press.
Nemati, A. (2009). Evaluation of an ESL English coursebook: A step towards systematic vocabulary evaluation. Journal of Social Sciences, 20(2), 91-99.
Nguyen, C. T. (2015). An evaluation of the textbook English 6: A case study from secondary schools in the Mekong delta provinces of Vietnam (Doctoral dissertation). Sheffield: Department of English Language and Linguistics, University of Sheffield.
Nimehchisalem, V., & Mukundan, J. (2013). Usefulness of the English language teaching textbook evaluation checklist. Pertanika Journal of Social Sciences & Humanities, 21(2), 797-816.
Nimehchisalem, V., & Mukundan, J. (2015). Refinement of the English language teaching textbook evaluation checklist. Pertanika Journal Social Science and Humanities, 23(4), 761-780.
Oflaz, T. D. (2009). Teachers’ perceptions on coursebook Time for English 5: The 5th grade state schools (Master’s thesis). Gaziantep: Institute of Social Sciences, Gaziantep University.
Özdemir, F. E. (2007). An evaluation of Time for English 4, the fourth grade English coursebook for public schools (Master’s thesis). Ankara: Institute of Social Sciences, Middle East Technical University.
Özeş, D. (2012). Textbook evaluation in foreign language teaching: Spot on grade 8 from students’ and teachers’ perspectives (Master’s thesis). Bursa: Institute of Educational Sciences, Uludağ University.
Park, M. R., & Suh, K. O. (2003). An analysis of Korean high school English textbooks under the 7th curriculum. English Teaching, 58(4), 319-347.
Pizam, A., & Ellis, T. (1999). Customer satisfaction and its measurement in hospitality entertprises. International Journal of Contemporary Hospitality Management, 11(7), 326- 339.
Prodromou, L., & Mishan, F. (2008). Materials used in Western Europe. In B. Tomlinson (Ed.), English language learning materials: A critical review (pp. 193-212). London: Continuum.
Rahimpour, M., & Hashemi, R. (2011). Textbook selection and evaluation in EFL context. World Journal of Education, 1(2), 62-68.
Razmjoo, S. A. (2010). Developing a textbook evaluation scheme for the expanding circle. Iranian Journal of Applied Language Studies, 2(1), 121-136.
Ravelonanahary, M. (2007). The use of textbooks and educational media: The Malagasy experience. In M. Horsley & J. McCall (Eds.), Ninth international conference on textbooks and educational media (pp. 166-175). Hogskolen: IARTEM.
Rezaeian, M., & Zamanian, M. (2015). Textbook evaluation: ELT teachers’ perspectives on Learning to Read English for pre-university students. International Journal of Research Studies in Language Learning, 4(2), 105-118.
Riazi, A. M. (2003). What textbook evaluation schemes tell us? A study of the textbook evaluation schemes of three decades. In W. A. Renanda (Ed.), Methodology and materials design in language teaching (pp. 52-68). Singapore: SEAMEO Regional Center.
Richards, J. C. (2001). The role of textbooks in a language program. RELC Guidelines, 23(2), 12-16.
Roberts, J. T. (1996). Demystifying materials evaluation. System, 24(3), 375-389.
Rohmah, Z. (2009). EFL materials in Madrasah Tsanawiyah: What do they really need? TEFLIN Journal, 20(1), 104-117.
Sabrina, H. (2016). The evaluation of the coursebook Spotlight on English One from teachers’ perspectives (Master’s thesis). Biskra: Faculty of Letters and Languages, Mohammed Kheider University of Biskra.
Sarı, F. (2007). The effects of coursebooks on the development of proficiency levels of Turkish learners of English (Master’s thesis). Istanbul: Institute of Educational Sciences, Marmara University.
Shabani, M. B., & Nejad, A. M. (2013). An evaluation of the third-grade high school English textbook: An Iranian case study. Journal of Studies in Social Sciences, 2(1), 67-80.
Shater, H., & Azargoon, M. (2012). Designing and developing a native checklist to evaluate general English coursebooks in Iran and comparing it with other existing checklists in the world (Paper presented at the 1st Conference on Language & Teaching: An Interdisciplinary Approach). Retrieved on January 6, 2018 from http://confnews.um.ac.ir/images/41/conferences/llt/cd51.pdf
Sheldon, L. E. (1988). Evaluating ELT textbooks and materials. ELT Journal, 42(4), 237–246.
Skierso, A. (1991). Textbook selection and evaluation. In M. Celce-Murcia (Ed.), Teaching English as a second or foreign language (pp. 432- 453). Boston: Heinle & Heinle.
Soori, A., Kafipour, R., & Soury, M. (2011). EFL textbook evaluation and graphic representation. European Journal of Social Sciences, 26(3), 481-493.
Song, B. (1991). A comparison of perceptions of EFL textbooks used in Chinese secondary schools (Doctoral dissertation). Retrieved on November 22, 2017 from ProQuest Dissertations and Theses Database. (Order No. 9216075).
Sözen, B. O. (2007). Significance of popular culture in English language teaching coursebooks (Master’s thesis). Istanbul: Institute of Educational Sciences, Marmara University.
Sümen, G. (2008). An evaluation of the new English coursebook of the fourth grade of state schools in Turkey and preparing supplementary materials in reference with the new English curriculum (Master’s thesis). Ankara: Institute of Educational Sciences, Gazi University.
Taylan, H. (2013). An evaluation of Breeze 9: The 9th grade English coursebook for Turkish public high schools (Master’s thesis). Çanakkale: Institute of Educational Sciences, Çanakkale 18 Mart University.
Tekir, S., & Arıkan, A. (2007). An analysis of English language teaching coursebooks by Turkish writers: Let's Speak English 7 example. International Journal of Human Sciences, 4(2), 1-18.
Timmis, I. (2014). Writing materials for publication: Questions raised and lessons learned. In N. Harwood (Ed.) English language teaching textbooks: Content, consumption, production (pp. 241-261). New York: Palgrave Macmillan.
Tok, H. (2010). TEFL textbook evaluation: From teachers’ perspectives. Educational Research and Reviews, 5(9), 508-517.
Tomlinson, B. (2001). Materials development. In D. Nunan & R. Carter (Eds.), The Cambridge guide to teaching English to speakers of other languages (pp. 66-71). Cambridge: Cambridge University Press.
Tomlinson, B. (2008). Language acquisition and language learning materials. In B. Tomlinson (Ed.), English language learning materials: A critical review (pp. 3-13). London: Continuum.
Tucker, C. A. (1975). Evaluating beginning textbooks. English Teaching Forum, 13, 355-361.
Ulum, Ö. G. (2014). Teachers’ views on Yes You Can, the ninth grade English coursebook for public schools. International Journal of Languages' Education and Teaching, 2(3), 132-148.
Ur, P. (1996). A course in language teaching: Practice and theory. Cambridge: Cambridge University Press.
Ur, P. (1999). A course in language teaching: Trainee book. Cambridge: Cambridge University Press.
Vahdany, F. (2015). Evaluating PNU General English textbook from two perspectives: Science vs. humanities students. International Journal of Asian Social Science, 5(11), 678-693.
Williams, D. (1983). Developing criteria for textbook evaluation. ELT Journal, 37(3), 251-255.
Xiao, J. (2010). Cultural contents of an in-use EFL textbook and English major students’ attitudes and perceptions towards culture learning at Jiangxi University of Science and Technology, China (Master’s thesis). Songkhla: Prince of Songkla University.
Xu, I. H. (2004). Investigating criteria for assessing ESL textbooks (Doctoral dissertation). Edmonton: University of Alberta.
Zabawa, J. (2001). Criteria for FCE textbook evaluation: An attempt at questionnaire formulation. Studia Anglica Resoviensia, 42, 161-169. Retrieved on April 6, 2018 from https://www.ur.edu.pl/file/1221/sar_v2_17.pdf
Zohrabi, M. (2011). Coursebook development and evaluation for English for general purposes course. English Language Teaching, 4(2), 213- 222.
Zohrabi, M., Sabouri, H., & Behroozian, R. (2012). An assessment of strengths and weaknesses of Iranian first year high school English coursebook using evaluation checklist. English Language and Literature Studies, 2(2), 89-99.
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
ISSN 1305-578X (Online)
Copyright © 2005-2022 by Journal of Language and Linguistic Studies