REVISITING CULTURAL BIASES IN TOEFL TEST: A PILOT STUDY

Nyak Mutia Ismail, Arisna Mukti, Marisa Yoestara, Hilda Delavari, Alireza Qadiri

Abstract


TOEFL test is required as one of the standardized test in English proficiency for scholars worldwide, across cultures. This study is a pilot study conducted to see cultural bias contained in the TOEFL test in various TOEFL Preparation modules that are frequently used as the source of questions for TOEFL equivalent in Aceh. Descriptive content analysis was considered as a method that well-conforms the needs of this study. The data were taken from four different TOEFL modules—Cambridge TOEFL Preparation, Barron’s TOEFL Practice, Cliff’s TOEFL Practice, and Longman TOEFL practice module.—specifically scoped into reading section alone. Hence, the instruments involved in this study were the documents of TOEFL preparation module. The questions which indicate cultural bias are verified as the data of this study. Later on, the data analysis was carried out by employing three-step analysis by Miles, Huberman, and Saldana (2014). The result shows that from those four TOEFL modules, particularly in reading section, the quantities of biased items are not cautious for EFL TOEFL test-takers. However, future research covering the Listening and Structure and Written Expression sections should be taken into account.


Keywords


Standardized test; TOEFL test; constructed bias; cultural bias; test bias

Full Text:

PDF

References


Adedayo, A. V. (2015). Proof of English proficiency tests. https://www.researchgate.net/publication/276268366_PROOF_OF_ENGLISH_PROFICIENCY_TESTS

Angoff, W. H. (1989). TOEFL research reports: Context bias in the Test of English as a Foreign Language. Princeton: Educational Testing Service.

Blau, J. R., Moller, S., & Jones, L. V. (2004). Why test? Talent loss and enrollment loss. Social Science Research, 33, 409-434.

Djiwandono, P I. (2006). Cultural bias in language testing. TEFLIN Journal, 17(1), 81-89

Gear, J., & Gear, R. (2002). Cambridge preparation for the TOEFL Test. Cambridge: Cambridge University Press.

Gear, J., & Gear, R. (2016). Cambridge preparation for the TOEFL Test. Internet-Based Test. Cambridge: Cambridge University Press.

Heine, S. J., Kitayama, S., Lehman, D. R., Takata, T., Ide, E., Leung, C., et al. (2001). Divergent consequences of success and failure in Japan and North America: An investigation of self-improving motivations and malleable selves. Journal of Personality and Social Psychology, 81, 599-615.

Kim, K. H., & Zabelina, D. (2015). Cultural bias in assessment: Can creativity assessment help? International Journal of Critical Pedagogy, 6(2), 129-148.

Omi, M., & Winant, H. (2015). Racial formations in the United States. New York, NY: Routledge.

Pan, Y. C. (2014). Learner washback variability in standardized exit tests. The Electronic Journal for English as a Second Language, 18(2), 1-30.

Phillips, D. (2004) Longman preparation course for the TOEFL test: The paper test. New York: Pearson Education.

Pyle, M. A. (2001). CliffsTestPrepTM: TOEFL CBT. Foster City: IDG Book Worldwide.

Reynolds, C. R., & Suzuki, L. A (2003). Bias in psychological assessment: An empirical review and recommendations. In I. R. Weiner (Ed.) Handbook of psychology (2nd Edition) (pp. 82-108). Hoboken: John Wiley & Sons, Inc.

Sharpe, P. J. (2004) Baron’s how to prepare TOEFL. New York: Barron's Educational Series, Inc.

Shirazi, M. A. (2019). For a greater good: Bias analysis in writing assessment. Sage Open, Jan-March 2019, 1-14. https://doi.org/10.1177/2158244018822377

Sims, J. M. (2015). A valid and reliable English proficiency exam: A model from a university language program in Taiwan. English as a Global Language Education (EaGLE) Journal, 2(1), 91-93

Van de Vijver, F., & Tanzer, N. K. (2004). Bias and equivalence in cross-cultural assessment: An overview. Tilburg: Elsevier SAS.

Wicherts, J. M., Dolan, C. V., & Hessen, D. J. (2005). Stereotype threat and group differences in test performance: A question of measurement invariance. Journal of Personality and Social Psychology, 89 (5), 696–719.

Xi, X. (2010). How do we go about investigating test fairness? Language Testing, 27, 147-170.


Refbacks

  • There are currently no refbacks.