Use of mantel haenszel differential item functioning in detecting item bias in a chemistry achievement test in four ethnic groups in Nigeria

This study used mantel haenszel Differential Item Functioning (DIF) approach to detect item bias in a Chemistry Achievement Test (CAT) in four ethnic groups in Nigeria. The CAT was administered to 400 chemistry students of senior secondary III in Niger State, Kwara State, Anambra State and Bayelsa State representing Hausa, Yoruba, Igbo and Ijaw respectively. A descriptive-comparative research design was used to describe and compare examinees of the four ethnic groups. A statistical and content analysis was done with Mantel Haenzel’s adjusted – DIF.The Mantel-Haenszel adjusted-DIF statistics flagged all items as showing significant differential item functioning between the focal group (Ijaw) and reference group (Yoruba, Hausa & Igbo). It was also observed that item 20 and 36 showed NO DIF for Ijaw/Yoruba and Ijaw/Hausa respectively. Based on the analysis, the study established that the CAT has some items that showed biases and not all items that exhibited differential functioning were flagged biased because some items fall within category A, (small effect) following the Educational Test Services (ETS) scale. Therefore, the study concludes that there was an ethnic bias in the CAT. The study recommends, among others, that Item Response Theory based DIF approaches should be used in educational testing.