Exploring DIF: Comparison of CTT and IRT Methods

36 Pages Posted: 6 Sep 2010

See all articles by Nabeel Abedlazeez

Nabeel Abedlazeez

University of Malaya - Department of Educational Psychology and Counseling, Faculty of Education

Date Written: September 6, 2010

Abstract

Assessment of test bias is important to establish the construct validity of tests. Assessment of differential item functioning (DIF) is an important first step in this process. DIF is present when examinees from different groups have differing probabilities of success on an item, after controlling for overall ability level. The study was conducted to answer the following questions: To what extent do the four methods (i.e. area differential index procedure for the 2- parameter Logistic model, TID, b-difference, and Chi-square) agree or disagree in the identification of DIF? Are there gender differences in mathematical proficiency? What is the content or nature of those items identified as revealing DIF? Achievement test covering the following subjects: Relations and functions, polynomial, Trigonometric functions, and triangles was developed. The test was administered to a sample of 1228 tenth grade students (656 males and 624 females) in Jordan. The study pointed out: (1) the percentage of agreement among the four methods in detecting DIF were from 41% to 85%. The highest agreement was between Chi-square and b-parameter difference methods (85%), whereas the lowest agreement was between Area index and TID methods (41%). The agreement among IRT based methods and CTT based methods were convergent. (3) females showed a statistically significant and consistent advantage over males on items involving Relations and functions, polynomial, Trigonometric functions, whereas men showed a less consistent advantage on items involving triangles, however It was concluded that gender differences in mathematics may well be linked to content.

Keywords: Transformed Item Difficulty, Area index , Chi-Square, b-Parameter Difference

Suggested Citation

Abedlazeez, Nabeel, Exploring DIF: Comparison of CTT and IRT Methods (September 6, 2010). OIDA International Journal of Sustainable Development, Vol. 1, No. 7, pp. 11-46, 2010, Available at SSRN: https://ssrn.com/abstract=1672735

Nabeel Abedlazeez (Contact Author)

University of Malaya - Department of Educational Psychology and Counseling, Faculty of Education ( email )

Kuala Lumpur, Wilayah Persekutuan 50603
University of Malaya (UM)
Kuala Lumpur, Wilayah Persekutuan 50603
Malaysia

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
364
Abstract Views
1,401
Rank
150,749
PlumX Metrics