Date of Award

May 2016

Degree Type

Dissertation

Degree Name

Doctor of Philosophy

Department

Educational Psychology

First Advisor

Cindy M. Walker

Committee Members

Bo Zhang, Razia Azen, Timothy C. Hass, Haiyan Ling

Abstract

The objective of this dissertation research is to investigate the possibility to improve both reliability and validity for test batteries under the framework of multi-stage testing (MST). Two test battery designs that incorporate MST components were proposed and evaluated, one is a multistage test battery (MSTB) design and the other is a hybrid multistage test battery (MSTBH) design. The MSTB design consists of three tests: The first test used the AMI (approximate maximum information) method as the routing strategy; and as for the second and third, the “On-the-Fly” strategy (OMST) was employed. The MSTBH design also consists of three tests; the first two are administered via MST while the third one via CAT.

This dissertation presents a new test battery design by combining the strengths from different testing models. To improve estimation precision, each subsequent test in the test battery for an examinee was assembled according to the examinee’s previous ability estimate. A set of simulation studies were conducted to compare MSTB, MSTBH with two baseline models for both measurement accuracy and test security control under various conditions. One of the baseline models is a MST design consisting of three MST procedures without borrowing information from each other’s; the other is a computerized adaptive test battery (CATB) design consisting of 1 to 3 CAT procedures, being the second and the third procedures borrowing information from the previous ones. The results demonstrated that the test battery designs yielded better measurement accuracy when considering previous subtest score as a predictor for the current subtest. All designs yielded acceptable mean exposure rates, but only the CATB design had ideal pool utilization. Finally, the discussion section presents some limitations on current studies

Share

COinS