Time in computerized tests: A multitrait, multimethod investigation of general- knowledge and mathematical-reasoning on-line examinations

This study provides a convergent and discriminant validation of the use of time related regimes in psychometric, on-line tests. A experimental setting contrasted two methods (correctness and time) by two traits (Mathematical Reasoning and General Knowledge) by two allocation strategies (per-item and per-test). Results replicate earlier findings that (1) reliability of speed measures in General Knowledge tests is as high or higher than the reliability of traditional correctness measures, and (2) speed measures in General Knowledge tests correlate highly with correctness scores. In addition, evidence is provided that (3) these reliability and (4) correlations of speed and accuracy measures hold only for "simple" items. However, (5), response time measures correlate rather highly across item types. (6) Since response time correlates with performance on other item types better than the correlation of the performance measures themselves, we have convergent/discriminant validation of this measure's stand alone validity. Keeping in mind resistance to change and examinee attitudes, further attention should be given to the information provided by measures of time in the human-computer communication bandwidth.