The effects of the reliability of an automatic target recognition system on image analyst performance

Motivation - To study the effects of the reliability of ATR (Automatic Target Recognition) designations on the performance of expert image analysts of SAR (Synthetic Aperture Radar) images. Research approach - A psychophysical study of the performance of 12 expert analysts of SAR images. Findings - Analyst performance was influenced by ATR reliability. Higher reliabilities yielded higher hit rates and higher false alarm rates, and low reliabilities the opposite results. This and a signal detection theory analysis indicate that ATR reliability affects the response criterion and not performance per se. (But see Discussion). Research Implications - The fact that the reliability of items designated by the ATR system affected the criterion of the analysts has important implications. The tendency to mark more items that were designated by the ATR as being true targets should improve the overall performance of analysts working with state-of-the-art ATR systems (see Discussion). Originality/Value - The research systematically manipulated the reliability levels of simulated ATR systems, and measured their influence on the performance of human analysts. In this context reliability rate means what percentage of the designated items by an ATR system are actually correct targets. Each ATR block was coupled with a similar non-ATR block, a design that aimed to extract the added value of the ATR system to the performance of the human analysts. In addition, a complete within subjects design was used. This procedure provided a good basis for comparing the different conditions in the experiment. Take away message - While developing an ATR system, one should provide the image analysts with valid assessments of the system's reliability.