We studied the effects of the reliability of ATR (Automatic Target Recognition) designations on the performance of expert image analysts of SAR (Synthetic Aperture Radar) images. We conducted an experiment to test the performance of 12 expert SAR images analysts while using ATR at different levels of reliability (80%, 50%, and 33%). The experiment systematically manipulated the reliability levels of simulated ATR systems, and measured their influence on the performance of human analysts. Reliability rate was defined as the percentage of the designated items (by an ATR system) which are actually correct targets. Each ATR block was paired with a similar non-ATR block, a design that aimed to evaluate the added value of the ATR system to the performance of the human analysts. In addition, a complete within-subjects design was used, providing a good basis for comparing the different conditions in the experiment. The study showed that analyst performance was influenced by ATR reliability. Higher reliability yielded higher hit rates and higher false alarm rates, while low reliability yielded opposite results. This pattern of results, combined with a signal detection theory analysis, indicates that ATR reliability affects response criterion and not performance per se. The tendency to mark more items that were designated by the ATR as true targets should improve the overall performance of analysts working with state-of-the-art ATR systems. We concluded that when developing an ATR system, the image analysts should be provided with valid assessments of the system's reliability.
