Response to Estimating Pr(X < Y) in Categorized Data using "ROC" Analysis

Simonoff, Hochberg, and Reiser (1986, Biometrics 42, 895-907) considered estimation of λ = Pr(X < Y) - Pr(Y < X) and R = Pr(X < Y) when observations on X and Y are presented in discretized (categorized) form. Two new methods were proposed and compared by Monte Carlo with maximum likelihood (ML) estimation assuming X, Y normal and known category boundaries, and with the nonparametric Wilcoxon-Mann-Whitney (WMW) estimator. One of the new methods, called PML, involves a WMW-type function of the cell probabilities estimated by ML assuming X, Y normal and known category boundaries. PML was recommended for situations where λ is large, data are not sparse, and X, Y "near normal." This note calls attention to another method that has seen considerable use in applications in psychology and medicine, and involves ML estimation assuming existence of an (unknown) transformation such that on the transformed scale X and Y are normal. Sometimes referred to as "ROC analysis," this method does not require known category boundaries (in contrast to PML) and is shown to compare favorably indeed with PML in a small Monte Carlo study. The choice of estimator, however, should depend also on whether a smooth underlying distribution is assumed for X and Y.