Optimizing the expected overlap of survey samples via the northwest corner rule

In survey sampling there is often a need to coordinate the selection of pairs of samples drawn from two overlapping populations so as to maximize or minimize their expected overlap, subject to constraints on the marginal probabilities determined by the respective designs. For instance, maximizing the expected overlap between repeated samples can stabilize the resulting estimates of change and reduce the costs of first contacts; minimizing the expected overlap can avoid overburdening respondents with multiple surveys. We focus on the important special case in which both samples are selected by simple random sampling without replacement (SRSWOR) conducted independently within each stratum. Optimizing the expected sample overlap can be formulated as a linear programming problem known as a transportation problem (TP). We show that by appropriately grouping and ordering the possible samples in each survey, one can reduce the initial TP to a much smaller TP amenable to solution by an algorithm known as the Northwest Corner Rule (NWCR). The proposed NWCR method proceeds in two easily implemented steps: first selecting the numbers of births (new units) and deaths (deleted units) by a random selection from a hypergeometric distribution, and then selecting the births and deaths by SRSWOR. We formally prove properties of the NWCR solutions, including a minimal variance property of the minimal overlap solution. In a simulation study, the NWCR method compares favorably with a popular method based on assignment of permanent random numbers to each sampling unit.