An assessment of estimation procedures for multilevel models with binary responses

We evaluate two software packages that are available for fitting multilevel models to binary response data, namely VARCL and ML3, by using a Monte Carlo study designed to represent quite closely the actual structure of a data set used in an analysis of health care utilization in Guatemala. We find that the estimates of fixed effects and variance components produced by the software packages are subject to very substantial downward bias when the random effects are sufficiently large to be interesting. In fact, the fixed effect estimates are no better than the estimates obtained by using standard logit models that ignore the hierarchical structure of the data. The estimates of standard errors appear to be reasonably accurate and superior to those obtained by ignoring clustering, although one might question their utility in the presence of large biases. We conclude that alternative estimation procedures need to be developed and implemented for the binary response case.