¹³One issue that has been of concern to some commentators is the use of the development team for the evaluation. The obvious concern here is that the development team might be so invested in the success of the instrument that it might unwittingly fail to provide an unbiased evaluation. A number of factors make this theoretical concern one that is very unlikely to have operated in reality. First, the development team was committed to improving the quality of care and quality of life experienced by nursing home residents. Thus, like others, the team wanted to know whether the system worked as intended. More important, a number of external safeguards were in place. The evaluation design and analysis plan were reviewed and approved by a group of senior external researchers brought in by CMS to ensure that the evaluation would be scientifically sound. In addition, all the evaluation results were published in peer-reviewed journals, and thus faced careful scrutiny by anonymous reviewers. Finally, "objectivity" in research is not something achieved merely by separating development from evaluation. Developers can be scrupulous in their adherence to scientific norms and as objective as any other evaluators in their analyses of results, and any evaluator may bring a set of preconceived notions to the task. Similarly, the demands of clients can drive evaluations performed by anyone. Individual researchers—developers or evaluators—can be so wedded to a hypothesis that they unconsciously skew their observations. There are no easy structural remedies for this problem. In the end, good science demands that researchers adhere to the fundamental rules of scientific inquiry and that they let others review their design and results. Both these requirements were met by the RAI evaluation. [Return to Text]