Introduction to NIREP
Non-rigid image registration (NIR) is an essential tool for morphologic comparisons in the presence of intra and inter-individual anatomic variations. Many NIR methods have been developed, but are especially difficult to evaluate since point-wise inter-image correspondence is usually unknown. We propose to develop and test a framework for comprehensive NIR method evaluation that does not require a "Gold Standard" or ground truth correspondence map. The Non-rigid Image Registration Evaluation Project (NIREP) will develop software tools and provide shared image validation databases for rigorous testing of non-rigid image registration algorithms. NIREP will extend the scope of prior validation projects by developing evaluation criteria and metrics using large image populations, using richly annotated image databases, using computer simulated data, and increasing the number and types of evaluation criteria.
Prior Studies
To date, few attempts have been made to objectively evaluate and compare the performance of image registration algorithms using standard evaluation criteria. Two projects that stand out in this regard are the “Retrospective Image Registration and Evaluation Project” led by J. Michael Fitzpatrick of Vanderbilt University for evaluating multimodality rigid registration accuracy and the non-rigid registration evaluation project entitled “Retrospective Evaluation of Inter-subject Brain Registration” led by Christian Barillot of IRISA/INRIACNRS Rennes, France. In both of these projects, a common set of images was used to evaluate the performance of registration algorithms. Developers from around the world participated in these projects by registering the images with their own registration algorithms and sending the resulting transformations back to the home site for analysis. The benefits of involving external participants include eliminating implementation biases, distributing the processing load, and providing an incentive to produce good results.
Another important validation/evaluation project is the VALMET software tool for assessing and improving 2D and 3D object segmentation developed by Guido Gerig et al. (www.ia.unc.edu/public/valmet/). The VALMET software was the first publicly available software tool for measuring and visualizing the differences between multiple corresponding medical image segmentations. It includes four algorithms for comparing segmentations: overlap ratio, Haussdorf distance, surface distance, and probabilistic overlap. The NIREP software evaluates image registration algorithm performance similar to the way the VALMET software evaluates image segmentation performance. In the future, all the VALMET evaluation metrics will be incorporated in to the NIREP software since automatic image segmentation produced from image registration is often used to evaluate registration performance.
