Multiscale image segmentation using joint texture and shape analysis

We develop a general framework to simultaneously exploit texture and shape characterization in multiscale image segmentation. By posing multiscale segmentation as a model selection problem, we invoke the powerful framework offered by minimum description length (MDL). This framework dictates that multiscale segmentation comprises multiscale texture characterization and multiscale shape coding. Analysis of current multiscale maximum a posteriori (MAP) segmentation algorithms reveals that these algorithms implicitly use a shape coder with the aim to estimate the optimal MDL solution, but find only an approximate solution. Towards achieving better segmentation estimates, we first propose a shape coding algorithm based on zero-trees which is well-suited to represent images with large homogeneous regions. For this coder, we design an efficient tree-based algorithm using dynamic programming that attains the optimal MDL segmentation estimate. To incorporate arbitrary shape coding techniques into segmentation, we design an iterative algorithm that uses dynamic programming for each iteration. Though the iterative algorithm is not guaranteed to attain exactly optimal estimates, it more effectively captures the prior set by the shape coder. Experiments demonstrate that the proposed algorithms yield excellent segmentation results on both synthetic and real world data examples.