VSS Poster
in Conferences
Measuring and Modeling Human Probabilistic Segmentation Maps. Check the poster here and the short video presentation here. Joint work with Pascal Mamassian and Ruben Coen-Cagli
Visual segmentation is a core function of biological vision, key to adaptive behavior in complex environments. Early models inspired by the feedforward processing in the visual system described texture-based human segmentation as a comparison of the summary statistics of low-level image features across space. Here we consider the alternative view that, due to image ambiguity and sensory noise, perceptual segmentation requires probabilistic inference.
To test this hypothesis, we develop a novel paradigm to measure perceptual segmentation maps and their variability. We use composite textures: each segment is characterized by a different distribution of oriented features. Participants briefly view an image followed by two spatial cues and report whether the cued locations belong to the same segment. We repeat the sequence with different locations and reconstruct the full segmentation map from the binary choices, solving a system of equations. In a second set of experiments, we manipulate uncertainty by controlling the overlap between feature distributions and smoothing the texture boundary and measure texture discrimination performance.
We find that segmentation maps are similar across observers but variable: perceptual variability correlates with intrinsic image uncertainty, and both are higher near segment boundaries. We then test the inference model that consists in assigning pixels to segments by evaluating which distribution explains best the observed features. Quantitative model comparison shows that perceptual variability reflects image uncertainty beyond sensory noise and that human segmentation is better explained by optimal probabilistic inference than by comparing summary statistics. Lastly, we find an interaction between the effects of contour uncertainty and feature distribution overlap.
These results support the probabilistic inference hypothesis and suggest extending the model with contour specific components. Our work provides a normative explanation of human perceptual segmentation as probabilistic inference and demonstrates a novel framework to study perceptual segmentation, which could be extended to natural images.
