Texture Interpolation for Probing Visual Perception

09 Dec 2020 in Publications

with A. Davila, A. Kohn and R. Coen-Cagli.

Live stream 12/9/20 at 7:10 PM PST

Q&A session 7:40 PM PST

Poster

Rocket Chat

GatherTown

We use the Optimal Transport framework to interpolate between textures and use these textures in a perceptual task. We also recorded the macaque primary and mid-level visual cortex.

1. Vacher, J., Davila, A., Kohn, A. & Coen-Cagli, R. Texture Interpolation for Probing Visual Perception. in Advances in Neural Information Processing Systems (2020).

   @inproceedings{vacher2020texture,
     title = {Texture Interpolation for Probing Visual Perception},
     author = {Vacher, Jonathan and Davila, Aida and Kohn, Adam and Coen-Cagli, Ruben},
     journal = {Advances in Neural Information Processing Systems},
     year = {2020}
   }
   

Abstract

Texture synthesis models are important to understand visual processing. In particu-lar, statistical approaches based on neurally relevant features have been instrumentalto understanding aspects of visual perception and of neural coding. New deeplearning-based approaches further improve the quality of synthetic textures. Yet, itis still unclear why deep texture synthesis performs so well, and applications ofthis new framework to probe visual perception are scarce. Here, we show that dis-tributions of deep convolutional neural network (CNN) activations of a texture arewell described by elliptical distributions and therefore, following optimal transporttheory, constraining their mean and covariance is sufficient to generate new texturesamples. Then, we propose the natural geodesics (i.e.the shortest path between twopoints) arising with the optimal transport metric to interpolate between arbitrarytextures. The comparison to alternative interpolation methods suggests that oursmatches more closely the geometry of texture perception, and is better suited tostudy its statistical nature. We demonstrate our method by measuring the perceptualscale associated to the interpolation parameter in human observers, and the neuralsensitivity of different areas of visual cortex in macaque monkeys.