Why are pickpockets so efficient? It’s partly because they can identify objects they’ve never seen before just by touching them. Rather than removing a notebook or water bottle from your bag, pickpockets can immediately locate your phone or wallet and take only those valuable items.
“We all have similar abilities to imagine what an object looks like by feeling it and, conversely, what an object feels like by looking at it,” said Gabor Lengyel, a PhD student in CEU’s Department of Cognitive Science and lead author of a new study in eLife examining this phenomenon.
Now imagine that you go into a shop looking for a new wallet to replace the one that was stolen from your bag. You can anticipate how it would feel to hold and open each wallet in the display window just by seeing them—you don’t actually need to take them down one-by-one to know.
These skills rely on our ability to break up the continuous stream of our sensory input into discrete chunks. When reaching into your bag, the pickpocket interprets the sequence of small depressions on their fingers as a set of identifiable objects. When you look at the shop window, your visual system interprets the myriad photons reaching your eyes as a selection of wallets, purses and other goods. Our ability to identify discrete objects by vision or touch alone, as well as the abilities to predict how objects will feel from vision and how they will look from touch, are critical to how we interact with the world.
The human brain achieves these feats by performing clever statistical analyses of previous experiences, according to the study “Unimodal statistical learning produces multimodal object-like representations” co-authored by several researchers from CEU’s Department of Cognitive Science, the University of Cambridge’s Computational and Biological Learning Lab and Columbia University’s Zuckerman Mind Brain Behavior Institute.
“By analyzing previous experiences, the brain can immediately identify objects without the need for clear-cut boundaries or other specialized cues. It can also predict unknown properties of new objects,” said Mate Lengyel, senior research fellow at CEU and professor of computational neuroscience at Cambridge.
Study participants were asked either to observe visually (without touching) or gather haptic information (by physically pulling on) different scenes created with objects resembling jigsaw puzzle pieces. They were then tested on whether they could predict additional properties of those objects. Those who only visually observed the objects were asked to predict how hard it would be to physically tear apart the objects. Those who only touched the objects were asked how similar the objects looked to actual jigsaw puzzle pieces. The majority of study participants formed correct mental models of the objects from either visual or tactile experience alone and were able to immediately perform “zero-shot generalization” and accurately infer tactile properties from visual experiences and vice versa.
These results challenge the commonly accepted idea about how humans learn about the physical world around us.
“Classical views assumed that specialized tactile or visual cues—such as edges, borders between different objects or even explicit instructions—were necessary to learn about objects in our environment. Instead, our study shows that general-purpose statistical computations known to operate in even the youngest infants are powerful enough to achieve such cognitive feats,” explained Jozsef Fiser, director of CEU’s Center for Cognitive Computation.
Notably, the participants in the study were not selected for being professional pickpockets, suggesting there is a secret, statistically savvy pickpocket in all of us.
“Unimodal statistical learning produces multimodal object-like representations” was co-authored by Gabor Lengyel, Jozsef Fiser and Mate Lengyel of CEU and Goda Zalalyte, Alexandros Pantelides, James Ingram and Daniel Wolpert of Cambridge and Columbia.
