A map of object space in primate inferotemporal cortex
Abstract
The inferotemporal (IT) cortex is responsible for object recognition, but it is unclear how the representation of visual objects is organized in this part of the brain. Areas that are selective for categories such as faces, bodies, and scenes have been found, but large parts of IT cortex lack any known specialization, raising the question of what general principle governs IT organization. Here we used functional MRI, microstimulation, electrophysiology, and deep networks to investigate the organization of macaque IT cortex. We built a low-dimensional object space to describe general objects using a feedforward deep neural network trained on object classification. Responses of IT cells to a large set of objects revealed that single IT cells project incoming objects onto specific axes of this space. Anatomically, cells were clustered into four networks according to the first two components of their preferred axes, forming a map of object space. This map was repeated across three hierarchical stages of increasing view invariance, and cells that comprised these maps collectively harboured sufficient coding capacity to approximately reconstruct objects. These results provide a unified picture of IT organization in which category-selective regions are part of a coarse map of object space whose dimensions can be extracted from a deep network.
Additional Information
© 2020 Springer Nature Limited. Received 21 January 2019; Accepted 17 March 2020; Published 03 June 2020. This work was supported by NIH (DP1-NS083063, R01-EY030650), the Howard Hughes Medical Institute, and the Tianqiao and Chrissy Chen Institute for Neuroscience at Caltech. We thank A. Flores for technical support, and members of the Tsao laboratory, N. Kanwisher, A. Kennedy, S. Kornblith, and A. Tsao for critical comments. Author Contributions: P.B. and D.Y.T. designed the experiments, P.B. and L.S. collected the data, and P.B. analysed the data. M.M. provided technical advice on neural networks. P.B. and D.Y.T. interpreted the data and wrote the paper. Data availability: The data that support the findings of this study are available from the lead corresponding author (D.Y.T.) upon reasonable request. The authors declare no competing interests.Attached Files
Accepted Version - nihms-1603784.pdf
Submitted - Bao_Nature_Manuscript.pdf
Supplemental Material - 41586_2020_2350_Fig10_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig11_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig12_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig13_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig14_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig15_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig16_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig6_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig7_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig8_ESM.jpg
Supplemental Material - 41586_2020_2350_Fig9_ESM.jpg
Supplemental Material - 41586_2020_2350_MOESM1_ESM.pdf
Supplemental Material - 41586_2020_2350_MOESM2_ESM.pdf
Files
Name | Size | Download all |
---|---|---|
md5:237a97cc55eb014346b1b7d2a7edb51d
|
538.8 kB | Preview Download |
md5:b0dc5d648d3a0ed1adfe0c7119985614
|
395.9 kB | Preview Download |
md5:927e21960c0135c41f8908c28fb2d23b
|
290.0 kB | Preview Download |
md5:938351bf04da0f52b04dff4a950205d9
|
330.2 kB | Preview Download |
md5:a454d0641bbd9f982b2a8d85658c9555
|
405.4 kB | Preview Download |
md5:693a72932b3756a9d57958905a7ab672
|
396.4 kB | Preview Download |
md5:af7543dec42d56b728356d436b97b4cd
|
747.8 kB | Preview Download |
md5:5179b6ecd5610b32e066ad50c4321679
|
2.7 MB | Preview Download |
md5:9bb3233b6b959cf2843477017a770328
|
142.2 kB | Preview Download |
md5:428341f5564ddc44b2cc45388a873557
|
201.3 kB | Preview Download |
md5:97c6e234e578b2932c2918c91311aff3
|
74.9 kB | Preview Download |
md5:4e9b599c599563e2d495111d7af91e08
|
346.6 kB | Preview Download |
md5:fd7b7de740022a40540aac076cc76b1c
|
332.9 kB | Preview Download |
md5:ddaec64364de59e135dcc25fe4eff48f
|
486.6 kB | Preview Download |
md5:eaa16419fc519c43ee0923b107ced2ec
|
199.9 kB | Preview Download |
Additional details
- PMCID
- PMC8088388
- Eprint ID
- 101514
- DOI
- 10.1038/s41586-020-2350-5
- Resolver ID
- CaltechAUTHORS:20200224-161613363
- NIH
- DP1-NS083063
- NIH
- R01-EY030650
- Howard Hughes Medical Institute (HHMI)
- Tianqiao and Chrissy Chen Institute for Neuroscience
- Created
-
2020-04-27Created from EPrint's datestamp field
- Updated
-
2023-07-18Created from EPrint's last_modified field
- Caltech groups
- Tianqiao and Chrissy Chen Institute for Neuroscience, Division of Biology and Biological Engineering