Social profiling through image understanding: Personality inference using convolutional neural networks
Abstract
The role of images in the last ten years has changed radically due to the advent of social networks: from media objects mainly used to communicate visual information, images have become personal, associated with the people that create or interact with them (for example, giving a "like"). Therefore, in the same way that a post reveals something of its author, so now the images associated to a person may embed some of her individual characteristics, such as her personality traits. In this paper, we explore this new level of image understanding with the ultimate goal of relating a set of image preferences to personality traits by using a deep learning framework. In particular, our problem focuses on inferring both self-assessed (how the personality traits of a person can be guessed from her preferred image) and attributed traits (what impressions in terms of personality traits these images trigger in unacquainted people), learning a sort of wisdom of the crowds. Our characterization of each image is locked within the layers of a CNN, allowing us to discover more entangled attributes (aesthetic patterns and semantic information) and to better generalize the patterns that identify a trait. The experimental results show that the proposed method outperforms state-of-the-art results and captures what visually characterizes a certain trait: using a deconvolution strategy we found a clear distinction of features, patterns and content between low and high values in a given trait.
Additional Information
© 2016 Elsevier Inc. Received 28 December 2015, Revised 30 September 2016, Accepted 19 October 2016, Available online 29 October 2016.Additional details
- Eprint ID
- 76607
- Resolver ID
- CaltechAUTHORS:20170417-134153718
- Created
-
2017-04-17Created from EPrint's datestamp field
- Updated
-
2021-11-15Created from EPrint's last_modified field