• ISSN: 2010-3646
    • Frequency: Bimonthly (2011-2014); Monthly (2015-2018); Quarterly (Since 2019)
    • DOI: 10.18178/IJSSH
    • Editor-in-Chief: Prof. Aurica Briscaru
    • Executive Editor: Mr. Ron C. Wu
    • Abstracting/ Indexing: Google Scholar, Index Copernicus, Crossref, Electronic Journals Library
    • E-mail: ijssh@ejournal.net
IJSSH 2019 Vol.9(1): 6-12 ISSN: 2010-3646
doi: 10.18178/ijssh.2019.V9.981

Integrating Vision and Language in Social Networks for Identifying Visual Patterns of Personality Traits

Pau Rodriguez, Jordi Gonzàlez, Josep M. Gonfaus, and F. Xavier Roca
Abstract—Social media, as a major platform for communication and information exchange, is a rich repository of the opinions and sentiments of 2.3 billion users about a vast spectrum of topics. In this sense, user text interactions are widely used to sense the whys of certain social user’s demands and cultural- driven interests. However, the knowledge embedded in the 1.8 billion pictures which are uploaded daily in public profiles has just started to be exploited. Following this trend on visual-based social analysis, we present a novel methodology based on neural networks to build a combined image-and-text based personality trait model, trained with images posted together with words found highly correlated to specific personality traits. So, the key contribution in this work is to explore whether OCEAN personality trait modeling can be addressed based on images, here called MindPics, appearing with certain tags with psychological insights. We found that there is a correlation between posted images and the personality estimated from their accompanying texts. Thus, the experimental results are consistent with previous cyber-psychology results based on texts, suggesting that images could also be used for personality estimation: classification results on some personality traits show that specific and characteristic visual patterns emerge, in essence representing abstract concepts. These results open new avenues of research for further refining the proposed personality model under the supervision of psychology experts, and to further substitute current textual personality questionnaires by image-based ones.

Index Terms—Personality trait analysis, deep learning, visual classification, OCEAN model, social networks.

P. Rodriguez, F. Xavier Roca, and J. Gonzàlez are with the Computer Vision Center, Edifici O, Campus Universitat Autònoma de Barcelona, 08193 Bellaterra (Cerdanyola del Vallès), Barcelona, Catalonia Spain (e-mails: prodriguez, xavir, poal@cvc.uab.es).
J. M. Gonfaus is with Visual Tagging Services S.L., Parc de Recerca UAB, Edifici Eureka, Campus Universitat Autònoma de Barcelona, 08193 Bellaterra (Cerdanyola del Vallès), Barcelona, Catalonia Spain (e-mail: pep.gonfaus@visual-tagging.com).

[PDF]

Cite: Pau Rodriguez, Jordi Gonzàlez, Josep M. Gonfaus, and F. Xavier Roca, "Integrating Vision and Language in Social Networks for Identifying Visual Patterns of Personality Traits," International Journal of Social Science and Humanity vol. 9, no. 1, pp. 6-12, 2019.

Copyright © 2008-2019. International Journal of Social Science and Humanity. All rights reserved.
E-mail: ijssh@ejournal.net