Модель предсказания аффективных характеристик цветовой палитры изображений

Svetlana Morozova

doi:10.21638/spbu16.2025.106

Authors

Svetlana Morozova St. Petersburg State University, 7–9, Universitetskaya nab., St. Petersburg, 199034, Russian Federation

DOI:

https://doi.org/10.21638/spbu16.2025.106

Abstract

The study uses computer vision and machine learning technologies to assess the relationship between the emotional labeling of images and the characteristics of their color palette. The color palette of images in the HSV space was estimated using a modified image clustering algorithm, in which the cluster centroids were determined arbitrarily. The colors of the Luscher test were taken as the centroids of the clusters. The stimulus material for training the random forest model was taken from the Open affective standardized image set (OASIS) database. The parameters of the final random forest model for the test set: AUC = 0.77, Accuracy = 0.744, Kappa = 0.489; for the training set: AUC = 0.969, Accuracy = 0.904, Kappa = 0.808. The achieved classification accuracy can be interpreted as sufficient, provided that the original corpus labeling, on which the training took place, has emotional labeling that considers not only low-level characteristics of images, but also the semantics of scenes. To test the validity of the model, we: 1) used a pre-trained random forest model to estimate the emotional valence of a base of artistic photographs and a base of abstract images; 2) statistically assessed the quality of valence assessment of specific emotions in artistic photographs and abstract
images; 3) compared the results obtained for both bases. The obtained results allow us to conclude that the proposed random forest model is applicable to solving problems of image classification by color palette characteristics. If the confidence in classifying images as positive or negative is more than 60 %, we can predict that the color palette of the stimulus material will induce amusement, awe, excitement, fear, sad and content. The quality of recognition of negative emotions such as anger and disgust are not good enough. The proposed model is recommended for hybrid labeling of stimulus material, especially in cases of primary assessment of the emotional valence of images.

Keywords:

affective marking, Lusher color test, image clustering, random forest model

Downloads

Download data is not yet available.

References

Базыма Б. А. Психология цвета: Теория и практика. СПб.: Речь, 2007.

Дубровская О. Ф. (сост.). Руководство по использованию восьмицветового теста Люшера. М.: Когито-Центр, 2008.

Маккини У. Python и анализ данных. М.: ДМК Пресс, 2020.

Мастицкий С. Э., Шитиков В. К. Статистический анализ и визуализация данных с помощью R. М.: ДМК, 2015.

Морозова С. В. Анализ текстов и изображений в психологических исследованиях с помощью библиотеки «psyscaling» // Ананьевские чтения — 2023. 60 лет социальной психологии в СПбГУ: Человек в современном мире: потенциалы и перспективы психологии развития. [Б. м.]: Кириллица, 2023.

Солем Я. Программирование компьютерного зрения на языке Python. М.: ДМК Пресс, 2016.

Хейт М., Груздев А. Изучаем Pandas. М.: ДМК Пресс, 2019.

Bartoszek G., Cervone D. Toward an implicit measure of emotions: Ratings of abstract images reveal distinct emotional states // Cognition and Emotion. 2017. Vol. 31 (7). P. 1377–1391.

Benjamini Yo., Yekutieli D. The control of the false discovery rate in multiple testing under dependency // Annals of statistics. 2001. Vol. 29 (4). P. 1165–1188.

Kurdi B., Lozano S., Banaji M. R. Introducing the open affective standardized image set (OASIS) // Behavior research methods. 2017. Vol. 49. P. 457–470.

Lüscher M. Psychologie der Farben. English The Lüscher color test / transl. and ed. by I. A. Scott. New York: Random House, [1969].

Machajdik J., Hanbury A. Affective image classification using features inspired by psychology and art theory // Proceedings of the 18th ACM international conference on Multimedia (MM ‘10). New York: ACM, 2010. P. 83–92. https://doi.org/10.1145/1873951.1873965

Sokolov E. N., Boucsein W. A psychophysiological model of emotion space // Integrative Physiological and Behavioral Science. 2000. Vol. 35. P. 81–119.

Yeo I. K., Johnson R. A. A new family of power transformations to improve normality or symmetry // Biometrika. 2000. Vol. 87 (4). P. 954–959. https://doi.org/10.1093/biomet/87.4.954

Zhao S., Huang Q., Tang Yo., Yao X., Yang J., Ding G., Schuller B. W. Computational emotion analysis from images: Recent advances and future directions // Human Perception of Visual information: psychological and computational perspectives. Berlin: Springer International Publishing, 2022. P. 85–113.

Zhao S., Yao Z., Yang J., Jia G., Ding G., Chua T.-S., Schuller B. W., Keutzer K. Affective image content analysis: Two decades review and new perspectives // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2021. Vol. 44 (10). P. 6729–6751.

References

Bartoszek, G., Cervone, D. (2017). Toward an implicit measure of emotions: Ratings of abstract images reveal distinct emotional states. Cognition and Emotion, 31 (7), 1377–1391.

Bazyma, B. A. (2007). Psychology of color: Theory and practice. St. Petersburg, Rech Publ (In Russian)

Benjamini, Yo., Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of statistics, 29 (4), 1165–1188.

Dubrovskaya, O. F. (2008). Manual on the use of the eight-color Lusher test. Moscow, Cogito-Centre Publ (In Russian)

Haight, M., Gruzdev, A. (2019). Learning pandas. Moscow, DMK Publ (In Russian)

Kurdi, B., Lozano, S., Banaji, M. R. (2017). Introducing the open affective standardized image set (OASIS). Behavior Research Methods, 49, 457–470.

Lüscher, M. ([1969]). Psychologie der Farben. English The Lüscher color test, transl. and ed. by I. A. Scott. New York, Random House

Machajdik, J., Hanbury, A. (2010). Affective image classification using features inspired by psychology and art theory. In: Proceedings of the 18th ACM international conference on Multimedia (MM ‘10) (pp. 83–92). New York, ACM https://doi.org/10.1145/1873951.1873965

Mastitsky, S. E., Shitikov, V. K. (2015). Statistical analysis and data visualisation using R. Moscow, DMK Publ (In Russian)

McKinney, W. (2020). Python and data analysis. Moscow, DMK Publ (In Russian)

Morozova, S. V. (2023). Text and image analysis in psychological research using the library ‘psyscaling’. In: Anan’evskie chteniia — 2023. 60 let social’noi psikhologii v SPbGU: Chelovek v sovremennom mire: potentsialy i perspektivy psikhologii razvitiia. Kirillitsa Publ (In Russian)

Sokolov, E. N., Boucsein, W. (2000). A psychophysiological model of emotion space. Integrative Physiological and Behavioral Science, 35, 81–119.

Solem, J. (2016). Programming computer vision in the Python language. Moscow, DMK Publ (In Russian)

Yeo, I. K., Johnson, R. A. (2000). A new family of power transformations to improve normality or symmetry. Biometrika, 87 (4), 954–959. https://doi.org/10.1093/biomet/87.4.954