close
Skip to main content
Springer Nature Link
Log in

Advancing neural aesthetic assessment of artistic images based on bundle features integration

  • Research
  • Published:
Save article
View saved research
The Visual Computer Aims and scope Submit manuscript

Abstract

The aesthetic assessment of images is a popular research topic due to its practical applications in various fields such as image recommendation, image ranking, and image search. Currently, most research on image aesthetic assessment relies on large-scale photography datasets, such as AVA and AADB, primarily composed of photos taken by users in real-world scenarios. Few studies specifically focus on the automatic aesthetic assessment of artistic images. Artistic images are more complex, diverse, and abstract compared to photographic images. In this paper, we propose a convolutional neural network model to automatically generate aesthetic scores for input artistic images. Unlike previous research, this study explores artistic theories and introduces the analysis of aesthetic features in artistic images from three dimensions: color, brightness, and contour. These features are integrated to generate an overall aesthetic score. We utilize our own large-scale dataset of artistic images for aesthetic assessment, consisting of over 7,000 artistic images, each accompanied by corresponding average aesthetic scores assigned by users. We compare our model with state-of-the-art image aesthetic assessment models, demonstrating the effectiveness of our approach. Code is available at: https://github.com/ysmyan/aiaa

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
The alternative text for this image may have been generated using AI.
Fig. 2
The alternative text for this image may have been generated using AI.
Fig. 3
The alternative text for this image may have been generated using AI.
Fig. 4
The alternative text for this image may have been generated using AI.
Fig. 5
The alternative text for this image may have been generated using AI.
Fig. 6
The alternative text for this image may have been generated using AI.
Fig. 7
The alternative text for this image may have been generated using AI.
Fig. 8
The alternative text for this image may have been generated using AI.
Fig. 9
The alternative text for this image may have been generated using AI.
Fig. 10
The alternative text for this image may have been generated using AI.
Fig. 11
The alternative text for this image may have been generated using AI.
Fig. 12
The alternative text for this image may have been generated using AI.
Fig. 13
The alternative text for this image may have been generated using AI.
Fig. 14
The alternative text for this image may have been generated using AI.
Fig. 15
The alternative text for this image may have been generated using AI.

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Data availability

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Murray, N., Marchesotti, L., Perronnin, F.: Ava: a large-scale database for aesthetic visual analysis. In: 2012 IEEE conference on computer vision and pattern recognition (IEEE, 2012). https://doi.org/10.1109/cvpr.2012.6247954

  2. Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation, 662–679 (Springer International Publishing, 2016)

  3. Tang, X., Luo, W., Wang, X.: Content-based photo quality assessment. IEEE Trans. Multimed. 15, 1930–1943 (2013). https://doi.org/10.1109/tmm.2013.2269899

    Article  Google Scholar 

  4. Yi, R., Tian, H., Gu, Z., Lai, Y.-K., Rosin, P.L.: Towards artistic image aesthetics assessment: a large-scale dataset and a new method. In: 2023 IEEE/CVF conference on computer vision and pattern recognition (CVPR), https://doi.org/10.1109/cvpr52729.2023.02144 (IEEE, 2023)

  5. Lisa, C.: Colour theory: understanding and working with colour (RMIT Open Press, 2023)

  6. Stephen, P.E.: Photons to Phenomenology (The MIT Press, 1999)

  7. Mamassian, P.: Ambiguities and conventions in the perception of visual art. Vision. Res. 48, 2143–2153 (2008). https://doi.org/10.1016/j.visres.2008.06.010

    Article  Google Scholar 

  8. Cheng, Z., Yang, Q., Sheng, B.: Deep colorization. In: 2015 IEEE international conference on computer vision (ICCV) 415–423.https://doi.org/10.1109/ICCV.2015.55 (2015)

  9. Jin, Y., Sheng, B., Li, P., Chen, C.L.P.: Broad colorization. IEEE Trans. Neural Netw. Learn. Syst. 32, 2330–2343 (2021). https://doi.org/10.1109/TNNLS.2020.3004634

    Article  Google Scholar 

  10. Sheng, B., Li, P., Jin, Y., Tan, P., Lee, T.-Y.: Intrinsic image decomposition with step and drift shading separation. IEEE Trans. Visual Comput. Graphics 26, 1332–1346 (2020). https://doi.org/10.1109/TVCG.2018.2869326

    Article  Google Scholar 

  11. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition. https://doi.org/10.1109/cvpr.2018.00745 (IEEE, 2018)

  12. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models (2021). arXiv:2112.10752

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR) https://doi.org/10.1109/cvpr.2016.90 (IEEE, 2016)

  14. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in neural information processing systems, vol. 25 (Curran Associates, Inc., 2012)

  15. Lin, X., et al.: Eapt: Efficient attention pyramid transformer for image processing. IEEE Trans. Multimedia 25, 50–61 (2023). https://doi.org/10.1109/TMM.2021.3120873

    Article  Google Scholar 

  16. Huang, S., et al.: Transmrsr: transformer-based self-distilled generative prior for brain mri super-resolution. Vis. Comput. 39, 3647–3659 (2023). https://doi.org/10.1007/s00371-023-02938-3

    Article  Google Scholar 

  17. Zhou, Y., et al.: Fsad-net: Feedback spatial attention dehazing network. IEEE Trans. Neural Netw. Learn. Syst. 34, 7719–7733 (2023). https://doi.org/10.1109/TNNLS.2022.3146004

    Article  Google Scholar 

  18. Vaswani, A., et al.: Attention is all you need. In: Guyon, I. et al. (eds.) Advances in neural information processing systems, vol. 30 (Curran Associates, Inc., 2017)

  19. Ren, J., Shen, X., Lin, Z., Mech, R., Foran, D.J.: Personalized image aesthetics. In: 2017 IEEE international conference on computer vision (ICCV). https://doi.org/10.1109/iccv.2017.76 (IEEE, 2017)

  20. Yang, Y., et al.: Personalized image aesthetics assessment with rich attributes. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/cvpr52688.2022.01924 (IEEE, 2022)

  21. He, S., Zhang, Y., Xie, R., Jiang, D., Ming, A.: Rethinking image aesthetics assessment: models, datasets and benchmarks. In: Raedt, L. D. (ed.) Proceedings of the thirty-first international joint conference on artificial intelligence, IJCAI-22, 942–948, https://doi.org/10.24963/ijcai.2022/132 (International Joint Conferences on Artificial Intelligence Organization, 2022). Main Track

  22. Amirshahi, S.A., Hayn-Leichsenring, G.U., Denzler, J., Redies, C.: Jenaesthetics subjective dataset: analyzing paintings by subjective scores. In: Agapito, L., Bronstein, M. M. & Rother, C. (eds.) Computer Vision - ECCV 2014 Workshops 3–19. (Springer International Publishing, Cham, 2015)

  23. Talebi, H., Milanfar, P.: Nima: Neural image assessment. IEEE Trans. Image Process. 27, 3998–4011 (2018). https://doi.org/10.1109/tip.2018.2831899

    Article  MathSciNet  Google Scholar 

  24. Levina, E., Bickel, P.: The earth mover’s distance is the mallows distance: some insights from statistics. In: Proceedings eighth ieee international conference on computer vision. ICCV 2001 ICCV-01. https://doi.org/10.1109/iccv.2001.937632 (IEEE Comput. Soc, 2001)

  25. Hosu, V., Goldlucke, B., Saupe, D.: Effective aesthetics prediction with multi-level spatially pooled features. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) (2019)

  26. Ma, S., Liu, J., Chen, C.W.: A-lamp: adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR) https://doi.org/10.1109/cvpr.2017.84 (IEEE, 2017)

  27. Wang, L., Wang, X., Yamasaki, T.: Image aesthetics prediction using multiple patches preserving the original aspect ratio of contents. Multimed. Tools Appl. 82, 2783–2804 (2022). https://doi.org/10.1007/s11042-022-13333-w

    Article  Google Scholar 

  28. She, D., Lai, Y.-K., Yi, G., Xu, K.: Hierarchical layout-aware graph convolutional network for unified aesthetics assessment. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/cvpr46437.2021.00837 (IEEE, 2021)

  29. Lv, P., et al.: User-guided personalized image aesthetic assessment based on deep reinforcement learning. IEEE Trans. Multimedia 25, 736–749 (2023). https://doi.org/10.1109/TMM.2021.3130752

    Article  Google Scholar 

  30. Zhu, H., Zhou, Y., Li, L., Li, Y., Guo, Y.: Learning personalized image aesthetics from subjective and objective attributes. IEEE Trans. Multimedia 25, 179–190 (2023). https://doi.org/10.1109/TMM.2021.3123468

    Article  Google Scholar 

  31. Zhu, H., et al.: Personalized image aesthetics assessment with attribute-guided fine-grained feature representation. In: Proceedings of the 31st ACM International Conference on Multimedia, MM ’23, https://doi.org/10.1145/3581783.3611942 (ACM, 2023)

  32. He, S., Ming, A., Zheng, S., Zhong, H., Ma, H.: Eat: An enhancer for aesthetics-oriented transformers. In: Proceedings of the 31st ACM international conference on multimedia MM ’23. https://doi.org/10.1145/3581783.3611881 (ACM, 2023)

  33. Niu, Y., Chen, S., Song, B., Chen, Z., Liu, W.: Comment-guided semantics-aware image aesthetics assessment. IEEE Trans. Circuits Syst. Video Technol. 33, 1487–1492 (2023). https://doi.org/10.1109/TCSVT.2022.3201510

    Article  Google Scholar 

  34. Nie, X., et al.: Bmi-net: a brain-inspired multimodal interaction network for image aesthetic assessment. In: Proceedings of the 31st ACM International Conference on Multimedia MM ’23. https://doi.org/10.1145/3581783.3611996 (ACM, 2023)

  35. Sheng, X., et al.: Aesclip: multi-attribute contrastive learning for image aesthetics assessment. In: Proceedings of the 31st ACM international conference on multimedia, MM ’23. https://doi.org/10.1145/3581783.3611969 (ACM, 2023)

  36. Amirshahi, S.A., Denzler, J.: Judging aesthetic quality in paintings based on artistic inspired color features. In: 2017 international conference on digital image computing: techniques and applications (DICTA), https://doi.org/10.1109/dicta.2017.8227452 (IEEE, 2017)

  37. Guo, X., Kurita, T., Asano, C.M., Asano, A.: Visual complexity assessment of painting images. In: 2013 IEEE international conference on image processing. https://doi.org/10.1109/icip.2013.6738080 (IEEE, 2013)

  38. Li, C., Chen, T.: Aesthetic visual quality assessment of paintings. IEEE J. Sel. Top. Sign. Proc. 3, 236–252 (2009). https://doi.org/10.1109/jstsp.2009.2015077

    Article  Google Scholar 

  39. Zhang, J., Miao, Y., Zhang, J., Yu, J.: Inkthetics: a comprehensive computational model for aesthetic evaluation of chinese ink paintings. IEEE Access 8, 225857–225871 (2020). https://doi.org/10.1109/access.2020.3044573

    Article  Google Scholar 

  40. Chang, H., Fried, O., Liu, Y., DiVerdi, S., Finkelstein, A.: Palette-based photo recoloring. ACM Tran. Gr. 34, 1–11 (2015). https://doi.org/10.1145/2766978

    Article  Google Scholar 

  41. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, 315–323 (JMLR Workshop and Conference Proceedings, 2011)

  42. Dubey, S.R., et al.: diffgrad: An optimization method for convolutional neural networks. IEEE Trans. Neural Netw. Learn. Syst. 31, 4500–4511 (2020). https://doi.org/10.1109/tnnls.2019.2955777

    Article  MathSciNet  Google Scholar 

  43. Selvaraju, R.R., et al.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: 2017 IEEE international conference on computer vision (ICCV). https://doi.org/10.1109/iccv.2017.74 (IEEE, 2017)

  44. He, S., Ming, A., Zheng, S., Zhong, H., Ma, H.: Eat: An enhancer for aesthetics-oriented transformers. In: Proceedings of the 31st ACM international conference on multimedia, MM ’23. https://doi.org/10.1145/3581783.3611881 (ACM, 2023)

  45. He, S., et al.: Thinking image color aesthetics assessment: Models, datasets and benchmarks. In: 2023 IEEE/CVF international conference on computer vision (ICCV). https://doi.org/10.1109/iccv51070.2023.01996 (IEEE, 2023)

  46. Zhu, H., et al.: Personalized image aesthetics assessment via meta-learning with bilevel gradient optimization. IEEE Trans. Cybern. 52, 1798–1811 (2022). https://doi.org/10.1109/tcyb.2020.2984670

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University, Hangzhou, China

    Simin Yan & Sanyuan Zhang

  2. College of Information Science and Technology, Hangzhou Normal University, Hangzhou, China

    Shuchang Xu

  3. Beijing Dailybread Co., Ltd., Beijing, China

    Aiping Lei

Authors
  1. Simin Yan
    View author publications

    Search author on:PubMed Google Scholar

  2. Shuchang Xu
    View author publications

    Search author on:PubMed Google Scholar

  3. Aiping Lei
    View author publications

    Search author on:PubMed Google Scholar

  4. Sanyuan Zhang
    View author publications

    Search author on:PubMed Google Scholar

Contributions

SY contributed to writing (original draft), coding and conducting experiments. SX contributed to algorithm idea, writing (review and editing) and experiments design. AL contributed to conducting experiments. SZ contributed to supervision and project administration. All authors reviewed the manuscript.

Corresponding author

Correspondence to Shuchang Xu.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, S., Xu, S., Lei, A. et al. Advancing neural aesthetic assessment of artistic images based on bundle features integration. Vis Comput 41, 5447–5459 (2025). https://doi.org/10.1007/s00371-024-03732-5

Download citation

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s00371-024-03732-5

Keywords