| Titre : | Recognizing the artistic style of fine art paintings with deep learning for an augmented reality application |
| Auteurs : | Baya Lina Menai, Auteur ; Mohamed Chaouki Babahenini, Directeur de thèse |
| Type de document : | Thése doctorat |
| Editeur : | Biskra [Algérie] : Faculté des Sciences Exactes et des Sciences de la Nature et de la Vie, Université Mohamed Khider, 2023 |
| Format : | 1 vol. (94 p.) / ill., couv. ill. en coul / 30 cm |
| Langues: | Français |
| Mots-clés: | Computer vision, Image processing, Convolutional neural network, Style Classification, Optimizers, Transfer learning. |
| Résumé : |
The rapid digitalization of artwork collections in libraries, museums, galleries, and art centers has resulted in a growing interest in developing autonomous systems capable of understanding art concepts and categorizing fine art paintings as it became difficult to manually manipulate the content of these collections. However, the task of automatic categorization comes with significant challenges due to the subjective interpretation and perception of art elements and the reliance on accurate annotations provided by art experts. As in recent years, deep learning approaches and computer vision techniques have shown remarkable performance in automating painting classification; this research aims to develop efficient deep learning systems that can automatically classify the artistic style of fine-art paintings. In this thesis, we investigate the effectiveness of seven pre-trained EfficientNet models for identifying the style of a painting and propose custom models based on pre-trained EfficientNet architectures. In addition, we analyzed the impact of deep retraining the last eight layers on the performance of the custom models. The experimental results on the standard fine art painting classification dataset, Painting-91 indicate that deep retraining of the last eight layers of the custom models yields the best performance, achieving a 5% improvement compared to the base models. This demonstrates the effectiveness of leveraging pre-trained EfficientNet models for automatic artistic style identification in paintings. Moreover, the study presents a framework that compares the performance InceptionResNetV2, DenseNet121, and EfficientNet B3) for identifying artistic styles in paintings. Notably, Xception architecture is employed for this purpose for the first time. Furthermore, the impact of different optimizers (SGD, RMSprop, and Adam) and two learning rates (1e-2 and 1e-4) on model performance is studied using transfer learning. The experiments on two different art classification datasets, Pandora18k and Painting-91 revealed that InceptionResNetV2 achieves the highest accuracy for style classification on both datasets when trained with the Adam optimizer and a learning rate of 1e-4. Integrating deep learning algorithms and transfer learning techniques into fine art painting analysis and classification offers promising avenues for automating style identification tasks. The proposed models and findings contribute to the development of automatic methods that enable the art community to efficiently analyze and categorize the vast number of digital paintings available on the internet. |
| Sommaire : |
Acknowledgments i Dedication ii Abstract iii R´esum´e iv Contents vi List of Figures x List of Tables xiii List of Abbreviations xv List of Publications xvii 1 Introduction 1 1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Research Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Fine-art Paintings: A Comprehensive Exploration of Artistic Styles and Challenges in Painting 5 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 Preview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2.1 Fine art paintings . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2.2 Artistic style . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2.3 The role of artistic style and techniques . . . . . . . . . . . . . 9 2.3 Style classification by art experts . . . . . . . . . . . . . . . . . . . . 9 2.3.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3.2 Challenges and difficulties . . . . . . . . . . . . . . . . . . . . 10 2.4 Fine art paintings and augmented reality . . . . . . . . . . . . . . . . 13 2.4.1 Augmented reality . . . . . . . . . . . . . . . . . . . . . . . . 13 2.4.2 Integration of augmented reality in art . . . . . . . . . . . . . 14 2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3 Style Classification : Literature Review 16 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.2 Approaches for style classification . . . . . . . . . . . . . . . . . . . . 17 3.2.1 Classical approaches for style classification . . . . . . . . . . . 18 3.2.2 Deep learning approaches for style classification . . . . . . . . 21 3.2.2.1 Supervised learning approaches for style classification 21 3.2.2.1.1 Convolutional Neural Networks(CNN) . . . 22 3.2.2.1.2 Transfer learning . . . . . . . . . . . . . . . 25 3.2.2.1.3 Multi-task learning approaches: . . . . . . . 29 3.2.2.1.4 Hierarchical approaches . . . . . . . . . . . 30 3.2.2.2 Unsupervised learning approaches for style classification 31 3.3 Summary of literature review for style classification . . . . . . . . . . 33 3.4 Paintings datasets: . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.4.1 Wikiart dataset . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.4.2 Paintings-91 dataset . . . . . . . . . . . . . . . . . . . . . . . 40 3.4.3 Pandora dataset . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.4.4 MultitaskPainting100k . . . . . . . . . . . . . . . . . . . . . . 44 3.5 Evaluation metrics in image classification . . . . . . . . . . . . . . . . 45 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4 Recognizing the Style of a Fine-art Painting with EfficientNet and Transfer Learning 48 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.3 Proposed methodologies . . . . . . . . . . . . . . . . . . . . . . . . . 50 4.3.1 Pre-trained EfficientNet models for style classification . . . . . 50 4.3.2 Custom pre-trained EfficientNet models for style classification 51 4.3.3 Deep retraining of the custom pre-trained EfficientNet models for style classification . . . . . . . . . . . . . . . . . . . . . . . 52 4.4 Experimental validation . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.4.1 Exprimental dataset: Painting-91 . . . . . . . . . . . . . . . . 52 4.4.2 Data pre-processing . . . . . . . . . . . . . . . . . . . . . . . 53 4.4.3 EfficientNet architecture . . . . . . . . . . . . . . . . . . . . . 54 4.4.4 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . 58 4.5 Results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.5.1 Style classification accuracy for all experiments . . . . . . . . 59 4.5.2 Classification results for style classification . . . . . . . . . . . 61 4.5.3 Confusion matrices . . . . . . . . . . . . . . . . . . . . . . . . 62 4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 5 The Effect of Optimizers on CNN Architectures for Art Style Classification 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 5.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 5.3 Proposed methodology . . . . . . . . . . . . . . . . . . . . . . . . . . 67 5.4 Exprimental validation . . . . . . . . . . . . . . . . . . . . . . . . . . 68 5.4.1 Exprimental datasets . . . . . . . . . . . . . . . . . . . . . . . 68 5.4.1.1 Dataset 1: Panting-91 . . . . . . . . . . . . . . . . . 68 5.4.1.2 Dataset 2: Pandora18k . . . . . . . . . . . . . . . . . 69 5.4.2 Data pre-processing: . . . . . . . . . . . . . . . . . . . . . . . 70 5.4.3 Convolutional neural network architectures . . . . . . . . . . . 72 5.4.3.1 InceptionV3 . . . . . . . . . . . . . . . . . . . . . . . 72 5.4.3.2 Xception . . . . . . . . . . . . . . . . . . . . . . . . 73 5.4.3.3 ResNet50 . . . . . . . . . . . . . . . . . . . . . . . . 75 5.4.3.4 InceptionResNetV2 . . . . . . . . . . . . . . . . . . . 76 5.4.3.5 DenseNet121 . . . . . . . . . . . . . . . . . . . . . . 77 5.4.3.6 EfficientNet B3 . . . . . . . . . . . . . . . . . . . . . 78 5.4.4 Optimizers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5.4.4.1 SGD optimizer . . . . . . . . . . . . . . . . . . . . . 80 5.4.4.2 RMSprop optimizer . . . . . . . . . . . . . . . . . . 80 5.4.4.3 Adam optimizer . . . . . . . . . . . . . . . . . . . . 81 5.4.4.4 Advantages and limitations of SGD, RMSprop, and Adam optimizers . . . . . . . . . . . . . . . . . . . . 81 5.4.5 Exprimental setup . . . . . . . . . . . . . . . . . . . . . . . . 82 5.5 Results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.5.1 Results of style classification on the Painting-91 dataset . . . . 83 5.5.2 Results of style classification on the Pandora 18k dataset . . . 84 5.5.3 Analysis of classification results . . . . . . . . . . . . . . . . . 86 5.5.4 Confusion Matrices . . . . . . . . . . . . . . . . . . . . . . . . 90 5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6 Conclusion and Future Work 94 6.1 Summary and findings of the thesis . . . . . . . . . . . . . . . . . . . 95 6.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 References 98 |
| En ligne : | http://thesis.univ-biskra.dz/id/eprint/6322 |
Disponibilité (1)
| Cote | Support | Localisation | Statut |
|---|---|---|---|
| TINF/191 | Théses de doctorat | bibliothèque sciences exactes | Consultable |
