Self-reflection
I am an individual enthusiastic about areas that span from AI and high-tech, through economics and business, to art and sports.
Motto
Lean into each moment expecting magic.
Machine Learning and Feature Engineering for Art Classification
Final Project
Artificial Intelligence
For the final project in the Machine Learning & Artificial Intelligence class, I led a team of 4 in tackling the problem of art classification and feature engineering for art.
Experts in the realm of art develop their knowledge for years and even so, many paintings by unknown artists have never been classified to a painter.
These two constitute the motivation for the project:
- it takes years to develop art-related expertise allowing to classify artwork
- machine learning can contribute to some unresolved discussions
In the project, we explored many image feature engineering techniques, applied various Machine Learning methods, and wrote scripts to perform art classification.
Slides from our final presentation are posted below, whereas to read the final paper itself, feel free to scroll down to the bottem where it is posted.
Project Outline
The following were the 3 objectives for the project.
First of all, getting creative about feature engineering with respect to art; it is particularly intriguing since one classifies to an artist and her style, rather than an object
Second, showing and contrasting how machine learning techniques can be applied to perform this classification
Finally, writing an easy-to-use script that allows anyone to perform such classification with less than 5 lines of code
The dataset used consists of over 8000 scans of paintings by 50, arguably, most influential artists of all time and can be found here.
My contributions:
- coded a robust and extensive script in Jupyter that is simple to use and scale
- wrote documentation for the script
- applied shallow Machine Learning techniques to our task
- optimized classification and obtained the results
- presented our findings at a presentation session
Feature Engineering Techniques
In this slide, there are Computer Vision techniques we used as well as their examples on the William Turner painting.
Overall, these methods can be divided into 3 main groups: gradient-based, color, and grayscale features.
Details about all the methods applied can be found within the paper itself.
In color features, the idea was that the same painter would usually use a similar set of colors, their tone or brightness.
With gradient-based features, the aim is to check whether a particular painter uses a lot of edges and how pronounced they are.
Finally, greyscale features examine pixel intensities and measure their consistency patterns across an image.
Shallow Machine Learning Results
With all these features, we looked for the combinations of features that yields the highest classification accuracy.
Learning models used are presented on the slide. We used sklearn to implement them.
By default, the algorithms applied were evaluated by test accuracies in 5-fold cross-validation.
We strarted out with 3 artists and then having trained algorithms on all 255 possible combinations of features, we scaled the dataset up from 3 artists to 50 artists.
90% accuracy was the highest achieved for Paul Cezanne, Rembrandt, and van Gogh. The subset of features that yielded this accuracy was then applied to the rest of the dataset.
The graph on the right side of the slide shows that the accuracy starts at around 93% for 2 artists and drops to about 22% for all 50 artists.
In terms of methods, random forest initially performs best, but for more than 5 artists, it is Softmax Regressiion, KNN and SVM that have the highest accuracy.
Convolutional Neural Network Results & Comparison
In the other part of the project, we applied a convolutional neural network to compare the capabilities of shallow Machine Learning and Deep Learning in this task.
In terms of CNN's architecture, the one on the graph with 7 fully connected layers yielded us the highest accuracy.
The plot on the right-hand side of the slide shows a comparison between the best shallow ML algorithm and CNN. For a small number of artists, the performance is similar, but as there are more categories, CNN becomes superior.
For instance, for 2 artists, the accuracy is about the same but for 25 artists, CNN performs 13% better than shallow ML.
Then, for the whole dataset, CNN has 40% accuracy, which means it outperforms shallow ML by 18%.
Findings & Paper
The main takeaways are:
1. CNN performed at least as good or significantly better than shallow Machine Learning algorithms.
2. In shallow Machine Learning, the performance drops steadily with an increasing number of artists, while for the CNN, the drop in accuracy slows down with more artists.
3. Interestingly, the highest accuracy shallow Machine Learning yielded was only for a subset of our Computer Vision techniques.
Self-reflection:
Overall, the project was extremely interesting and I have found it inspiring to explore how Computer Vision and different layers of a CNN interpret elements of art. The fact that it was art rather than pictures of objects made it challenging to adapt and interpret some of these techniques. Another thing that I particularly enjoyed was writing the script! It is quite powerful in how the dataset can analyzed and scaled, too.
The documentation can be found below and is here mostly to show just how extensive the code is. If you were interested in accessing the Jupyter notebook, I will be more than happy to share it!