Can algorithms measure creativity?

Mitali Barnejee, Professor of Strategy - April 10th, 2017
Can algorithms measure creativity? Mitali Barnejee HEC Paris ©Fotolia-chetverikov

Creative breakthroughs in art, music, and science profoundly impact society, and yet our understanding of creativity and how it is valued remains limited. To better understand creativity and how creative output is valued in society, a researcher is using a computational model, based on big-data approaches, to evaluate the creative merit of artworks.

Mitali Barnejee - HEC Paris

Mitali Banerjee joined HEC Paris in 2016 as assistant professor of strategy and business policy. She studied economics and mathematics at the University of Rochester in the US, (...)

See CV

Creativity is a crucial aspect of human culture, yet it is hard to define and harder yet to measure. “The essence behind the term is being able to come up with new ways of seeing and doing,” HEC researcher Mitali Banerjee explains. Creativity not only defines the work of artistic pioneers or visionary scientists, but its different forms also animate the activities of businesses in industries ranging from technology to entertainment. “In some instances, such as the iPhone, creativity can involve recombining existing technologies in a new design; in other instances, creativity can involve designing new organizational processes,” says Mitali Banerjee. Apart from the difficulties of measuring creativity, little is understood about how creativity is valued in our society. Does everything creative become successful? “It is possible that some very creative works fail to rise to attention because of biases against such work or because of market structures,” the researcher points outs out. 

Measuring creativity and creative success on a large scale

Given that creativity is a key performance outcome in many fields, how does one go about evaluating the creative merit of an individual work? Past measures of creativity have often relied on expert evaluations, but, according to Mitali Banerjee, such evaluations tend to give an incomplete picture of the relationship between creative merit and success: “Asking experts to evaluate the creativity of even 100 artists entails a significant drain on their cognitive resources; having multiple experts might address the problem, but that involves introducing the experts' own subjective aesthetics and hence more noise in the measures.” The need for a big-data approach is about more than overcoming the simple biases and limitations of experts: “Prior research suggests that creative work becomes famous or canonized. However, because of data limitations, such work has focused on a small set of creators, who often happen to be already the most celebrated ones,” says Mitali Banerjee. “With a computational measure of creativity, we can start examining if such results hold for a larger sample of innovators. Research in psychology and sociology suggests that even though we like the notion of creativity in principle, we are actually biased against creativity in practice. More often than not, new ideas and technologies encounter resistance rather than a red carpet.” Professor Banerjee decided to work with computer scientists to measure the creativity of paintings on a large scale and thereby gain a better understanding of how creativity is understood and valued in our society. 

Guillemet
There is a strong correlation between the expert and machine evaluations of creativity--so in that respect the computational measure reflects how we conceive of creativity”

A computational model for artworks

This novel measure of creativity relies on recent advances in statistics and computer science, which enable the exploitation of what is known as big data. The computation model was applied to visual features of more than 4,000 images of 55 early 20th-century artists. They include Natalia Goncharova, Pablo Picasso, Paul Klee and Robert Delaunay, who are associated with a range of abstract art movements including Cubism, Futurism, Orphism, German Expressionism and Rayism. The researcher chose this period because it represents a paradigm shift in the history of modern art: “Until the 1900s, representational art had dominated much of the Western fine art world,” she comments. “Around then, we witness the emergence of several innovations in artistic style, broadly grouped as abstract art, which represent a distinct and radical break from the representational paradigm.” Mitali Banerjee worked with several different algorithms with different computer scientists to extract features (some recognizable to human beings, like “ruggedness,” others recognizable only to computers without any such human interpretation), which in turn provided numerical representations known as feature vectors. (For the more enlightened, these approaches include unsupervised learning methods such as k-means clustering and artificial neural networks which are used to extract features vectors to represent a painting. Other methods such as k-nearest neighbors are used to classify the style and genre of the paintings). In parallel, Mitali Banerjee asked art historians to rate the artists' creativity, broken down along several dimensions: originality, stylistic diversity, abstraction, uniqueness and innovativeness. She then combined these scores into a single value (factor scores) for each artist, and compared the human measure and the machine measure of creativity.  

The complementarity of human and machine evaluations

Mitali Banerjee observed a strong correlation between the expert and machine evaluations of creativity. This correlation provides some validation of the computational measure, “so in that respect the computational measure reflects how we conceive of creativity,” she comments. The correlation between expert and machine evaluations was strongest when the machine was evaluating the most creative works of an artist (according to the machine). This suggests that human experts are thinking of the most creative work of the artist, even though Professor Banerjee had asked them to evaluate the average creativity of each artist between 1910-25. Given the strong correlation between both types of measures, she plans to apply this approach to visual art on a larger scale, in particular to understand how creativity and fame evolve together. It may sound shocking to have a machine-based measure of something so ineffable as creativity, but Professor Banerjee likes to see it as a tool to complement human skills: “We humans are great at recognizing and evaluating complex products. We can learn from a relatively small set of examples whereas a machine needs more input, but machines can reveal what we have ignored because of our inherent biases. They can illuminate new ways of seeing. In order for these tools to give us meaningful insights it is important to take an interdisciplinary approach which integrates theories and tools from art history, computer science, psychology and sociology.”

Concerning the impact of her work, Mitali Banerjee speculates, “given how much our society already values creativity, and the demonstrable impact that creative ideas have on the economy and culture, insight into how creativity functions can impact how our institutions incentivize, recognize and support creativity.”

Based on a written exchange with Mitali Banerjee and her research, including working paper “Understanding and valuing creativity using expert and computational measures of visual artists' output”.

Practical Applications
Practical Applications

This research has immediate applications in understanding how products and producers are valued in the multi-billion-dollar art market. This approach can also be used to measure the creativity of other complex objects such as 3-D printed objects, video games and other consumer products. Furthermore, an art market is a market of ideas. Understanding how such a market rewards or ignores certain kinds of creativity can shed light on how other similar markets, such as scientific labor markets, can be refined to better recognize creative talent. Also, artists are in a sense the ultimate entrepreneurs who have to navigate considerable uncertainty while endeavoring to create something new. Understanding how some artists are more successful in achieving rewards for their creative output can tell us something about how entrepreneurs choose, implement and manage their objectives.

Methodology
Methodology

For the machine evaluation of creativity, Mitali Banerjee used different machine vision algorithms to extract features from 4,175 images of 55 artists, creating a feature vector (a numerical representation of a painting), from which she computed a computational creativity score and then took the mean, mode, median, variance and extreme values. For the human evaluation, she asked four art historians to rate artists on a five-point scale along six dimensions, then combined these ratings into a factor score, and compared both types of evaluations.