This year, however, GMV’s Big Data and artificial Intelligence Department (BDA) conducted a little experiment: participate in the contest with an AI-generated Christmas card, obviously without the panel’s knowledge. Would we be able to fool the panel into thinking that the card had been created by a child? Could it win the contest? What lessons could we learn from the whole thing?

Creating and training GrinchGAN

GrinchGAN is a system developed by GMV to meet this challenge. It is based on Generative Adversarial Networks (GANs); the main idea is to get two neural networks to compete against each other. A generating network called G produces noise-based images that are then sent to another discriminating network called D, which then tries to determine whether the image received is genuine or G-generated.

By repeating this process, the two networks learn from each other, thus producing a generator capable of creating totally new, uncannily-realistic images without copying or blending original training images. The discriminator’s ability to tell the real images from the generated images is also honed, forcing the generator in turn to up its game. This idea is outlined in Figure 1

 

To conduct the experiment, training images were first downloaded from Google Images. A sample of some 800 Christmas trees proved to be sufficient (some examples are shown in the upper left-hand section of Figure 3). Once the GAN had been trained, we were able to obtain sufficiently tree-like images with reasonable resolution. We then made a manual selection of the most promising images (upper right-hand section of Figure 3) and short-listed the ones that seemed capable of getting the best results. One of the competition constraints was that the Christmas card had to include GMV’s logo, so we decided to draw this in by hand, using the Paint program.

The final step involved the Neural Style Transfer technique. It was this final touch that gave the result a high-quality, drawing-like look. This technique extracts the style and context of two images, reference and target, transferring the style from the reference image to the target while maintaining its context. Figure 2 outlines how this technique works. In our case, we chose an image from a previous year's Christmas card contest as the reference (bottom center of Figure 3) and our tree as target. This enabled us to separate the reference style (colors, texture) from the context (hands) and replace the style in the target, maintaining the context (tree, logo). The result was the Christmas card shown at the bottom center of Figure 3)

 

And the winner is…

The experiment result was intriguing; none of the panelists suspected the image was fake, proving that our generated image could compete on equal terms with children's drawings and that they are indistinguishable to the human eye.

 

Some issues still need to be resolved, however.

• First, the resolution, though not bad, is insufficient to fill an A4 sized sheet without forfeiting quality.
• Second, one of the comments from the panel was that they would have liked to see the original drawing. That brings with it a series of headaches, as we are unable to print the image with the textures of an actual drawing, such as a watercolor, which limits credibility.

The good news is that, although the panel liked the drawing, they sensed something “different” about it, which not even the panelists were able to put their finger on. Either we didn't get it quite right, or a child’s drawing is still able to evoke emotions better than an AI-generated one. But despite those limitations, the image came second in its age category, suggesting that, in the near future, the Grinch might be able to wrench the Christmas card prize from the hands of GMV employees’ children with nothing more than a computer and some patience.

Although we would have liked to win the competition, personal and professional ethics forced us to set certain limits for ourselves. Although it is true that the panel was not informed of the AI input, the contest manager was told, to keep her in the know. Far from forbidding the experiment, she encouraged it and congratulated us. However, she was on hand at all times in case the card needed to be disqualified (if it had won in its category).

What might initially seem to be an innocent experiment should make us reflect on the short- and medium-term future of artificial intelligence, which is now capable of producing increasingly realistic Deep Fakes, of swaying the electorate in political campaigns, and even publishing AI-generated fake news. Does this mean AI needs to be regulated? Can just any model be published for the general public? A fascinating discussion on this issue in the context of language models (GPT-2) is available here.

We will continue to innovate.

 

Author: Antón Makarov

[1] https://medium.com/analytics-vidhya/intuitive-explanation-of-gan-93de2dbcce85

[2] https://becominghuman.ai/creating-intricate-art-with-neural-style-transfer-e5fee5f89481