What if Picasso was a Robot? Teaching art to the machine

Machine learning is revolutionizing the world as we know it: recommender systems that know exactly what we’re into, ear phones capable of translating entire conversations on the fly and even predictive power to help us guess what the future may look like. But what about art? Can we teach machines how to be creative and come up with new art styles? In this talk you’ll see what happens when we mix computational power and the greatest works of art the world has ever seen! And guess what? All that using Javascript. Hold tight, art class is about to start!

Transcript

1. What if Picasso was a robot? Teaching art to the machine @silveira_bells
2. X
3. X
4. X
5. X
6. Mini Isa
7. X
11. X
12. Teaching the computer how to make art ✨
13. “What do you mean, Isa? Wtf?”
14. Isa Silveira @silveira_bells Developer @Work&Co Carioca, math freak, slalom skater, peculiar sense of humor @life
15. Art 01
16. What is art?
17. 1. The quality, production, expression, or realm, 
 according to aesthetic principles, of what is beautiful, appealing, or of 
 more than ordinary significance. [Dictionary] 2. “Art is either plagiarism or revolution.” [Paul Gauguin] 3. “Art is harmony." [Georges Seurat] 4. “[…] And it is upon this capacity of man to receive another man’s expression of feeling and experience those feelings himself, that is what the activity of art is based.” [Leo Tolstoy]
18. Does it have to be beautiful or original?
19. Does it have to be complex or have monetary value?
20. Does it have to be made by an artist?
21. What if the artist has an idea and the art 
 is created by someone else?
22. One and Three Chairs - Joseph Kosuth 1965
24. Kosuth didn’t make the chair, take the photo, or wrote the definition
25. So where’s the art?
26. Machine Learning 02
27. What is machine learning?
28. Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data and use it learn for themselves. The process of learning begins with observations or data, such as examples, direct experience, or instruction, in order to look for patterns in data and make better decisions in the future based on the examples that we provide. The primary aim is to allow the computers learn automatically without human intervention or assistance and adjust actions accordingly.
29. It allows the machine to actually learn. With ML algorithms, it’s possible to parse data, learn from it and make mass predictions or classifications. Instead of creating an algorithm full of specific rules on how to execute a certain task, we train the machine with huge datasets so it can learn how to execute it on its own.
30. We can use machine learning to draw conclusions about large chunks data automatically.
31. Several kinds of algorithms to each kind of problem
32. Our mission
33. Generating new art images from existing art works
34. 1. A powerful algorithm; 2. A couple thousands of paintings; 3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
35. 1. A powerful algorithm; 2. A couple thousands of paintings; 3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
36. Deep Convolutional Generative Adversarial Networks
38. Capable of creating new images from existing ones
39. 2 convolutional neural networks that keep battling each other
40. “Eita™, what do you mean?"
41. Baby steps!
42. Each algorithm works based
 on a model
43. Types of models
44. On the discriminative model we want to formulate descriptions to our results based on the data and parameters we have, in other words, finding patterns to classify or predict future results. On the generative model, we want to generate new samples based on what we learned about the data previously fed to the algorithm on the training part. Generative model Discriminative model Types of models
45. In this algorithm we’ll use both
 of these models
46. E.g.: generating images of IDs
47. Neural Net 1 - The discriminator His job is to recognize if an image is a legitimate ID
48. Neural Net 2 - The generator His job is to create
 images of IDs Photoshop mad skillz
50. They suck at the beginning
51. Deep Conv Neural Network True! Seems legit Discriminator
53. Deep Conv Neural Network False! Super fake Discriminator
54. Round 2
55. Reversed Deep Conv Neural Network 500 numbers Generator
56. Over and over and over
60. https://github.com/alantian/ganshowcase
61. 1. A powerful algorithm; 2. A couple thousands of paintings; 3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
63. My first idea was to filter by Picasso
64. But there were only 1130 paintings
65. Plan B: filtering by genre
66. 24.832 images of landscapes
67. DIR_PATH=“../images” DATA_FILE=“dataset.npz” SIZE=64
68. $ ./datatool.py --task ./src --dir_path $DIR_PATH —npz_path $DATA_FILE --size $SIZE
69. 1. A powerful algorithm; 2. A couple thousands of paintings; 3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
71. 4GB GPU
73. 1. A powerful algorithm; 2. A couple thousands of paintings; 3. A mighty processor; 4. 6 buckets of patience; 5. 2 cups of persistence; 6. Love to taste ❤
74. Training
75. DATA_FILE_SIZE_64=“dataset_64.npz” DCGAN64_OUT="./dist"
76. $ ./chainer_dcgan.py \ --arch dcgan64 \ --image_size 64 \ --npz_path $DATA_FILE_SIZE_64 \ --out $DCGAN64_OUT
77. The whole training took 16 hours
79. Stopped to make some adjustments, repeat cycle
80. $ ./dcgan_chainer_to_keras.py \ --arch dcgan64 \ --chainer_model_path $DCGAN64_OUT/SmoothedGenerator.npz \ --keras_model_path $DCGAN64_OUT/Keras_SmoothedGenerator.h5 \ --tfjs_model_path $DCGAN64_OUT/tfjs_SmoothedGenerator
81. $ ./dcgan_chainer_to_keras.py \ --arch dcgan64 \ --chainer_model_path $DCGAN64_OUT/SmoothedGenerator.npz \ --keras_model_path $DCGAN64_OUT/Keras_SmoothedGenerator.h5 \ --tfjs_model_path $DCGAN64_OUT/tfjs_SmoothedGenerator
82. let all_model_info = { dcgan64: { description: 'DCGAN, 64x64 (16 MB)', model_url: “/dist/tfjs_SmoothedGenerator/model.json”, model_size: 64, model_latent_dim: 128, draw_multiplier: 4, animate_frame: 200, }, };
84. Conclusions
86. 84 64x64 samples were generated
87. High variance of quality in each generated art work
88. Some styles are a lot harder to reproduce than others
89. Creative Computers 03
90. We have to get used with a future
 where machines can also do creative work
93. https://youtu.be/LSHZ_b05W7o
94. There’s always the training phase, but also a certain “creative unpredictability”
96. Art is making meaning
97. What's art to you?
98. https://github.com/bella-silveira
99. @silveira_bells
100. ¡Muchas gracias! ❤