Design in the Age of Synthetic Realities

Presentation at UX Australia 2019

AI-generated and mixed realities are blurring the boundaries of “truth” and challenging how we value it. Synthetic Realities have reached new heights of sophistication, sparking controversy, and also fascination, about its creative possibilities.

Customers will increasingly expect brands to meet them halfway to supplement the realities they desire. Next, they will expect reality to adapt to them in real time without any conscious request. And the role of designers will be to set the stage on which these experiences happen, often curating AI systems rather than designing themselves.

UX practitioners face the double task of designing the interfaces to these new products, tools and services as well as seeing design for digital disrupted as radically as the invention of Photoshop and desktop publishing did three decades ago. If anyone can summon up an entirely generated image or an interface just by speaking its description, what is the future role of the designer?

The mixture of fear and fascination of the power of synthetic media is understandable but wrongly frames its future trajectory. This talk takes a view on synthetic realities not as something brand new, but the latest iteration of a long history of re-defining truth and reality through media manipulation.

Using visually compelling examples from the latest research and work in synthetic and generated media and charting their convergence, we'll see just how quickly they will be absorbed by culture, radically shake-up the fields of design, advertising and marketing in a very short time-frame, and transform the role of future designers and creatives. It raises important questions of design, ethics, and craft that the UX and broader design community will have to find answers to pretty damn quickly.

Transcript

1. Design in the age of synthetic realities Design and Innovation from Accenture Interactive Andy Polaine Group Design Director – Client Evolution @apolaine | @fjord Slides: pln.me/uxaus-synthetic UX Australia, Sydney, 2019
2. 2 Andy Polaine – Design in the age of synthetic realities Synthetic realities trends.fjordnet.com
3. 3 Andy Polaine – Design in the age of synthetic realities Image source: Adrian on Unsplash Jordan Peele’s Obama PSA
4. 4 Andy Polaine – Design in the age of synthetic realities Deep Fakes
5. 5 Andy Polaine – Design in the age of synthetic realities Bundesweite Umfrage Jetzt Umfrage beantworten und exklusives Dankeschön-Paket sichern MDM ÖFFNEN The Post's View Deepfakes are dangerous — and they target a huge weakness Opinion • June 16 DEEPFAKES ARE dangerous — not only the synthetic videos themselves but also the way they promise to blur reality in a country where facts are already up for debate. "Deepfakes" have changed the idea that seeing is believing - and could have a huge impact on how future political campaigns unfold. (Elyse Samuels/The Washington Post) By Editorial Board " Inside the race to beat 'deepfakes' in 2020 ' Sections Sign In ( ) Summer Sale Democracy Dies in Darkness There's a terrifying trend on the internet that could be used to ruin your reputation, and no one knows how to stop it Lisa Eadicicco, Business Insider 10.07.2019, 17:01 Samantha Lee/Business Insider The rise of deepfakes, or videos created using AI that can make it look like someone said or did something they have never done, has FACEBOOK ! LINKEDIN " TWITTER # EMAIL $ PRINT % El Paso Shooting This PSA About Fake News From Barack Obama Is Not What It Appears Oscar-winning filmmaker Jordan Peele has a warning for viewers about trusting material they encounter online. David Mack BuzzFeed News Reporter Posted on April 17, 2018, at 11:26 a.m. ET Sitting before the Stars and Stripes, another flag pinned to his lapel, former president Barack Obama appears to be delivering an important message about fake news — but something seems slightly...off. video-player.buzzfeed.com ADVERTISEMENT ADVERTISEM Tweet Share Copy Technology Home Video World UK Business Tech Science Stories Entertainment & Arts Health World News TV More In fighting deep fakes, mice may be great listeners By Dave Lee North America technology reporter Top Stories Hong Kong airport cancels flights over protests 2 hours ago News Sport Reel Worklife Travel Future Search Search the BBC Sign in More
6. 6 Andy Polaine – Design in the age of synthetic realities “As soon as it works, no one calls it AI anymore.” - John McCarthy, 1955
7. Fake Olds 7 Andy Polaine – Design in the age of synthetic realities
8. Cover ©1982 National Geographic 8 Andy Polaine – Design in the age of synthetic realities Image ©Gordan Gahan National Geographic, 1982
9. 9 Andy Polaine – Design in the age of synthetic realities Photoshop 1.0, 1990
10. 10 Andy Polaine – Design in the age of synthetic realities Image: Wikimedia Stalin removes Nikolai Yezhov, 1940
11. 11 Andy Polaine – Design in the age of synthetic realities Storytelling is the most potent technology of all
12. 12 Andy Polaine – Design in the age of synthetic realities
13. 13 Andy Polaine – Design in the age of synthetic realities A journey through synthetic media
14. 14 Andy Polaine – Design in the age of synthetic realities Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan Hi-fidelity synthetic imagery Dogball
15. 15 Andy Polaine – Design in the age of synthetic realities Synthetic People
16. 16 Andy Polaine – Design in the age of synthetic realities Four years of GANs Ian Goodfellow: https://twitter.com/goodfellow_ian/status/1084973596236144640
17. 17 Andy Polaine – Design in the age of synthetic realities StyleGAN: https://github.com/NVlabs/stylegan
18. Dove #realmums 18 Andy Polaine – Design in the age of synthetic realities
19. 19 Andy Polaine – Design in the age of synthetic realities Vid2Vid: https://tcwang0509.github.io/vid2vid/
20. 20 Andy Polaine – Design in the age of synthetic realities Vid2Vid: https://tcwang0509.github.io/vid2vid/
21. 21 Andy Polaine – Design in the age of synthetic realities Jurassic Park, 1993 Expression & Pose Editing Deep Video Portraits: https://arxiv.org/abs/1805.11714
22. 22 Andy Polaine – Design in the age of synthetic realities Few/Single Shot Living Portraits Few-Shot Adversarial Learning of Realistic Neural Talking Head Models: https://arxiv.org/abs/1905.08233v1
23. 23 Andy Polaine – Design in the age of synthetic realities Text to image synthesis AttnGAN: https://arxiv.org/abs/1711.10485
24. 24 Andy Polaine – Design in the age of synthetic realities Synthetic artists
25. 25 Andy Polaine – Design in the age of synthetic realities Jurassic Park, 1993 ©1993 Universal Pictures | Amblin Entertainment
26. 26 Andy Polaine – Design in the age of synthetic realities Rogue One, 2016 Rogue One Deep Fake Derpfakes: https://www.youtube.com/watch?v=RiBqZoVe92U
27. 27 Andy Polaine – Design in the age of synthetic realities Data Grid https://datagrid.co.jp
28. 28 Andy Polaine – Design in the age of synthetic realities Willing suspension of disbelief
29. 29 Andy Polaine – Design in the age of synthetic realities lilmiquela Venice Beach View Profile View More on Instagram 36,346 likes lilmiquela *Nicki voice* Let's go to the beach, beach!!! That is all. Have a blessed day bbs " # $ % ‗ view all 296 comments Add a comment... Lil Miquela at the beach. Do synthetic people need sun protection? Given the rise of virtual Instagram inhuencer Lil Miquela (who currently has 1.6m followers) and her forays into music, advertising and interviewing, it seems likely that the Vrst full GAN-generated artist is around the corner. Lil Miquela https://www.instagram.com/lilmiquela/
30. 30 Andy Polaine – Design in the age of synthetic realities Hatsune Miku https://www.instagram.com/hatsune.miku_/
31. 31 Andy Polaine – Design in the age of synthetic realities Dalí Lives Dalí Museum, Florida: https://www.youtube.com/watch?v=BIDaxl4xqJ4
32. 32 Andy Polaine – Design in the age of synthetic realities AI news anchor https://twitter.com/XHNews/status/1098221286491271168
33. “There is nothing so amazing that we can’t get used to it.” 33 Andy Polaine – Design in the age of synthetic realities Ian McEwan, Machines Like Me
34. 34 Andy Polaine – Design in the age of synthetic realities Synthetic media and the future of design
35. Synthesise people, faces, objects, animals Capture and re-generate poses and movement Modify facial expressions Synthesise places and landscapes Clone voices Generate completely new imagery Generate writing Effortlessly erase anything from images/video Generate GUIs and UX flows Link them all together in workflows 35 Andy Polaine – Design in the age of synthetic realities We can now:
36. 36 Andy Polaine – Design in the age of synthetic realities AI-assisted design
37. 37 Andy Polaine – Design in the age of synthetic realities UIzard https://uizard.io
38. 38 Andy Polaine – Design in the age of synthetic realities Adobe’s Content-Aware Fill
39. 39 Andy Polaine – Design in the age of synthetic realities Designer as AI curator Image: “squishybrain” on Ganbreeder
40. 40 Andy Polaine – Design in the age of synthetic realities Ganbreeder http://ganbreeder.app
41. 41 Andy Polaine – Design in the age of synthetic realities Runway ML http://runwayml.com
42. 42 Andy Polaine – Design in the age of synthetic realities Runway ML
43. 43 Andy Polaine – Design in the age of synthetic realities Designers’ AI assistants will become valuable IP Technology Home Video World Asia UK Business Tech Science Stories Entertainment & Arts Health World News T 1 August 2019 AI system 'should be recognised as inventor' By Leo Kelion Technology desk editor An artificial intelligence system should be recognised as the inventor of two ideas in patents filed on its behalf, a team of academics says. The AI has designed interlocking food containers that are easy for robots to grasp Top Stories Amazon fires an 'interna crisis', says Macron The French president's call fo be discussed at the G7 summ Brazil’s president. 2 hours ago Cathay Pacific staff spea climate of fear 22 August 2019 Parents fail to free sons daughter's murder 22 August 2019 Features Why India wants to brea decades-old nuclear ple Share GETTY IMAGES News Sport Reel Worklife Travel Future Search Search the BBC Sign in More
44. 44 Andy Polaine – Design in the age of synthetic realities Synthetic Twins
45. 30 July 2019 Chinese vlogger who used filter to look younger caught in live-stream glitch By Dhruti Shah & Kerry Allen BBC News & BBC Monitoring Your Highness Qiao Biluo's real appearance (left) and one of her filtered faces (right) from an earlier livestream on Douyu Fans of a popular Chinese video blogger who called herself "Your Highness Latest Posts Anti-natalists: The peo want you to stop havin Who are “anti-natalists” and want the gradual extinction Twitch CEO apologises pornography on Ninja' Gundam: The 'nerdy' e that led to marriage TikTok videos spread c change awareness Subscribe to our BBC Trending Share DOUYU 45 Andy Polaine – Design in the age of synthetic realities Synthetic Twins AI Avatars
46. 46 Andy Polaine – Design in the age of synthetic realities Synthetic Twins Finstas and digital afterlives Image: eterni.me
47. 47 Andy Polaine – Design in the age of synthetic realities Synthetic Twins Privacy & surveillance avoidance Image: URME by Leonardo Selvaggio
48. 48 Andy Polaine – Design in the age of synthetic realities Ethical challenges
49. 49 Social media ex-removal?
50. 50 Andy Polaine – Design in the age of synthetic realities Creative Selection StyleGAN: https://github.com/NVlabs/stylegan
51. “The whole thing looks very natural and no one would likely even notice without prior knowledge of the feature.” Andrew O’Hara — AppleInsider 51 Image: Fernando Frazão/Agência Brasil Andy Polaine – Fjord – Towards a mindful service design – Service Design Days 2018 Images: Will Sigmon @WSig https://twitter.com/WSig/status/1146146914985009154 Corrected Uncorrected Apple’s FaceTime Attention Correction
52. 52 Andy Polaine – Design in the age of synthetic realities Emotion intervention Emotion inference Real time emotion inference using CycleGAN: https://www.youtube.com/watch?v=JaQQVrAv9ok
53. 53 Andy Polaine – Design in the age of synthetic realities Viewer advisory labels? Warning: emotion filters active
54. “You’ve got to rely on people’s ethics. You don’t cheat just because the technology is available. Machines aren’t ethical or unethical — people are.” 54 Andy Polaine – Design in the age of synthetic realities Brian Steffans, graphics editor at The Los Angeles Times, 1987.
55. Thank you. Andy Polaine @apolaine @fjord Newsletter: pln.me/nws Slides: pln.me/uxaus-synthetic UX Australia, Sydney, 2019