The Magical Side-Project - Dafna Mordechai, Liron Soffer, Lior Dagan Leib

Starting a side-project can be rather intimidating, and even more so in the deep learning domain. Do I have the right background? Shall I start it on my own or with a partner? Where to start? What is a reasonable goal for a project?
When Lior, Dafna and Liron started a side-project together, they faced all these questions, and some more.

In this talk they will share with us they’re journey- working together on a shared side project, and the answers they found along the way. This adventure led them to implement GANILLA, a magical Generative Adversarial Network that translates natural images into a specific illustrator style. If you are curious how neural networks, children’s book illustrations, excel sheets and friendship all blend together, this talk is for you!

Transcript

1. The Magical Side-Project How to Create a Successful Learning Adventure Liron Soffer Dafna Mordechai Lior Dagan Leib February 2022
2. • Liron Soffer is a data scientist at Sunbit with experience in fintech startups. • She holds a BSc from the Technion in Industrial Engineering with two specialisations, one in Information Systems and the other in Entrepreneurship. • Dafna Mordechai is a deep learning compiler engineer. • She has a vast experience in designing, implementing, and testing software systems on various architectures, from microcontrollers to high-performance computing systems. • Lior Dagan Leib is a software engineer with a decade of mobile and backend development experience and an ever growing passion for machine learning and everything data. • She holds a BSc Cum Laude in Geophysics and East Asia studies, as well as a B.Sc. in Computer Science.
3. Our Side Project: Implementing the Ganilla Model GANILLA When you provide a photograph to a trained GANILLA model, it will output an illustration of the input image, based on the style of a specific illustrator. A photograph Generated illustration
4. The Ganilla Article
5. Agenda Motivation Planing Implementation Tools, Tips & Tricks
6. Motivation
7. Why Should You Start A Technical Side-Project? • Work on anything that interests you with no constraints • Explore new technical domains and gain an “hands-on” experience • Deepen your technical knowledge in an existing field HAVE FUN WITH TECHNOLOGY :) What is A Side Project? A side-project can be anything that you are working on aside from your day to day job.
8. It Can Be (Almost) Anything! Any task that has a well defined outcome can be a good side-project • Building a product or designing a device from scratch • Implementing a theoretical paper • Learning a programming methodology by implementing it • Creating a research project with “real world” data • Implementing a small task in order to learn a new language • Building a small tool / a vscode extension / a Chrome Add-on / a mobile application • Writing an interactive tutorial • Contributing to open-source
9. Individually or in a Team?
10. Individually or in a Team? • Making your own decisions • Working on your own time • Fully owning your project
11. Individually or in a Team? • Setting more ambitious goals • Benefiting the collaborative learning: ◦ Peer teaching ◦ Pair programming ◦ Code reviewing / Debugging • Having an accountability system • Improving your teamwork skills in a friendly environment • Having fun!
12. • Working as a team, but dividing the tasks and schedule deadlines. • Working alone, but alongside a friend who is also working alone. Hybrid might be the answer! Individually or in a Team?
13. The Right Partners • Know each other’s technical background ◦ Ask! ◦ Different backgrounds can be an advantage • Align expectations: ◦ Make sure you have the same goals and interests for the project ◦ Make sure all sides want to invest the same amount of time and efforts into the project (or not, and that can also be OK…) • Working framework can be flexible, and meet the needs of the group (can be online, partially online or in person)
14. Where to Find Them? • Facebook groups • Side-project dedicated programs • Community courses • Volunteering projects • University (if you’re a student) • Hackathons • Meetups • Colleagues at work
15. Planing
16. What’s within your learning goals What sparks you with joy WHAT YOU ARE LOOKING FOR What scares you (just a bit)
17. What’s within your learning goals What sparks you with joy WHAT YOU ARE LOOKING FOR What scares you (just a bit)
18. What’s within your learning goals What sparks you with joy WHAT YOU ARE LOOKING FOR What scares you (just a bit)
19. What’s within your learning goals What sparks you with joy WHAT YOU ARE LOOKING FOR What scares you (just a bit)
20. Don’t Skip the Background Check… • Look for previous works in the field you are aiming to learn • Search for the latest cutting edge works • Tap into relevant professional groups and look for current trends • Follow notable professionals on social network and get inspired • Google “side project <your technology here>” • Look for open problems / published challenges / hackathons etc.
21. Artistic Style Transfer But this was not a GAN…
22. What Else Is Out There? Pix2Pix CycleGAN
23. 2 Minutes Papers Examples Everybody Dance Now Semantic Image Synthesis with Spatially-Adaptive Normalization
24. What is GANILLA? GANILLA is a type of GAN (Generative Adversarial Network) that solves the image-to-image translation problem, in the domain of children’s book illustrations. GANILLA excels in preserving both content from the original input photograph and the style of the desired illustrator.
25. Implementation
26. Working Process Understand the Theory Choose and Prepare Your Tools Get Your Hands Dirty
27. Working Process Understand the Theory Prepare Your Tools Get Your Hands Dirty
28. Understand the Theory: GANILLA Paper
29. Overcoming Obstacles: Scraping Data • Original data wasn’t available for us • We scraped the data form Google, but needed to fine-tune the search in order to get data that is more of landscape and less characters. • And yet there was a need to clean the data
30. Overcoming Obstacles: Cleaning the Data
31. Train, Test, Debug, Repeat! While (Not satisfied with model results) { > Training the model > Testing the trained model > Debugging }
32. Debugging
33. Using Overfitting as a Testing Tool
34. ״םוֹהְת יֵנְפּ-לַע , ֶשֹׁ חְו ,וּהֹ בָו וּהֹ ת הָתְיָה ,ץ ֶ ראָָהְו .ץ ֶ ראָָה תֵאְו ,םִיַמָשַּׁה תֵא ,םיִה ֱא א ָ רָבּ ,תיִשׁא ֵ רְבּ״
35. 4 Epochs Are Like…
36. 32 Epochs Are Like…
37. 36 Epochs Are Like…
38. 40 Epochs Are Like…
39. 50 Epochs Are Like…
40. Cherry Picking…
43. Tools, Tips & Tricks
44. Manage Your Project Like a Pro! • Stick to the basic requirements • Make Github your friend • Document thoughts and conclusions along the way • Take plenty of Screenshots (and then some more…) • Plan your next meeting
45. Aim High Meet Regularly Take Breaks Track Your Progress Give Technical and Emotional Support Celebrate Milestones Maintain Self Discipline
46. Publish Your Results
47. THANK YOU! Lior Dagan Leib Liron Soffer Dafna Mordechai