Google engineer Working on problems with SPEED AND SCALE is a challenge. Engineers keep raising the bar on the tools and infrastructure. Google Culture: • Collaboration and co-development • Sharing between products and teams (tools, libraries, services) • Engineers have autonomy. • Agile/Scrum, daily stand-up meetings
versioning - One source of truth • Extensive code sharing and reuse • Collaboration across teams • Simplified dependency management • Large scale refactoring • Flexible team boundaries & code ownership • Code visibility
its own version control system called: Piper Sync workspace Write code Code Review Commit Read/Write Access per folder Code Quality & Syntax Check (by humans and by tooling) Create personal copy Auto Rollback if needed MANDATORY A single code tree, with fast access to the code through tooling. All types of code languages. Everyone, works in Trunk. - Branches are for releases.
build systems? Well code has a lot of dependencies and you don’t want to compile and link these all manually. The steps of a general build system: 1. Loading 2. Analysis 3. Execution by build system
Google has its own continuous build & test system. Remember, at Google we develop everything at HEAD in the repo. Endless CPU, Cross User Caching, because of Cloud Computing.
• Docker is a popular software container platform. • Containers are a way to package software in a format that can run isolated on a shared operating system.
clusters of servers • Networking and connectivity • Security and Access control • Logging, Monitoring, and Debugging • Health checks and uptime preservation • ...
Borg 2004 2016 • It’s software that manages all production machines at Google and runs jobs (binaries) that engineers give it on them. • Borg ran pretty much everything inside the company, including Google Search, Gmail, Google Maps, Google Docs... • These binaries are run in a container environment. • When tasks die, they are automatically started up again, and they may run on a different machine.
specialized job function that focuses on the reliability and maintainability of large systems. • SRE is also a mindset, and a set of engineering approaches to running better production systems • Google has SRE teams of site reliability engineers responsible for a service globally available. https://landing.google.com/sre/book.html
Internal Google • Internal Build System • Borg Container Orchestration • Machine Learning • Go Lang • Google Chrome Open Source • Bazel • Kubernetes • Tensorflow • Go Lang • Chromium
internal machine learning projects, and now it’s available to you! Google made it open source. More than 480 contributions 10,000 commits in a year 53k star rating Tutorials to get started at https://www.tensorflow.org
system, if you work with teams. Google’s build system, is now available open source. Google has been working on this for more than 10 years. Now you can benefit from this. https://bazel.build/ • Scalable: Bazel helps you scale your organization, codebase and Continuous Integration system. It handles codebases of any size, in multiple repositories or a huge monorepo. • Platform independent: Works on Cloud or On Premise. • Any language: Build and test Java, C++, Android, iOS, Go and a wide variety of other language platforms (via extensions).
whole data center as a single enormous computing resource. • Multiple container engines (Docker, rkt, Windows) • Cloud and bare-metal environments • Container Engine = Managed Kubernetes in Google Cloud Kubernetes https://kubernetes.io
unique contributors Top 0.001% of all GitHub Projects 4000+ External Projects Based on Kubernetes Companies Contributing Supported by a broad ecosystem of partners, offering you cloud provider flexibility:
monitoring services • Secure and monitor traffic for microservices and legacy services without requiring any changes to application code • An open platform with key contributions from Google, IBM, Lyft and others • Allows developers to authenticate and secure the communications between different applications using a TLS connection • Multi-environment and multi-platform, but Kubernetes first Istio https://istio.io
Google searches annually • 65 billion downloads of apps from its Google Play store. • More than 1 billion people are using the Chrome browser on mobile devices every month. • 200 million people per month are using its online photo service, Google Photos.
Script kiddies Insider Risk Hacktivist groups Malicious users Criminal organizations Nation-state actors How are they attacking? DDoS Spear-phishing Malware XSS Man-in-the-middle User error Social 0-days What do they want? $$$$$ Intellectual property Espionage Vandalism Public perception Notoriety
software on large scale. Conclusion Your company can make use of the same infrastructure like Google does. Scalable, Secure and Open. The learnings are shared through whitepapers and contributed back through open source.