Serverless Measurement Limits

Exploring cold starts and warm starts in serverless compute platforms.

Expanded version of the 2017 dotScale talk, "Searching for the Server in Serverless"

Transcript

1. SERVERLESS MEASUREMENT LIMITS @smithclay New Relic Based on my dotScale 2017 talk: "Searching for the Server in Serverless"
2. λ HTTP Requests Client App App Architecture with FaaS Functions + API Gateway Is this going to (not) work for me? Managed Infrastructure λ λ λ
3. Crazy idea: Let's run SSH in a λ λ ssh process
4. FaaS: Typical Metrics Error Count Function Invocation Count Function Duration (ms)
5. Event Trigger 1. Invoke λ 2. Run 3. End Result Error Timeout or or
6. Why does my function invocation time vary so much? Extremely common question with FaaS...
7. Is your λ triggered by user interaction? Yes No Monitor & make it fast (> 100ms) Monitor & still make it fast-ish
8. Cold Start vs Warm Start Event Trigger Handler Warm Function Invocation Time Create Initialize Handler Cold
9. λ Running Commands const exec = require('child_process').exec; exports.handler = (trigger, cb) => { exec('whoami', (err, stdout) => { console.log(stdout); return cb(null); }); } [LOG TIME] sbxuser_1066
10. λ is a UNIX system?! I know this!
11. Interacting this way is slow.
12. Go as a SSH Server + Tunnel-er Copy and paste Go SSH server + tunnel code (no inbound connections allowed) Execute binary on trigger Max(λ timeout)
13. Aside: Old But Still Good Advice λ Cloud vendor code "Secret Sauce Library" Keep it portable Write short handlers
14. https://github.com/smithclay/faassh
15. Info from /proc 2x Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz cat /proc/cpuinfo 3857664 kB cat /proc/meminfo ixgbevf (EC2 10Gbps Network Driver) cat /proc/modules c4.large EC2 Compute-Optimized Instance (?)
16. c4.large instance λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λs in theory on a VM 10 Gbps λ = 128 MB Function λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ
17. Cold Start Discovery in Code var SO_SO_COLD = true; exports.handler = function(trigger, cb) { console.log('Cold? %s', SO_SO_COLD); SO_SO_COLD = false; return callback(cb) } https://github.com/smithclay/lambda-proc-info
18. How are functions frozen? https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt cgroup freezer!
19. Warming methodology Scheduled Event λ Only effective for non- concurrent execution! 4 minute interval
20. Data collection from event-driven logs λ console.log(data) Logs λ POST to Metrics DB (Insights) Log Filter Trigger
21. Cold Starts Visualized ~7 hrs ~8 hrs
22. λ Host Hopping 10.13 10.12 10.11 10.13 10.12 10.12 10.12 10. 11 # of AZs in us-west-2: 3
23. Serverless, in brief High availability (multi-AZ or region) Elastic VMs and Automagic Freezing Containers Really good scheduling algorithm
24. 4. Better utilization @ cloud scale... for the environment?
25. Measure Your Own FAAS Cost, Worst-case perf Ops/dev benefits Easier to measure Harder to measure tech people
26. FaaS Production Reality λ λ Dev Prod Orchestration (!?) Version/Deploy Monitoring Security Cold Start Mgmt The "learning cliff" Great tweet from @mfdii
27. FaaS Isn't a Silver Bullet "I've got a quick, computationally-intensive task that I need to perform occasionally in response to a well-defined event that isn't that sensitive to latency." —The Ideal FaaS Developer // TO DO: measure && share
28. Thanks. @smithclay New Relic