Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Guide to Load Test gRPC Application

Ken Wagatsuma
September 28, 2018

The Guide to Load Test gRPC Application

You can understand …
- How to perform load testing to gRPC application
- What is “github.com/bojang/ghz” and How to use it
- Master/Minion Patterns
- TICK Stack & InfluxDB basis for Reporting the load testing results

Ken Wagatsuma

September 28, 2018
Tweet

More Decks by Ken Wagatsuma

Other Decks in Programming

Transcript

  1. TOC Overview Specification for Load Testing Overall Load Testing tools

    (HTTP & gRPC) github.com/bojang/ghz Master/Minion Patterns (Multi-Threaded ghz wrapper) Reporting (InfluxDB & Grafana)
  2. Overview You can understand … - How to perform load

    testing to gRPC application - What is “bojang/ghz” and How to use it - Master/Minion Patterns - TICK Stack & InfluxDB basis for Reporting the load testing results
  3. Specification for Load Testing gRPC supported. 3 RPC will be

    called at the same time: - 1,200 RPS - 800 RPS - 150 RPS
  4. Load Testing Tools - gRPC supports are rare - bojand/ghz

    is the very nice tool I was looking for https://github.com/httperf/httperf - one of major tool for HTTP stress testing, no support for gRPC https://github.com/loadimpact/k6 - also one of major HTTP stress testing tool, no support for gRPC https://github.com/tsenart/vegeta - yet another HTTP load testing, no support for gRPC https://grpc.io/docs/guides/benchmarking.html - benchmark test at gRPC https://gist.github.com/voluntas/5c9e0f778e36c1e934e83611a94ffdfa#id 5 - Shigredo (時雨堂) Oribe, no support for gRPC https://github.com/bojand/ghz - gRPC-oriented stress testing tool
  5. simple usage, nice reporting can be configured… - num of

    concurrency (implemented via goroutines internally) - rate limit (= QPS / Query Per Second) - max duration (sec) - HTML/JSON/CSV/InfluxDB Line Protocol output
  6. Example Usage Template Action supported - implemented by text/template (Go

    standard API) - {{.RequestNumber}}, {{.MethodName}}, {{.ServiceName}}, .etc. Fixtures can be file - handy & reusable $ cat testdata/Heart.json { "episode_id": "{{.RequestNumber}}", "heart_key": "heart key", "user_id": 999, "count": 9999 } $ cat testdata/metadata.json {} $ ghz \ -proto protobuf-definitions/myapp/v1/mydefinition.proto \ -call myapp.v1.Mydefinition.Heart \ -D testdata/Heart.json -M testdata/metadata.json \ -o log/Heart.log # output log file path -n 10000 \ # number of requests -q 5000 \ # QPS -insecure \ # for non TLS connection -O influx-details \ # output type format (e.g. csv/json/pretty/html/influx-summary) 127.0.0.1:5555
  7. Master/Minion Patterns 時雨堂 シナリオ負荷試験ツール Oribe - Master has N minions

    - Scenario can be written in Lua - HTTP/1.1 - No support for reporting https://gist.github.com/voluntas/5c9e0f778e36c1e934e83611a94ffdfa#id5
  8. Master/Minion Patterns Master has responsibility for: - common options (host

    & port, log, .etc) - handles multiple minions - reporting Minion has responsibility for: - run `ghz` to a single RPC - delegate results to the master class Minion def run system "ghz #{options}" end def rpc raise NotImplementedError end end class HelloRPCMinion < Minion def rpc; "Hello" end end class PingRPCMinion < Minion def rpc; "Ping" end end class Master def load_test minions = [ HelloRPCMinion.new, PingRPCMinion.new ] threads = minions.map {|m| Thread.new { m.run } } threads.each(&:join) end end
  9. Sample Code Check it out -> “Run Multiple gRPC Load

    Testing using ghz” http://itiskj.hatenablog.com/entry/2018/09/14/122110
  10. Reporting - InfluxDB + Grafana Dashboard - InfluxDB … Time-series

    oriented - bojand/ghz supports InfluxDB Line Protocol - https://github.com/bojand/ghz/pull/26
  11. TICK Stack Telegraf (Golang) - similar to Fluentd agent -

    pluggable architecture (AWS Cloudwatch, Consul, ElasticSearch, Fluentd, HAProxy,, Nginx, ...) Kapacitor (Golang) - streaming data processing engine - similar to AWS Kinesis or Apache Kafka Chronograf (TypeScript, Golang) - monitoring + visualization https://www.influxdata.com/time-series-platform/
  12. InfluxDB Line Protocol - whitespace are separator of each components

    - support floats, integers, string or booleans as field values type - use HTTP API with curl or CLI to write to InfluxDB with Line Protocol Syntax) weather,location=us-midwest temperature=82 1465839830100400200 | --------------------------------- ------------------------- | | | | | | | | | +---------------------+-----------+-+-----------------+-+------------------+ |measurement|,tag_set| | field_set | |timestamp| +---------------------+-----------+-+-----------------+-+------------------+ Example) ghz_detail,proto="/testdata/greeter.proto",call="helloworld.Greeter.SayHello",host="0.0.0. 0:50051",n=1000,c=50,qps=0,z=0,timeout=20,dial_timeout=10,keepalive=0,data="{\"name\ ":\"{{.InputName}}\"}",metadata="{\"rn\":\"{{.RequestNumber}}\"}",hasError=false latency=5157328,error=,status=OK 681023506