Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Performance Optimization 
on Google AppEngine

Performance Optimization 
on Google AppEngine

The presentation at GoConference 2018 Spring

Seiji Takahashi

April 15, 2018
Tweet

More Decks by Seiji Takahashi

Other Decks in Programming

Transcript

  1. Who Am I • Seiji Takahashi • @__timakin__ • Gunosy

    • Application Dev: LUCRA • Serverside Engineering (Go) • iOS Client Side (Swift) • Engieering • Language: Go (Contributed), Swift • Platform: GAE, AWS
  2. "CPVUUIJTUBML • We’ll not cover intro to AppEngine. • General

    guideline to speed up API performance on GAE.
  3. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  4. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  5. Warming Up Building an endpoint of warmup, You can prepare

    before serving data for user’s request. For example, you retrieve the data from datastore and store it on memcache to respond faster.
  6. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  7. Shortest Route 1. No custom domain If you send an

    internal request to a different service, you should not use custom domain to decrease name resolution cost.
  8. Shortest Route 2. Turn off FollowRedirects Internal API should not

    return other than 200, 400, and 500 series statues. And you should exclude redirects with setting FollowRedirects to false.
  9. Shortest Route 3. Single Project Networking between multi-projects will add

    latency, so if you purpose the best performance, merge services into one project.
  10. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  11. Task Queue A heavy job should be scheduled with Task

    Queue, not user’s request. And it’ll be effective on cron job like crawling too.
  12. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  13. Delayed Job delay package is also background job handling system.

    It depends on Task Queue. It simplify the asynchronous API of Task Queue, and it can receive arguments of struct.
  14. Delayed Job delay package is also background job handling system.

    It depends on Task Queue. It simplify the asynchronous API of Task Queue, and it can receive arguments of struct. default Task Queue runs tasks through HTTP request and can pass only a string query parameter.
  15. *Caution delay.Func will search an execution target with file name

    and task key, so if you change the file name, waiting tasks will drop because it can’t find the func. Additionally, it must be declared on init() or as a global variable. If it was declared inside of request handling, a receiver instance may fail because Func wasn’t initialized just after spinning-up.
  16. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  17. Batching Executing Get/Put/Delete for multi resources in the loop will

    generate RPC calls. If you can get keys for the records, reduce them with GetMulti/PutMulti/DeleteMulti. And Task Queue also supports AddMulti for bulk insertion.
  18. Caching Storing/Fetching data from Memcache is better for performance. The

    cache will be shared among instances. Some third-party packages are good for using Datastore and Memcache simultaneously. • mjibson/goon • mercari/datastore
  19. Caching Almost all of interfaces of these packages is same.

    But goon wins on the current reading benchmark.
  20. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  21. Asynchronous requests with goroutine AppEngine environment forces GOMAXPROCS=1. But within

    I/O wait time, Go will schedule to other goroutine. Like Datastore/Search API call can be artificially asynchronous and it performs excellent latency.
  22. Asynchronous requests with goroutine Basically GetMulti/PutMulti is effective. But when

    you cannot know the datastore.Key before fetching. For example, goroutine is better for query filtering.
  23. goroutine in goroutine is sensitive technique from the aspect of

    error handling, but good for performance.
  24. 4VNNBSZ • AppEngine is tunable with routing, background job, goroutine,

    batch, cache, etc… • Async func is really attractive if your primary goal is performance. However, you must take care of error handling and initialization.