Share
Embed
From Service to Platform: A Ranking System in Go
What started out as an experimental service to rank livestreams evolved to a platform powering all types of content recommendations. Go's stringent code philosophy paved the way to a modular pipeline-based system for scatter-gather workflows enabling anyone to add new ranking algorithms.
GopherCon Europe 2022, Berlin
https://www.youtube.com/watch?v=5jSyctW1rPg
GopherCon UK 2022, London
https://www.youtube.com/watch?v=TNyoKBLxfTM
Konrad Reiche
August 18, 2022
Transcript
176 slides
-
From Service To Platform A Ranking System in Go Konrad
Reiche r/streetwear r/aww r/dogecoin r/golang r/yoga -
Once upon a time… … or more like last year
Redis Cluster -
Once upon a time… … or more like last year
Redis Cluster -
Once upon a time… … or more like last year
Redis Cluster -
Redis Monitor 1655767305.672853 [0 10.8.144.28:61122] "get" "post:3hg9w" 1655767305.672862 [0 10.8.144.28:61122]
"get" "post:5cm6n9" 1655767305.672869 [0 10.8.144.28:61122] "get" "post:62s7bk" 1655767305.672876 [0 10.8.144.28:61122] "get" "post:2zqpf" 1655767305.672880 [0 10.8.144.28:61122] "get" "post:2fn65o" ... -
Redis Monitor 1655767305.672853 [0 10.8.144.28:61122] "get" "post:3hg9w" 1655767305.672862 [0 10.8.144.28:61122]
"get" "post:5cm6n9" 1655767305.672869 [0 10.8.144.28:61122] "get" "post:62s7bk" 1655767305.672876 [0 10.8.144.28:61122] "get" "post:2zqpf" 1655767305.672880 [0 10.8.144.28:61122] "get" "post:2fn65o" ... Timestamp Database, Network Address Command and Key -
Redis Monitor 1655767305.672853 [0 10.8.144.28:61122] "get" "post:3hg9w" 1655767305.672862 [0 10.8.144.28:61122]
"get" "post:5cm6n9" 1655767305.672869 [0 10.8.144.28:61122] "get" "post:62s7bk" 1655767305.672876 [0 10.8.144.28:61122] "get" "post:2zqpf" 1655767305.672880 [0 10.8.144.28:61122] "get" "post:2fn65o" ... -
Redis Monitor 1655767305.672853 [0 10.8.144.28:61122] "get" "deleted_posts" 1655767305.672862 [0 10.8.144.28:61122]
"get" "post:5cm6n9" 1655767305.672869 [0 10.8.144.28:61122] "get" "deleted_posts" 1655767305.672876 [0 10.8.144.28:61122] "get" "deleted_posts" 1655767305.672880 [0 10.8.144.28:61122] "get" "post:2fn65o" ... -
func main() { flag.Parse() b, err := os.ReadFile(path) if err
!= nil { log.Fatal(err) } countByKey := make(map[string]int) lines := strings.Split(string(b), "\n") for _, line := range lines { split := strings.Split(line, " ") if len(split) < 5 { continue } key := split[4] countByKey[key] += 1 } type keyCount struct { key string count int } counts := make([]keyCount, 0, len(countByKey)) for key, count := range countByKey { counts = append(counts, keyCount{key: key, count: count}) } sort.Slice(counts, func(i, j int) bool { return counts[i].count > counts[j].count }) for i := 0; i < topKeys; i++ { fmt.Println(keyCounts[i].count, keyCounts[i].key) } } -
func main() { flag.Parse() b, err := os.ReadFile(path) if err
!= nil { log.Fatal(err) } countByKey := make(map[string]int) lines := strings.Split(string(b), "\n") for _, line := range lines { split := strings.Split(line, " ") if len(split) < 5 { continue } key := split[4] countByKey[key] += 1 } type keyCount struct { key string count int } counts := make([]keyCount, 0, len(countByKey)) for key, count := range countByKey { counts = append(counts, keyCount{key: key, count: count}) } sort.Slice(counts, func(i, j int) bool { return counts[i].count > counts[j].count }) for i := 0; i < topKeys; i++ { fmt.Println(keyCounts[i].count, keyCounts[i].key) } } Parse flags, read file into memory -
func main() { flag.Parse() b, err := os.ReadFile(path) if err
!= nil { log.Fatal(err) } countByKey := make(map[string]int) lines := strings.Split(string(b), "\n") for _, line := range lines { split := strings.Split(line, " ") if len(split) < 5 { continue } key := split[4] countByKey[key] += 1 } type keyCount struct { key string count int } counts := make([]keyCount, 0, len(countByKey)) for key, count := range countByKey { counts = append(counts, keyCount{key: key, count: count}) } sort.Slice(counts, func(i, j int) bool { return counts[i].count > counts[j].count }) for i := 0; i < topKeys; i++ { fmt.Println(keyCounts[i].count, keyCounts[i].key) } } Parse flags, read file into memory Parse each line, split by column and count keys -
func main() { flag.Parse() b, err := os.ReadFile(path) if err
!= nil { log.Fatal(err) } countByKey := make(map[string]int) lines := strings.Split(string(b), "\n") for _, line := range lines { split := strings.Split(line, " ") if len(split) < 5 { continue } key := split[4] countByKey[key] += 1 } type keyCount struct { key string count int } counts := make([]keyCount, 0, len(countByKey)) for key, count := range countByKey { counts = append(counts, keyCount{key: key, count: count}) } sort.Slice(counts, func(i, j int) bool { return counts[i].count > counts[j].count }) for i := 0; i < topKeys; i++ { fmt.Println(keyCounts[i].count, keyCounts[i].key) } } Parse flags, read file into memory Parse each line, split by column and count keys Convert to slice of tuples, sort and print -
Redis: Finding Hot Keys $ ./find-hotkeys -file monitor.log -n 5
8252 "deleted_posts" 1907 "post:2tk95" 1756 "post:2xcv7" 772 "post:3nasz" 509 "post:2qjpg" -
$ cat monitor.log | awk '{print $4}' | sort |
uniq -c | sort -nr | head -n 5 -
UNIX Pipes Output log Print 4th column Count unique lines
Sort numerical Print first 5 lines The same can be achieved with existing programs and UNIX pipes. $ cat monitor.log | awk '{print $4}' | sort | uniq -c | sort -nr | head -n 5 -
UNIX Pipes Output log Print 4th column Count unique lines
Sort numerical Print first 5 lines The same can be achieved with existing programs and UNIX pipes. UNIX Toolbox Philosophy Write programs that: • Do one thing well • Compose • Easily communicate $ cat monitor.log | awk '{print $4}' | sort | uniq -c | sort -nr | head -n 5 -
22 Essential Complexity Accidental Complexity No Silver Bullet—Essence and Accident
in Software Engineering Brooks, Frederick P. (1986) -
23 Go is a language that helps us to reduce
accidental complexity -
What is a ranking (recommendation) system? A recommendation system helps
users to find content they find compelling. -
What is a ranking (recommendation) system? A recommendation system helps
users to find content they find compelling. 1. Candidate Generation Start from a potentially huge corpus and generate a much smaller subset of candidates. -
What is a ranking (recommendation) system? A recommendation system helps
users to find content they find compelling. 1. Candidate Generation Start from a potentially huge corpus and generate a much smaller subset of candidates. 2. Filtering Some candidates should be removed, for example content already watched or content the user marked as something they do not want to consume. -
What is a ranking (recommendation) system? A recommendation system helps
users to find content they find compelling. 1. Candidate Generation Start from a potentially huge corpus and generate a much smaller subset of candidates. 2. Filtering Some candidates should be removed, for example content already watched or content the user marked as something they do not want to consume. 3. Scoring Assign scores to sort the candidates. -
31 From Service to Platform: A Ranking System in Go
Example: Ranking Service -
32 From Service to Platform: A Ranking System in Go
Popular Posts Example: Ranking Service -
33 From Service to Platform: A Ranking System in Go
Popular Posts Fetch posts Example: Ranking Service -
34 From Service to Platform: A Ranking System in Go
Popular Posts Fetch posts Filter posts Example: Ranking Service -
35 From Service to Platform: A Ranking System in Go
Popular Posts Fetch posts Filter posts User Post Views Example: Ranking Service -
36 From Service to Platform: A Ranking System in Go
Example: Ranking Service Popular Posts Fetch posts Filter posts Score posts User Post Views -
37 From Service to Platform: A Ranking System in Go
Example: Ranking Service Popular Posts Fetch posts Filter posts Score posts User Post Views Model -
38 From Service to Platform: A Ranking System in Go
Example: Ranking Service Popular Posts Fetch posts Filter posts Score posts User Post Views Model -
39 From Service to Platform: A Ranking System in Go
Example: Ranking Service Popular Posts Fetch posts Filter posts Score posts Video Posts User Post Views Model -
40 From Service to Platform: A Ranking System in Go
Example: Ranking Service Popular Posts Fetch posts Filter posts Score posts Video Posts User Post Views Model remove duplicates -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
Example: Ranking Service func (s *Service) GetPopularFeed(ctx context.Context, req *pb.FeedRequest)
(*pb.PopularFeed, error) { posts, err := s.fetchPopularAndVideoPosts(ctx) if err != nil { return nil, err } imagePosts, err := s.cache.FetchImagePosts(ctx) if err != nil { return nil, err } posts = s.filterPosts(posts, imagePosts) posts, scores, err := s.model.ScorePosts(ctx, req.UserID, posts) if err != nil { return nil, err } posts = s.sortPosts(posts, scores) return pb.NewPopularFeed(posts), nil } -
53 From Service to Platform: A Ranking System in Go
UNIX Toolbox Philosophy Candidate Generation Filter Score -
54 From Service to Platform: A Ranking System in Go
UNIX Toolbox Philosophy Stage 1 Stage 2 … -
55 From Service to Platform: A Ranking System in Go
UNIX Toolbox Philosophy Stage 1 Stage 2 … type Stage interface { Rank(ctx context.Context, req *pb.Request) } -
56 From Service to Platform: A Ranking System in Go
UNIX Toolbox Philosophy Stage 1 Stage 2 … type Stage interface { Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) } -
57 From Service to Platform: A Ranking System in Go
UNIX Toolbox Philosophy Stage 1 Stage 2 … type Stage interface { Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) } type Request struct { Context *Entity Candidates []*Entity } -
58 From Service to Platform: A Ranking System in Go
UNIX Toolbox Philosophy Stage 1 Stage 2 … type Stage interface { Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) } type Request struct { Context *Entity Candidates []*Entity } -
59 From Service to Platform: A Ranking System in Go
type Request struct { Context *Entity Candidates []*Entity } -
60 From Service to Platform: A Ranking System in Go
type Request struct { Context *Entity Candidates []*Entity } -
61 From Service to Platform: A Ranking System in Go
type Request struct { Context *Entity Candidates []*Entity } type Entity struct { ID string Features map[string]*Feature Score float64 } -
62 From Service to Platform: A Ranking System in Go
type Request struct { Context *Entity Candidates []*Entity } type Entity struct { ID string Features map[string]*Feature Score float64 } -
63 From Service to Platform: A Ranking System in Go
type Request struct { Context *Entity Candidates []*Entity } type Entity struct { ID string Features map[string]*Feature Score float64 } -
64 From Service to Platform: A Ranking System in Go
type Request struct { Context *Entity Candidates []*Entity } type Entity struct { ID string Features map[string]*Feature Score float64 } -
65 From Service to Platform: A Ranking System in Go
UNIX Toolbox Philosophy Stage 1 Stage 2 … type Stage interface { Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) } type Request struct { Context *Entity Candidates []*Entity } -
66 From Service to Platform: A Ranking System in Go
UNIX Toolbox Philosophy Stage 1 Stage 2 … type Stage interface { Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) } type Request struct { Context *Entity Candidates []*Entity } -
gRPC Protobuf Definition syntax = "proto3"; service Ranking { rpc
Rank (Request) returns (Request); } -
gRPC Protobuf Definition syntax = "proto3"; service Ranking { rpc
Rank (Request) returns (Request); } message Request { Entity context = 1; repeated Entity candidates = 2; RequestOptions options = 3; } -
gRPC Protobuf Definition message Entity { string id = 1;
map<string, Feature> features = 2; double score = 3; } message Feature { oneof value { string as_string = 1; int64 as_int = 2; double as_float = 3; bool as_bool = 4; // ... }; } -
gRPC Protobuf Request Example context: { id: "t2_bd5ts" features: {
key: "geo_city" value: { as_string: "SAN_FRANCISCO" } } features: { key: "geo_country" value: { as_string: "US" } } } options: { method: "rank_popular_feed" limit: 20 } -
gRPC Service type server struct { *grpc.Server stage stage.Stage }
func (s *server) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { return s.stage.Rank(ctx, req) } -
72 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit -
73 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Fetch Image Posts -
74 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Fetch Image Posts Series -
75 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Fetch Image Posts Series -
76 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Merge Candidates Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Fetch Image Posts Series -
77 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Merge Candidates Fetch Recently Viewed Posts Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Series Fetch Image Posts Series Filter Recently Viewed Posts -
78 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Merge Candidates Fetch Recently Viewed Posts Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Series Fetch Image Posts Series Score Candidates Filter Recently Viewed Posts Sort Candidates -
79 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Merge Candidates Fetch Recently Viewed Posts Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Series Fetch Image Posts Series Score Candidates Filter Recently Viewed Posts Sort Candidates Candidates Features Filtering Meta-Stages -
80 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Merge Candidates Fetch Recently Viewed Posts Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Series Fetch Image Posts Series Score Candidates Filter Recently Viewed Posts Sort Candidates Candidates Features Filtering Meta-Stages -
Stage: Fetch Popular Posts type fetchPopularPosts struct { cache *store.PostCache
} func FetchPopularPosts(cache *store.PostCache) *fetchPopularPosts { return &fetchPopularPosts{cache: cache} } func (s *fetchPopularPosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { postIDs, err := s.cache.FetchPopularPostIDs(ctx) if err != nil { return nil, err } for _, id := range postIDs { req.Candidates = append(req.Candidates, pb.NewCandidate(postID)) } return req, nil } -
Stage: Fetch Popular Posts type fetchPopularPosts struct { cache *store.PostCache
} func FetchPopularPosts(cache *store.PostCache) *fetchPopularPosts { return &fetchPopularPosts{cache: cache} } func (s *fetchPopularPosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { postIDs, err := s.cache.FetchPopularPostIDs(ctx) if err != nil { return nil, err } for _, id := range postIDs { req.Candidates = append(req.Candidates, pb.NewCandidate(id)) } return req, nil } -
Stage: Fetch Popular Posts type fetchPopularPosts struct { cache *store.PostCache
} func FetchPopularPosts(cache *store.PostCache) *fetchPopularPosts { return &fetchPopularPosts{cache: cache} } func (s *fetchPopularPosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { postIDs, err := s.cache.FetchPopularPostIDs(ctx) if err != nil { return nil, err } for _, id := range postIDs { req.Candidates = append(req.Candidates, pb.NewCandidate(id)) } return req, nil } -
Stage: Fetch Popular Posts type fetchPopularPosts struct { cache *store.PostCache
} func FetchPopularPosts(cache *store.PostCache) *fetchPopularPosts { return &fetchPopularPosts{cache: cache} } func (s *fetchPopularPosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { postIDs, err := s.cache.FetchPopularPostIDs(ctx) if err != nil { return nil, err } for _, id := range postIDs { req.Candidates = append(req.Candidates, pb.NewCandidate(id)) } return req, nil } -
Stage: Fetch Popular Posts type fetchPopularPosts struct { cache *store.PostCache
} func FetchPopularPosts(cache *store.PostCache) *fetchPopularPosts { return &fetchPopularPosts{cache: cache} } func (s *fetchPopularPosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { postIDs, err := s.cache.FetchPopularPostIDs(ctx) if err != nil { return nil, err } for _, id := range postIDs { req.Candidates = append(req.Candidates, pb.NewCandidate(id)) } return req, nil } -
86 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Merge Candidates Fetch Recently Viewed Posts Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Series Fetch Image Posts Series Score Candidates Filter Recently Viewed Posts Sort Candidates Candidates Features Filtering Meta-Stages -
Stage: Filtering Recently Viewed Posts func (s *filterRecentlyViewedPosts) Rank(ctx context.Context,
req *pb.Request) (*pb.Request, error) { seen := req.Context.Features["recently_viewed_post_ids"].GetAsBoolMap() var filtered []*pb.Entity for _, candidate := range req.Candidates { if !seen[candidate.Id] { filtered = append(filtered, candidate) } } req.Candidates = filtered return req, nil } -
Stage: Filtering Recently Viewed Posts func (s *filterRecentlyViewedPosts) Rank(ctx context.Context,
req *pb.Request) (*pb.Request, error) { seen := req.Context.Features["recently_viewed_post_ids"].GetAsBoolMap() var filtered []*pb.Entity for _, candidate := range req.Candidates { if !seen[candidate.Id] { filtered = append(filtered, candidate) } } req.Candidates = filtered return req, nil } -
Stage: Filtering Recently Viewed Posts func (s *filterRecentlyViewedPosts) Rank(ctx context.Context,
req *pb.Request) (*pb.Request, error) { seen := req.Context.Features["recently_viewed_post_ids"].GetAsBoolMap() n := 0 for _, candidate := range req.Candidates { if !seen[candidate.Id] { req.Candidates[n] = candidate n++ } } req.Candidates = req.Candidates[:n] // in-place filtering return req, nil } -
90 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Merge Candidates Fetch Recently Viewed Posts Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Series Fetch Image Posts Series Score Candidates Filter Recently Viewed Posts Sort Candidates Candidates Features Filtering Meta-Stages -
91 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Merge Candidates Fetch Recently Viewed Posts Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Series Fetch Image Posts Series Score Candidates Filter Recently Viewed Posts Sort Candidates Candidates Features Filtering Meta-Stages -
Meta-Stage: Series type series struct { stages []Stage } func
Series(stages ...Stage) *series { return &series{stages: stages} } func (s *series) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { var err error resp := req for _, stage := range s.stages { resp, err = stage.Rank(ctx, req) if err != nil { return nil, err } req = resp } return resp, nil } -
type series struct { stages []Stage } func Series(stages ...Stage)
*series { return &series{stages: stages} } func (s *series) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { var err error resp := req for _, stage := range s.stages { resp, err = stage.Rank(ctx, req) if err != nil { return nil, err } req = resp } return resp, nil } Meta-Stage: Series -
type series struct { stages []Stage } func Series(stages ...Stage)
*series { return &series{stages: stages} } func (s *series) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { var err error resp := req for _, stage := range s.stages { resp, err = stage.Rank(ctx, req) if err != nil { return nil, err } req = resp } return resp, nil } Meta-Stage: Series -
type series struct { stages []Stage } func Series(stages ...Stage)
*series { return &series{stages: stages} } func (s *series) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { var err error resp := req for _, stage := range s.stages { resp, err = stage.Rank(ctx, req) if err != nil { return nil, err } req = resp } return resp, nil } Meta-Stage: Series -
type series struct { stages []Stage } func Series(stages ...Stage)
*series { return &series{stages: stages} } func (s *series) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { var err error resp := req for _, stage := range s.stages { resp, err = stage.Rank(ctx, req) if err != nil { return nil, err } req = resp } return resp, nil } Meta-Stage: Series -
Meta-Stage: Parallel func (s *parallel) Rank(ctx context.Context, req *pb.Request) (*pb.Request,
error) { resps := make([]*pb.Request, len(s.stages)) g, groupCtx := errgroup.WithContext(ctx) for i := range s.stages { i := i g.Go(func() error { defer log.CapturePanic(groupCtx) resp, err := s.stages[i].Rank(groupCtx, pb.Copy(req)) if err != nil { return err } resps[i] = resp return nil }) } if err := g.Wait(); err != nil { return nil, err } return s.merge(ctx, req, resps...) } -
Meta-Stage: Parallel func (s *parallel) Rank(ctx context.Context, req *pb.Request) (*pb.Request,
error) { resps := make([]*pb.Request, len(s.stages)) g, groupCtx := errgroup.WithContext(ctx) for i := range s.stages { i := i g.Go(func() error { defer log.CapturePanic(groupCtx) resp, err := s.stages[i].Rank(groupCtx, pb.Copy(req)) if err != nil { return err } resps[i] = resp return nil }) } if err := g.Wait(); err != nil { return nil, err } return s.merge(ctx, req, resps...) } golang.org/x/sync/errgroup -
Meta-Stage: Parallel func (s *parallel) Rank(ctx context.Context, req *pb.Request) (*pb.Request,
error) { resps := make([]*pb.Request, len(s.stages)) g, groupCtx := errgroup.WithContext(ctx) for i := range s.stages { i := i g.Go(func() error { defer log.CapturePanic(groupCtx) resp, err := s.stages[i].Rank(groupCtx, pb.Copy(req)) if err != nil { return err } resps[i] = resp return nil }) } if err := g.Wait(); err != nil { return nil, err } return s.merge(ctx, req, resps...) } Calls the function in a goroutine, first non-nil error to be returned cancels the group -
Meta-Stage: Parallel func (s *parallel) Rank(ctx context.Context, req *pb.Request) (*pb.Request,
error) { resps := make([]*pb.Request, len(s.stages)) g, groupCtx := errgroup.WithContext(ctx) for i := range s.stages { i := i g.Go(func() error { defer log.CapturePanic(groupCtx) resp, err := s.stages[i].Rank(groupCtx, pb.Copy(req)) if err != nil { return err } resps[i] = resp return nil }) } if err := g.Wait(); err != nil { return nil, err } return s.merge(ctx, req, resps...) } -
Meta-Stage: Parallel func (s *parallel) Rank(ctx context.Context, req *pb.Request) (*pb.Request,
error) { resps := make([]*pb.Request, len(s.stages)) g, groupCtx := errgroup.WithContext(ctx) for i := range s.stages { i := i g.Go(func() error { defer log.CapturePanic(groupCtx) resp, err := s.stages[i].Rank(groupCtx, pb.Copy(req)) if err != nil { return err } resps[i] = resp return nil }) } if err := g.Wait(); err != nil { return nil, err } return s.merge(ctx, req, resps...) } -
Meta-Stage: Parallel func (s *parallel) Rank(ctx context.Context, req *pb.Request) (*pb.Request,
error) { resps := make([]*pb.Request, len(s.stages)) g, groupCtx := errgroup.WithContext(ctx) for i := range s.stages { i := i g.Go(func() error { defer log.CapturePanic(groupCtx) resp, err := s.stages[i].Rank(groupCtx, pb.Copy(req)) if err != nil { return err } resps[i] = resp return nil }) } if err := g.Wait(); err != nil { return nil, err } return s.merge(ctx, req, resps...) } -
Meta-Stage: If-Else type Selector func(context.Context, *pb.Request) bool func (s *ifElse)
Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { if s.selector(ctx, req) { return s.ifStage.Rank(ctx, req) } return s.elseStage.Rank(ctx, req) } -
104 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Fetch Popular Posts Fetch Video Posts Series Series Parallel Merge Candidates Fetch Recently Viewed Posts Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit Series Fetch Image Posts Series Score Candidates Filter Recently Viewed Posts Sort Candidates Candidates Features Filtering Meta-Stages -
105 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit func PopularFeed(d *service.Dependencies) stage.Stage { return stage.Series( stage.Parallel(merger.MergeCandidates, stage.FetchPopularPosts(d.PostCache), stage.FetchVideoPosts(d.PostCache), stage.FetchImagePosts(d.PostCache), ), stage.FetchRecentlyViewedPosts(d.UserPostViews), stage.FilterRecentlyViewedPosts(), stage.ScoreCandidates(d.RankingModel), stage.SortCandidates(), ) } -
From Service to Platform When does a service become a
platform? Service Customers are the users / product management -
From Service to Platform When does a service become a
platform? Service Customers are the users / product management Platform Customers are engineers developing on the service -
109 From Service to Platform: A Ranking System in Go
What is your API? Path /api/user/{id}.json Description Get a user object Method GET -
package user type User struct { ID string Name string
permissions []string } 110 From Service to Platform: A Ranking System in Go What is your API? -
package user type User struct { ID string Name string
permissions []string } 111 From Service to Platform: A Ranking System in Go What is your API? -
package user type User struct { ID string Name string
permissions []string } 112 From Service to Platform: A Ranking System in Go What is your API? -
package user type User struct { ID string Name string
permissions []string } 113 From Service to Platform: A Ranking System in Go What is your API? -
package user type User struct { ID string Name string
permissions []string } 114 From Service to Platform: A Ranking System in Go What is your API? -
package user type User struct { ID string Name string
permissions []string } 115 From Service to Platform: A Ranking System in Go What is your API? -
package user type User struct { ID string Name string
permissions []string } 116 From Service to Platform: A Ranking System in Go What is your API? • Start with unexported identifiers • Make decisions for exporting identifiers based on how you want them to be consumed -
117 A General-Purpose Ranking Service From Service to Platform: A
Ranking System in Go Quickly and flexibly perform complex scatter-gather ranking workflows at Reddit func PopularFeed(d *service.Dependencies) stage.Stage { return stage.Series( stage.Parallel(merger.MergeCandidates, stage.FetchPopularPosts(d.PostCache), stage.FetchVideoPosts(d.PostCache), stage.FetchImagePosts(d.PostCache), ), stage.FetchRecentlyViewedPosts(d.UserPostViews), stage.FilterRecentlyViewedPosts(), stage.ScoreCandidates(d.RankingModel), stage.SortCandidates(), ) } -
118 Transition to Platform From Service to Platform: A Ranking
System in Go Ranking Service -
119 Transition to Platform From Service to Platform: A Ranking
System in Go Ranking Service Livestream Feed -
120 Transition to Platform From Service to Platform: A Ranking
System in Go Ranking Service Livestream Feed … -
121 Transition to Platform From Service to Platform: A Ranking
System in Go Ranking Service Livestream Feed Popular Feed … -
122 Transition to Platform From Service to Platform: A Ranking
System in Go Ranking Service Livestream Feed Home Feed Popular Feed … -
123 Transition to Platform From Service to Platform: A Ranking
System in Go Ranking Service Livestream Feed Home Feed Popular Feed … … -
124 Transition to Platform From Service to Platform: A Ranking
System in Go Ranking Service Livestream Feed Home Feed Popular Feed … Ranking Platform … -
package stage 125 Maintaining the Platform From Service to Platform:
A Ranking System in Go Series Parallel Shuffle Candidates Fetch Posts Filter By Feature … If-Else Fetch Subscriptions -
package stage 126 Maintaining the Platform From Service to Platform:
A Ranking System in Go Series Parallel Shuffle Candidates Fetch Posts Filter By Feature … If-Else Fetch Subscriptions Filter Private Posts Filter by Timestamp Shuffle Topics … -
Creating A Toolbox Four Principles for a Platform API 127
From Service to Platform: A Ranking System in Go Limited Scope Clear Naming Decoupling Strive for Reuse -
Applying the Four Principles const shuffleProbability = 0.2 type imagePosts
struct { cache *store.PostCache } func (s *imagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature("subreddit_ids") postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } s.shufflePostIDs(postIDs, shuffleProbability) for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } -
Applying the Four Principles const shuffleProbability = 0.2 type imagePosts
struct { cache *store.PostCache } func (s *imagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature("subreddit_ids") postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } s.shufflePostIDs(postIDs, shuffleProbability) for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } -
Applying the Four Principles const shuffleProbability = 0.2 type imagePosts
struct { cache *store.PostCache } func (s *imagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature("subreddit_ids") postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } s.shufflePostIDs(postIDs, shuffleProbability) for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } -
Applying the Four Principles const shuffleProbability = 0.2 type imagePosts
struct { cache *store.PostCache } func (s *imagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature("subreddit_ids") postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } s.shufflePostIDs(postIDs, shuffleProbability) for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } Clear Naming -
Applying the Four Principles const shuffleProbability = 0.2 type fetchImagePosts
struct { cache *store.PostCache } func (s *fetchImagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature("subreddit_ids") postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } s.shufflePostIDs(postIDs, shuffleProbability) for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } Clear Naming -
Applying the Four Principles const shuffleProbability = 0.2 type fetchImagePosts
struct { cache *store.PostCache } func (s *fetchImagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature("subreddit_ids") postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } s.shufflePostIDs(postIDs, shuffleProbability) for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } Limited Scope -
Applying the Four Principles type fetchImagePosts struct { cache *store.PostCache
} func (s *fetchImagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature("subreddit_ids") postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } Limited Scope -
Applying the Four Principles type fetchImagePosts struct { cache *store.PostCache
} func (s *fetchImagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature("subreddit_ids") postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } Strive for Reuse -
Applying the Four Principles type fetchImagePosts struct { cache *store.PostCache
} func (s *fetchImagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature("subreddit_ids") postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } Strive for Reuse -
Applying the Four Principles type fetchImagePosts struct { cache *store.PostCache
subredditFeature string } func (s *fetchImagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature(s.subredditFeature) postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs) if err != nil { return nil, err } for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } Decoupling -
Applying the Four Principles type fetchImagePosts struct { cache *store.PostCache
subredditFeature string } func (s *fetchImagePosts) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { subredditIDs := req.Context.GetStringArrayFeature(s.subredditFeature) postIDs, err := s.cache.FetchImagePosts(ctx, subredditIDs...) if err != nil { return nil, err } for _, postID := range postIDs { req.Candidates = append(req.Candidates, pb.Candidate(postID)) } return req, nil } Decoupling -
Functions Implementing Interfaces “The bigger the interface, the weaker the
abstraction” ― Rob Pike type RankFunc func(context.Context, *pb.Request) (*pb.Request, error) -
Functions Implementing Interfaces “The bigger the interface, the weaker the
abstraction” ― Rob Pike type RankFunc func(context.Context, *pb.Request) (*pb.Request, error) func (f RankFunc) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { return f(ctx, req) } -
Functions Implementing Interfaces “The bigger the interface, the weaker the
abstraction” ― Rob Pike type RankFunc func(context.Context, *pb.Request) (*pb.Request, error) func (f RankFunc) Rank(ctx context.Context, req *pb.Request) (*pb.Request, error) { return f(ctx, req) } -
func Pipeline(d *service.Dependencies) stage.Stage { return stage.Series( stage.FetchSubscriptions(d.SubscriptionService), stage.FetchPosts(d.Cache), stage.FilterPostsForUK(),
stage.ShufflePosts(0.2), ) } Anonymous Interface Implementations -
func Pipeline(d *service.Dependencies) stage.Stage { return stage.Series( stage.FetchSubscriptions(d.SubscriptionService), stage.FetchPosts(d.Cache), stage.FilterPostsForUK(),
stage.ShufflePosts(0.2), ) } Anonymous Interface Implementations -
func Pipeline(d *service.Dependencies) stage.Stage { return stage.Series( stage.FetchSubscriptions(d.SubscriptionService), stage.FetchPosts(d.Cache), stage.FilterPostsForUK(),
stage.ShufflePosts(0.2), ) } Anonymous Interface Implementations -
func Pipeline(d *service.Dependencies) stage.Stage { return stage.Series( stage.FetchSubscriptions(d.SubscriptionService), stage.FetchPosts(d.Cache), stage.RankFunc(func(context.Context,
*pb.Request) (*pb.Request, error) { if req.Context.Features["geo_country"] == "uk" { // ... } return req, nil }), stage.ShufflePosts(0.2), ) } Anonymous Interface Implementations -
Middlewares type Middleware func(stage Stage) Stage func ExampleMiddleware(next stage.Stage) stage.Stage
{ return stage.RankFunc(func(ctx context.Context, req *pb.Request) (*pb.Request, error) { // ... return next.Rank(ctx, req) }) } -
func Monitor(next stage.Stage) stage.Stage { return stage.RankFunc(func(ctx context.Context, req *pb.Request)
(*pb.Request, error) { startedAt := time.Now() method := req.Options.Method defer func() { stageLatencySeconds.With(prometheus.Labels{ methodLabel: method, stageLabel: stage.Name(next), }).Observe(time.Since(startedAt).Seconds()) }() return next.Rank(ctx, req) }) } Middleware: Monitor -
func Monitor(next stage.Stage) stage.Stage { return stage.RankFunc(func(ctx context.Context, req *pb.Request)
(*pb.Request, error) { startedAt := time.Now() method := req.Options.Method defer func() { stageLatencySeconds.With(prometheus.Labels{ methodLabel: method, stageLabel: stage.Name(next), }).Observe(time.Since(startedAt).Seconds()) }() return next.Rank(ctx, req) }) } Middleware: Monitor Record elapsed time in deferred statement Delegate to underlying stage -
Middleware: Log func Log(next stage.Stage) stage.Stage { return stage.RankFunc(func(ctx context.Context,
req *pb.Request) (resp *pb.Request, err error) { defer func() { if err != nil { log.Errorw( "stage failed", "error", err, "request", req.JSON(), "response", resp.JSON(), "stage", stage.Name(stage), ) } }() return stage.Rank(ctx, req) }) } -
Middleware: Log func Log(next stage.Stage) stage.Stage { return stage.RankFunc(func(ctx context.Context,
req *pb.Request) (resp *pb.Request, err error) { defer func() { if err != nil { log.Errorw( "stage failed", "error", err, "request", req.JSON(), "response", resp.JSON(), "stage", stage.Name(stage), ) } }() return stage.Rank(ctx, req) }) } func (r *Request) JSON() string { b, err := protojson.Marshal(r) if err != nil { log.Error(err) return "" } return string(b) } -
Middleware: Feature Flags for Incident Mitigation func FeatureFlag(next stage.Stage) stage.Stage
{ return stage.RankFunc(func(ctx context.Context, req *pb.Request) (*pb.Request, error) { key := "feature_flag.stage." + stage.Name(current) if !liveconfig.GetBool(key) { return req, nil } return next.Rank(ctx, req) }) } -
Middleware: Feature Flags for Incident Mitigation func FeatureFlag(next stage.Stage) stage.Stage
{ return stage.RankFunc(func(ctx context.Context, req *pb.Request) (*pb.Request, error) { key := "feature_flag.stage." + stage.Name(current) if !liveconfig.GetBool(key) { return req, nil } return next.Rank(ctx, req) }) } Skip stage entirely -
A Framework for Refactoring • Being forced to write your
business logic into small components helped to limit accidental complexity but it did not eliminate the need for refactoring • Our ability to refactor increased due stage interface providing a framework • This framework requires a set of principles for designing those components • Any code contributed to middlewares or meta-stages pays off due to their multiplier effect -
• Platform-centric thinking starts with the first developers outside of
our team contributing code—this is not something you can simulate. • Providing an opinionated framework will create friction. • Ways to resolve confusion or disagreement: 1. Enforce the existing design 2. Quick-and-dirty workaround 3. Rethink the existing design Platform Building is a Discourse -
Meta-Stage: Parallel func (s *parallel) Rank(ctx context.Context, req *pb.Request) (*pb.Request,
error) { resps := make([]*pb.Request, len(s.stages)) g, groupCtx := errgroup.WithContext(ctx) for i := range s.stages { i := i g.Go(func() error { defer log.CapturePanic(groupCtx) resp, err := s.stages[i].Rank(groupCtx, pb.Copy(req)) if err != nil { return err } resps[i] = resp return nil }) } if err := g.Wait(); err != nil { return nil, err } return s.merge(ctx, req, resps...) } -
172 What’s next? No-Code Abstraction --- name: PopularFeed stages: -
name: Series stages: - name: Parallel stages: - name: FetchPopularPosts - name: FetchVideoPosts - name: FetchImagePosts - name: FetchRecentlyViewedPosts - name: FilterRecentlyViewedPosts - name: ScoreCandidares - name: SortCandidates -
173 What’s next? Pipelines at Runtime --- name: PopularFeed stages:
- name: Series stages: - name: Parallel stages: - name: FetchPopularPosts - name: FetchVideoPosts - name: FetchImagePosts - name: FetchRecentlyViewedPosts - name: FilterRecentlyViewedPosts - name: ScoreCandidares - name: SortCandidates -
Summary • Essential complexity describes a problem at its core,
accidental complexity happens as part of solving the problem, creating unnecessary challenges; accidental complexity can be reduced • A recommendation system consists of a variety of different ranking flows with the goal to generate content that users find compelling • UNIX pipes as an inspiration to build a system of small, reusable components with the help of the single-method interface in Go • Reusability and clarity are competing concepts. This is Go: choose clarity.