On Ferrets and Gophers

MongoDB was a life-changing technology for many developers with its smooth UX. Unfortunately, reliability and real-life performance were lacking for a long time, and a license change made it unsuitable for many companies and projects. In my talk, I will introduce FerretDB – an open-source proxy between MongoDB drivers and PostgreSQL. It is built using Go 1.18 to benefit from type parameters (generics) and fuzzing, and I will show how they are useful already.

Transcript

1. On Ferrets and Gophers Introducing FerretDB Alexey Palazhchenko GDG Berlin Golang 2022-01-12
2. About me
3. About me
4. About me
5. About me
6. About me
7. About me
8. MongoDB
9. MongoDB • Great Developer Experience
10. MongoDB • Great Developer Experience • Used to send iPads to community members
11. MongoDB • Great Developer Experience • Used to send iPads to community members • Reliability
12. MongoDB • Great Developer Experience • Used to send iPads to community members • Reliability • Performance
13. MongoDB • Great Developer Experience • Used to send iPads to community members • Reliability • Performance • Scalability
14. MongoDB • Great Developer Experience • Used to send iPads to community members • Reliability • Performance • Scalability
15. MongoDB • Great Developer Experience • Used to send iPads to community members • Reliability • Performance • Scalability • … does too much?
16. MongoDB • Great Developer Experience • Used to send iPads to community members • Reliability • Performance • Scalability • … does too much? • License change
17. Not a legal advice
18. Not a legal advice
19. AGPL v3 SSPL
20. AGPL v3 • «If you modify the Program» SSPL
21. AGPL v3 • «If you modify the Program» • «All users interacting with it remotely through a computer network» SSPL
22. AGPL v3 • «If you modify the Program» • «All users interacting with it remotely through a computer network» • «Interacting» is not a term SSPL
23. AGPL v3 • «If you modify the Program» • «All users interacting with it remotely through a computer network» • «Interacting» is not a term • «Corresponding Source» SSPL
24. AGPL v3 • «If you modify the Program» • «All users interacting with it remotely through a computer network» • «Interacting» is not a term • «Corresponding Source» • «Including scripts to control those activities» SSPL
25. AGPL v3 • «If you modify the Program» • «All users interacting with it remotely through a computer network» • «Interacting» is not a term • «Corresponding Source» • «Including scripts to control those activities» • «If you make the functionality of the Program» SSPL
26. AGPL v3 • «If you modify the Program» • «All users interacting with it remotely through a computer network» • «Interacting» is not a term • «Corresponding Source» • «Including scripts to control those activities» • «If you make the functionality of the Program» • «available to third parties as a service» SSPL
27. AGPL v3 • «If you modify the Program» • «All users interacting with it remotely through a computer network» • «Interacting» is not a term • «Corresponding Source» • «Including scripts to control those activities» • «If you make the functionality of the Program» • «available to third parties as a service» • «includes, without limitation, […]»
 very broad, but not very clear SSPL
28. AGPL v3 • «If you modify the Program» • «All users interacting with it remotely through a computer network» • «Interacting» is not a term • «Corresponding Source» • «Including scripts to control those activities» • «If you make the functionality of the Program» • «available to third parties as a service» • «includes, without limitation, […]»
 very broad, but not very clear • «Service Source Code» SSPL
29. AGPL v3 • «If you modify the Program» • «All users interacting with it remotely through a computer network» • «Interacting» is not a term • «Corresponding Source» • «Including scripts to control those activities» • «If you make the functionality of the Program» • «available to third parties as a service» • «includes, without limitation, […]»
 very broad, but not very clear • «Service Source Code» • «including, without limitation, management software, user interfaces, application program interfaces, […], all such that a user could run an instance of the service using the Service Source Code you make available.» SSPL
30. SSPL FAQ
31. SSPL FAQ
32. The SSPL is Not 
 an Open Source License
33. The SSPL is Not 
 an Open Source License https://opensource.org/node/1099
34. Protocol speci fi cation license
35. FerretDB
36. FerretDB • MongoDB protocol proxy for PostgreSQL
37. FerretDB • MongoDB protocol proxy for PostgreSQL • Accepts unmodi fi ed client connections
38. FerretDB • MongoDB protocol proxy for PostgreSQL • Accepts unmodi fi ed client connections • Parses BSON queries
39. FerretDB • MongoDB protocol proxy for PostgreSQL • Accepts unmodi fi ed client connections • Parses BSON queries • Converts them to SQL
40. FerretDB • MongoDB protocol proxy for PostgreSQL • Accepts unmodi fi ed client connections • Parses BSON queries • Converts them to SQL • Sends them to PostgreSQL
41. FerretDB • MongoDB protocol proxy for PostgreSQL • Accepts unmodi fi ed client connections • Parses BSON queries • Converts them to SQL • Sends them to PostgreSQL • Apache 2.0 licensed
42. PostgreSQL
43. PostgreSQL • Reliable
44. PostgreSQL • Reliable • Performant
45. PostgreSQL • Reliable • Performant • Scalable
46. PostgreSQL • Reliable • Performant • Scalable • Feature-rich
47. MangoDB Day
48. MangoDB Day
49. Why Go 1.18?
50. Why Go 1.18? Generics
51. Why Go 1.18? Generics Fuzzing
52. Why Go 1.18? Generics Fuzzing
53. Generics
54. Generics • Generics do not work there; let's go back to interfaces
55. Generics • Generics do not work there; let's go back to interfaces • Let me extract a small example for meetup
56. Generics • Generics do not work there; let's go back to interfaces • Let me extract a small example for meetup • Hm, that's not that bad
57. BSON Document
58. BSON Document • Ordered map
59. BSON Document • Ordered map • Keys – string, values – any BSON value, including other documents
60. BSON Document • Ordered map • Keys – string, values – any BSON value, including other documents • The central data structure in FerretDB
61. Interfaces type BSONType interface { bsontype() // sealed } type Int int func (Int) bsontype() {} type String string func (String) bsontype() {}
62. Interfaces type BSONType interface { bsontype() // sealed } type Int int func (Int) bsontype() {} type String string func (String) bsontype() {} type Document struct { m map[string]BSONType keys []string } func (*Document) bsontype() {}
63. Interfaces func (d *Document) Set(key string, value BSONType) { if _, ok := d.m[key]; !ok { d.keys = append(d.keys, key) } d.m[key] = value }
64. Interfaces d.Set("foo", Int(42))
65. Interfaces d.Set("foo", Int(42)) d.Set("qux", nil)
66. Generics type BSONType interface { int | string | *Document }
67. Generics type BSONType interface { int | string | *Document } type Document struct { m map[string]any keys []string }
68. Generics func DocumentSet[T BSONType](d *Document, key string, value T) { if _, ok := d.m[key]; !ok { d.keys = append(d.keys, key) } d.m[key] = value }
69. Generics DocumentSet(d, "foo", 42)
70. Generics DocumentSet(d, "foo", 42) // DocumentSet(d, "qux", nil)
71. Interfaces func MakeDocument(pairs ...BSONType) (*Document, error) { l := len(pairs) if l%2 != 0 { return nil, fmt.Errorf("%d arguments", l)
72. Interfaces
73. Interfaces MakeDocument(String("invalid number of arguments"))
74. Interfaces MakeDocument(String("invalid number of arguments")) MakeDocument(Int(42), String("invalid key type"))
75. Interfaces MakeDocument(String("invalid number of arguments")) MakeDocument(Int(42), String("invalid key type")) MakeDocument( String("foo"), Int(42), String("bar"), String("baz"), )
76. Generics
77. Generics func MakeDocument[T BSONType](key string, value T) *Document
78. Generics func MakeDocument[T BSONType](key string, value T) *Document func MakeDocument2[T1, T2 BSONType](key1 string, value1 T1, key2 string, value2 T2) *Document
79. Generics func MakeDocument[T BSONType](key string, value T) *Document func MakeDocument2[T1, T2 BSONType](key1 string, value1 T1, key2 string, value2 T2) *Document func MakeDocument3[T1, T2, T3 BSONType](key1 string, value1 T1, key2 string, value2 T2, key3 string, value3 T3) *Document
80. Generics vs Interfaces
81. Generics vs Interfaces • Better in some aspects
82. Generics vs Interfaces • Better in some aspects • Worse in others
83. Generics vs Interfaces • Better in some aspects • Worse in others • https://github.com/AlekSi/generics-vs-interfaces
84. Use Generics?
85. Use Generics?
86. Fuzzing
87. Fuzzing • testing/quick on steroids
88. Fuzzing • testing/quick on steroids • https://go.dev/blog/fuzz-beta
89. Fuzzing • testing/quick on steroids • https://go.dev/blog/fuzz-beta • MongoDB binary protocol parsing packages
90. Fuzzing • testing/quick on steroids • https://go.dev/blog/fuzz-beta • MongoDB binary protocol parsing packages • They had fuzzing tests even before unit tests
91. Fuzzing func FuzzDocument(f *testing.F) { fuzzBSON( f, documentTestCases, func() BSONType { return new(Document) }, ) }
92. Fuzzing func fuzzBSON(f *testing.F, tcs []testCase, newF func() BSONType) { for _, tc := range tcs { f.Add(tc.b) } // f.Run(tc.name, func(t *testing.T) f.Fuzz(func(t *testing.T, b []byte) {
93. Fuzzing func fuzzBSON(f *testing.F, tcs []testCase, newF func() BSONType) { for _, tc := range tcs { f.Add(tc.b) } // f.Run(tc.name, func(t *testing.T) f.Fuzz(func(t *testing.T, b []byte) { https://github.com/golang/go/issues/47413 👍
94. Fuzzing
95. Fuzzing • Found many bugs in FerretDB
96. Fuzzing • Found many bugs in FerretDB • Found a few bugs in `go test -fuzz`
97. Back to FerretDB
98. Back to FerretDB • Can't wait for Go 1.18 release to make it easier for contributors
99. Back to FerretDB • Can't wait for Go 1.18 release to make it easier for contributors • Speaking of contributors – our community is awesome!
100. Back to FerretDB • Can't wait for Go 1.18 release to make it easier for contributors • Speaking of contributors – our community is awesome! • And FerretDB Inc. started hiring about 1 hour ago
101. Links • https://www.ferretdb.io • https://github.com/FerretDB ⭐ • https://www.ferretdb.io/careers/ • https://github.com/AlekSi • @paaleksey • https://t.me/HowToGoWrong
 (in Russian)