Introducing Hyperion

Transcript

1. Introducing...
2. What is Hyperion? • A persistence library for Ruby and Clojure that supports many different databases • An idea - choosing your database is a decision that can be deferred
3. Part 1: The Idea
4. Gaeshi • Clojure framework for building web apps
5. Gaeshi Datastore - GAE find-by-kind find-by-key find-by-keys save save* count-by-kind find-by-key
6. Clean Coders - built on Gaeshi
7. Hosemonster
8. Hosemonster • Desktop application • How do we store data? • Forget about it - decide later
9. Hosemonster • Months later - persistence decided • In the app - implement another datastore • Build a service with Gaeshi
10. <UI> DesktopUI Core <Datastore> InMemory File Remote Gaeshi REST API Hosemonster Architecture
11. Hosemonster • We had 3 datastore implementations ◦ our API was similar, but worse, than the GAE datastore API ◦ merge the APIs ◦ extract into reusable library - Hyperion
12. Hosemonster • Customer wants a web application
13. Hosemonster • Conversion to web-app complete ◦ Deploy to staging (GAE) - boom! ◦ PDF generation doesn't work • Move off of GAE ◦ Moved to Heroku ◦ Write migrations, fix bugs in Hyperion ◦ Switched databases in one weekend
14. Hosemonster • Fast-forward 9 months ◦ production deployment ◦ deploy manually to AWS • Switch to MongoDB ◦ Application code did not have to change ▪ only reconfigure Hyperion
15. Hosemonster • Started with a simple idea ◦ defer the persistence decision ◦ once the decision is made - keep the abstraction and make it better
16. Part 2: The Library
17. Goals • Defer deciding on your "real" database until you have to
18. Hyperion Clojure • In-memory - testing datastore • Google App Engine Datastore • Mongo DB • MySQL • PostgreSQL • Riak • SQLite • Redis
19. Hyperion Ruby • In-memory testing datastore • MySQL • PostgreSQL • Riak • SQLite • Redis
20. Design Principles 1. key/value store All Hyperion implementations, even for relational databases, conform to the simple key/value store API. 2. values are maps Every 'value' the goes in or out of a Hyperion datastore is map. 3. :key and :kind Every 'value' must have a :kind entry; a short string like "user" or "product". Persisted 'value's will have a :key entry; strings generated by the datastore. 4. search with data All queries are described by data.
21. Hyperion Code Examples Beware! Thar be Codez!
22. Examples: Instantiate Datastore (use 'hyperion.api) (new-datastore :implementation :memory) (new-datastore :implementation :mysql :connection-url "jdbc:mysql://localhost:3306/myapp?user=root" :database "myapp") (new-datastore :implementation :mongo :host "localhost" :port 27017 :database "myapp" :username "test" :password "test")
23. Examples: Installing a datastore ; with brute force (set-ds! (new-datastore ...)) ; with elegance (binding [*ds* (new-datastore ...)] ; persistence stuff here)
24. Examples: Saving stuff (save {:kind :foo}) ;=> {:kind "foo" :key "generated key"} (save {:kind :foo} {:value :bar}) ;=> {:kind "foo" :value :bar :key "generated key"} (save {:kind :foo} :value :bar) ;=> {:kind "foo" :value :bar :key "generated key"} (save {:kind :foo} {:value :bar} :another :fizz) ;=> {:kind "foo" :value :bar :another :fizz :key "generated key"} (save (citizen) :name "Joe" :age 21 :country "France") ;=> #<{:kind "citizen" :name "Joe" :age 21 :country "France" ...}>
25. Examples: Updating saved data (let [record (save {:kind :foo :name "Sue"}) new-record (assoc record :name "John")] (save new-record)) ;=> {:kind "foo" :name "John" :key "generated key"}
26. Examples: Finding Stuff ; if you have a key... (let [record (save {:kind :foo})] (find-by-key (:key record)) ; otherwise (find-by-kind :dog) (find-by-kind :dog :filters [:= :name "Fido"]) (find-by-kind :dog :filters [[:> :age 2][:< :age 5]]) (find-by-kind :dog :sorts [:name :asc]) (find-by-kind :dog :sorts [[:age :desc][:name :asc]]) (find-by-kind :dog :limit 10) (find-by-kind :dog :sorts [:name :asc] :limit 10) (find-by-kind :dog :sorts [:name :asc] :limit 10 :offset 10)
27. Examples: Deleting Stuff ; if you have a key... (delete-by-key my-key) ; otherwise (delete-by-kind :dog) (delete-by-kind :dog :filters [:= :name "Fido"]) (delete-by-kind :dog :filters [[:> :age 2][:< :age 5]])
28. Examples: defentity (defentity Citizen [name] [age :packer ->int] [gender :unpacker ->string] [occupation :type my.ns.Occupation] [spouse-key :type :key] [country :default "USA"] [created-at] [updated-at]) (save (citizen :name "John" :age "21" :gender :male :occupation coder :spouse-key "abc123"))
29. Implementing new Datastores 1. Implement the Protocol 2. Create a factory function 3. Pass the spec
30. Implement the Protocol Datastore (defprotocol Datastore (ds-save [this records]) (ds-delete-by-key [this key]) (ds-delete-by-kind [this kind filters]) (ds-count-by-kind [this kind filters]) (ds-find-by-key [this key]) (ds-find-by-kind [this kind filters sorts limit offset]) (ds-all-kinds [this]) (ds-pack-key [this value]) (ds-unpack-key [this value]))
31. Hyperion Ruby
32. Examples: Instantiate Datastore require 'hyperion' Hyperion.new_datastore(:memory) Hyperion.new_datastore(:mysql) Hyperion.new_datastore(:postgres, options) Hyperion.new_datastore(:sqlite, options)
33. Examples: Installing a datastore # datastore not installed Hyperion.with_datastore(:postgres, options) do # datastore is installed # some persistence code here end # datastore not installed Hyperion.datastore = Hyperion.new_datastore(:postgres, options)
34. Examples: Saving stuff Hyperion.save({:kind => :foo}) #=> {:kind=>"foo", :key=>"<generated key>"} Hyperion.save({:kind => :foo}, :value => :bar) #=> {:kind=>"foo", :value=>:bar, :key=>" <generated key>"}
35. Examples: Updating saved data record = Hyperion.save({kind: :foo, name: 'Sam'}) Hyperion.save(record, :name => 'John') #=> {:kind=>"foo", :name=>'John', :key=>" <generated key>"}
36. Examples: Finding Stuff # if you have a key... Hyperion.find_by_key(my_key) # otherwise Hyperion.find_by_kind(:dog) Hyperion.find_by_kind(:dog, :filters => [[:name, '=', "Fido"]]) Hyperion.find_by_kind(:dog, :filters => [[:age, '>', 2], [:age, '<', 5]]) Hyperion.find_by_kind(:dog, :sorts => [[:age, :desc], [:name, :asc]]) Hyperion.find_by_kind(:dog, :limit => 10, :offset => 15) Hyperion.find_by_kind(:dog, :offset => 15) Hyperion.find_by_kind(:dog, :sorts => [[:name, :asc]])
37. Examples: Deleting Stuff # if you have a key... Hyperion.delete_by_key(my_key) # otherwise Hyperion.delete_by_kind(:dog) Hyperion.delete_by_kind(:dog, :filters => [[:name, "=", "Fido"]]) Hyperion.delete_by_kind(:dog, :filters => [[:age, ">", 2], [:age, "<", 5]])
38. Examples: defentity Hyperion.defentity(:defaulted) do |kind| kind.field(:name) kind.field(:age, :default => 25) end
39. Excluded Features • Associations • Custom finders • Lazy bindings • Validations • Adding behavior to data (Objects)
40. Conclusion • Big idea - Defer the database decision • Hyperion is a library built for this purpose