Apache OpenWhisk - Pluggable Event Providers

Presentation to Apache OpenWhisk community call on my experiment building a pluggable trigger feed event provider.

Transcript

1. APACHE OPENWHISK - PLUGGABLE EVENT PROVIDERS James Thomas, OpenWhisk Committer, Serverless Dev Adv Lead @ IBM ☁
2. APACHE OPENWHISK TRIGGER FEED PROVIDERS ➤API Gateway ➤Apache Kafka ➤Apache CouchDB ➤Cron Alarm ➤Github ➤…
3. APACHE OPENWHISK TRIGGER FEED PROVIDERS Why don’t we have more? Lots of community contributions for runtimes Not so much for feed providers…
4. APACHE OPENWHISK TRIGGER FEED PROVIDERS Why don’t we have more? Trigger feeds are not that simple to implement. Documentation is not great.
5. APACHE OPENWHISK TRIGGER FEED PROVIDERS Was asked internally to produce an S3 compatible feed provider. Use polling to check bucket file etags for file changes.
6. HOW OPENWHISK TRIGGER FEEDS WORK wsk trigger create periodic \ --feed /whisk.system/alarms/alarm \ --param cron "*/2 * * * *" \ --param trigger_payload “{…}” \ --param startDate "2019-01-01T00:00:00.000Z" \ --param stopDate "2019-01-31T23:59:00.000Z" feed parameter refers to a normal action action invoked with the trigger details by cli
7. TRIGGER FEED API custom feed provider is supposed to manage listening to event source and firing triggers on events
8. EXISTING TRIGGER FEED PROVIDERS ARCHITECTURE - ADDING TRIGGER Feed Action Web Action Trigger Database Event Provider Event Source User Trigger wsk CLI - User creates trigger with feed with wsk CLI tool - CLI calls feed action with trigger & auth details - Feed action calls web action (?) - Web action writes to Trigger management database
9. EXISTING TRIGGER FEED PROVIDERS ARCHITECTURE - EVENT PROVIDER Feed Action Web Action Trigger Database Event Provider Event Source User Trigger wsk CLI - Event provider follows DB updates - Uses trigger details to start listening to external event source - Web action writes to Trigger management database
10. EXISTING TRIGGER FEED PROVIDERS ARCHITECTURE - EVENTS OCCUR Feed Action Web Action Trigger Database Event Provider Event Source User Trigger wsk CLI - Provider fires trigger with event details when they occur.
11. APACHE OPENWHISK TRIGGER FEED PROVIDERS Looking at the different feed providers, most of the code is not specific to the event source. 90% of code base = generic trigger management
12. “Can we build a generic event provider with a plugin architecture? - Easier to contribute event sources - Easier to maintain feed provider
13. GENERIC TRIGGER FEED PROVIDERS ARCHITECTURE Feed Action Web Action Trigger Database Generic Event Provider Custom Event Source - Plugin handles custom event source integration - Trigger management handled by generic provider Custom Event Source Plugin
20. RESULT = event provider in ~300 lines of JS
21. Is this interesting to others? Possible next steps… - Open-source code on Github? - Write more event provider plugins? - Donate to Apache (eventually)?