Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Riak CS Usage and Billing Statistics

Riak CS Usage and Billing Statistics

A brief overview of Riak [0] and Riak CS [1], followed by details on multi-tenant usage and billing statistics.

After the slides, there is an interactive demonstration [2] on how to consume the statistics from Riak CS.

[0] http://basho.com/riak/
[1] http://basho.com/riak-cloud-storage/
[2] https://gist.github.com/hectcastro/126b5657f228096775c6

Hector Castro

May 08, 2013
Tweet

More Decks by Hector Castro

Other Decks in Technology

Transcript

  1. // Who are we? * Founded in 2008 by group

    of ex-Akamai, Mitre, Apple * 120+ employees; > 50% eng; distributed company * Sponsors of Riak, the Apache 2.0-licensed project * Basho sells add-ons to Riak: Riak EDS and Riak CS
  2. // What is Riak CS? * An Apache 2.0-licensed project

    * Built on top Riak * S3-compatible API (Swift coming soon)
  3. // What is Riak CS? * An Apache 2.0-licensed project

    * Built on top Riak * S3-compatible API (Swift coming soon) * Large object support and multipart upload
  4. // What is Riak CS? * An Apache 2.0-licensed project

    * Built on top Riak * S3-compatible API (Swift coming soon) * Large object support and multipart upload * Multi-tenancy and per-user reporting
  5. "Distributed, masterless, highly-available key/value store." * Deployed as cluster of

    nodes (>= 5), scales horizontally * Any node can coordinate requests, data replicated to 3 nodes by default, no SPOF
  6. "Distributed, masterless, highly-available key/value store." * Deployed as cluster of

    nodes (>= 5), scales horizontally * Any node can coordinate requests, data replicated to 3 nodes by default, no SPOF * Eventually consistent with automatic failover
  7. "Distributed, masterless, highly-available key/value store." * Deployed as cluster of

    nodes (>= 5), scales horizontally * Any node can coordinate requests, data replicated to 3 nodes by default, no SPOF * Eventually consistent with automatic failover * Key/value model with additional query methods
  8. +---------------+ | | | Large object | | | +---------------+

    | | +----------+ +----------+ +----------+ | S3 API | | S3 API | | S3 API | |----------| |----------| |----------| | | | | | | | RiakCS | | RiakCS | | RiakCS | | | | | | | +----------+ +----------+ +----------+
  9. +---------------+ | | | Large object | | | +---------------+

    | | +----------+ +----------+ +----------+ | S3 API | | S3 API | | S3 API | |----------| |----------| |----------| | | | | | | | RiakCS | | RiakCS | | RiakCS | | | | | | | +----------+ +----------+ +----------+ +---+ +---+ +---+ +---+ +---+ +---+ +---+ +---+ |1MB| |1MB| |1MB| |1MB| |1MB| |1MB| |1MB| |1MB| +---+ +---+ +---+ +---+ +---+ +---+ +---+ +---+
  10. +---------------+ | | | Large object | | | +---------------+

    | | +----------+ +----------+ +----------+ | S3 API | | S3 API | | S3 API | |----------| |----------| |----------| | | | | | | | RiakCS | | RiakCS | | RiakCS | | | | | | | +----------+ +----------+ +----------+ +---+ +---+ +---+ +---+ +---+ +---+ +---+ +---+ |1MB| |1MB| |1MB| |1MB| |1MB| |1MB| |1MB| |1MB| +---+ +---+ +---+ +---+ +---+ +---+ +---+ +---+ +------+ +------+ +------+ | | | | | | | Riak | | Riak | | Riak | | | | | | | +------+ +------+ +------+
  11. // Multi-tenant reporting * Access and storage statistics * As

    users interact with the API, Riak CS collects per- user statistics on operations
  12. // Multi-tenant reporting * Access and storage statistics * As

    users interact with the API, Riak CS collects per- user statistics on operations * Statistics are sent to an aggregation subsystem
  13. // Multi-tenant reporting * Access and storage statistics * As

    users interact with the API, Riak CS collects per- user statistics on operations * Statistics are sent to an aggregation subsystem * Aggregation subsystem periodically sends its accumulated log to be archived
  14. // Multi-tenant reporting * Access and storage statistics * As

    users interact with the API, Riak CS collects per- user statistics on operations * Statistics are sent to an aggregation subsystem * Aggregation subsystem periodically sends its accumulated log to be archived * Archival subsystem sums all recorded accesses per-user and then stores a time-sliced rollup
  15. // Access statistics * Count * BytesIn * BytesOut These

    statistics are available for operations against:
  16. // Access statistics * Count * BytesIn * BytesOut These

    statistics are available for operations against: * Buckets * Keys * ACLs
  17. { "Access": "not_requested", "Storage": [ { "StartTime": "20120316T123318Z", "EndTime": "20120316T123319Z",

    "fooBucket": { "Objects": 1, "Bytes": 3243225 }, "barBucket": { "Objects": 4, "Bytes": 234326234 } }, { "Errors": [] } ] }
  18. JSON: { "Access": "not_requested", "Storage": "not_requested" } XML: <?xml version="1.0"

    encoding="UTF-8"?> <Usage> <Access>not_requested</Access> <Storage>not_requested</Storage> </Usage>
  19. // Future work * Swift API * Keystone integration *

    S3 COPY object * S3 object versioning
  20. // Future work * Swift API * Keystone integration *

    S3 COPY object * S3 object versioning * Server-side encryption