Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Shrine - File Upload Toolkit for Ruby

Shrine - File Upload Toolkit for Ruby

Accepting file uploads is very common in web applications, but also very delicate; we have to manage the fact that it's slow, and that it comes with a lot of security implications.

Shrine is a new Ruby library for file uploads, which aims to address limitations and issues of existing libraries (CarrierWave, Paperclip, Dragonfly and Refile).

In this presentation I show how Shrine's design differs, and how it makes Shrine simple where other libraries are complex, while also making some very advanced features possible.

Janko Marohnić

March 31, 2016
Tweet

More Decks by Janko Marohnić

Other Decks in Programming

Transcript

  1. • LOC = 470 (base) + 1550 (plugins) • 1

    tiny dependency • streaming uploads & downloads • closes files and deletes tempfiles • asynchronous processing & uploads & deletes • simple and flexible design • support for Sequel • security
  2. Storage class MyStorage # PORO def upload(io, id) # ...

    end def url(id) # ... end def exists?(id) # ... end def delete(id) # ... end # ... end
  3. Shrine.storages = { disk: Shrine::Storage::FileSystem.new("uploads") } uploader = Shrine.new(:disk) uploader.storage

    #=> #<Shrine::Storage::FileSystem> uploaded_file = uploader.upload(image) uploaded_file #=> #<Shrine::UploadedFile> uploaded_file.data #=> # { # "storage" => "disk", # "id" => "9260ea09d8effd.jpg", # "metadata" => { # "filename" => "image.jpg", # "size" => 819344, # "mime_type" => "image/jpeg", # }, # } Uploader
  4. uploaded_file.url #=> "uploads/9260ea09d8effd.jpg" uploaded_file.metadata #=> {...} uploaded_file.read #=> "..." uploaded_file.exists?

    #=> true uploaded_file.download #=> #<Tempfile:/tmp/sdf94ks.jpg> uploaded_file.delete ... Uploaded File
  5. Shrine.storages = { cache: Shrine::Storage::FileSystem.new(...), store: Shrine::Storage::FileSystem.new(...), } Shrine.plugin :sequel

    # or :activerecord class ImageUploader < Shrine # ... end class User < Sequel::Model include ImageUploader[:avatar] end Attachment add_column :users, :avatar_data, :text
  6. user = User.create(avatar: File.open(“avatar.jpg")) user.avatar_data #=> "{\"storage\":\"store\",\"id\": \"sg00943klsg8dd.jpg\",\"metadata\":{...}}" user.avatar #=>

    #<Shrine::UploadedFile> user.avatar.url #=> "uploads/9dkasd920s.jpg" user.destroy user.avatar.exists? #=> false Attacher
  7. require "image_processing/mini_magick" class ImageUploader < Shrine include ImageProcessing::MiniMagick def process(io,

    context) if context[:phase] == :store resize_to_limit(io.download, 800, 800) end end end
  8. require "image_processing/mini_magick" class ImageUploader < Shrine include ImageProcessing::MiniMagick plugin :versions,

    names: [:small, :medium, :large] def process(io, context) if context[:phase] == :store size_800 = resize_to_limit(io.download, 800, 800) size_500 = resize_to_limit(size_800, 500, 500) size_300 = resize_to_limit(size_500, 300, 300) {small: size_300, medium: size_500, large: size_800} end end end
  9. user.avatar_data #=> # { # "large": { # "id": "lg043.jpg",

    # "storage": "store", # "metadata" : {"size": 94832, ...} # }, # "medium": { # "id": "kd9fk.jpg", # "storage": "store", # "metadata" : {"size": 39291, ...} # }, # "small": { # "id": "932fl.jpg", # "storage": "store", # "metadata" : {"size": 10582, ...} # } # }
  10. user.avatar.class #=> Hash user.avatar #=> # { # :large =>

    #<Shrine::UploadedFile>, # :medium => #<Shrine::UploadedFile>, # :small => #<Shrine::UploadedFile>, # } user.avatar[:large].size #=> 94832 user.avatar[:medium].size #=> 39291 user.avatar[:small].size #=> 10582
  11. Choose a file Fill in other fields Submit the form

    Wait for file to be uploaded Get redirected
  12. Choose a file Fill in other fields Submit the form

    No waiting Get redirected File uploads using AJAX
  13. # config/routes.rb Rails.application.routes.draw do mount ImageUploader::UploadEndpoint => "/images" end Shrine.plugin

    :direct_upload // with jQuery-File-Upload $('[type="file"]').fileupload({ url: '/images/cache/avatar', paramName: 'file', add: (e, data) => { /* Disable submit button */ }, progress: (e, data) => { /* Add progress bar */ }, done: (e, data) => { /* Fill hidden field */ }, }); Generic
  14. Shrine.storages = { cache: Shrine::Storage::S3.new(...), store: Shrine::Storage::S3.new(...), } Shrine.plugin :direct_upload,

    presign: true GET /images/cache/presign { "url": "https://my-bucket.s3-eu-west-1.amazonaws.com", "fields": { "key": "image.jpg", "x-amz-credential": "...", "x-amz-algorithm": "...", "x-amz-date": "...", "x-amz-signature": "..." } } Amazon S3
  15. • Processing, uploading and deleting • Graceful degradation • Supports

    any backgrounding library • Handles parallel jobs • Adaptation to changes • Debuggable failures • Security
  16. Shrine.plugin :backgrounding Shrine::Attacher.promote { |data| UploadJob.perform_async(data) } Shrine::Attacher.delete { |data|

    DeleteJob.perform_async(data) } class UploadJob include Sidekiq::Worker def perform(data) Shrine::Attacher.promote(data) end end class DeleteJob include Sidekiq::Worker def perform(data) Shrine::Attacher.delete(data) end end
  17. class ImageUploader < Shrine plugin :versions, names: [...] def process(io,

    context) case context[:phase] when :recache # process cheap versions in foreground when :store # process expensive versions in background end end end Shrine.plugin :recache
  18. Shrine.plugin :logging STORE[cache] ImageUploader[:avatar] User[29543] 1 file (0.1s) PROCESS[store]: ImageUploader[:avatar]

    User[29543] 1-3 files (0.22s) DELETE[destroy]: ImageUploader[:avatar] User[29543] 3 files (0.07s) Shrine.plugin :logging, format: :json {“action”:”upload”, “phase”:”cache”, “duration”: 0.16, ...} Shrine.plugin :logging, format: :heroku action=upload phase=cache files=1 duration=0.16 record_class=User
  19. Other plugins • determine_mime_type
 – determines actual MIME type from

    file contents • store_dimensions
 – extracts and stores image width & height • validation_helpers
 – validation of filesize, extension, MIME type, dimensions • parallelize
 – parallelizes uploading and deleting versions • … (many more)