Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Server Side Swift

Yuki Takei
August 05, 2017

Serverless Server Side Swift

Yuki Takei(@noppoMan722)
Builderscon Tokyo 2017

Yuki Takei

August 05, 2017
Tweet

More Decks by Yuki Takei

Other Decks in Technology

Transcript

  1. Who am I? Yuki Takei Freelance Software Engineer • Job:

    Backend, Infrastructure, Data mining/analysis, iOS • Organizer of Tokyo Server Side Swift Meetup noppoMan noppoman722
  2. What is Swift? • Released from Apple on 2014 •

    For making app on Apple Platforms such as iOS • (Strong)static typing programming language • Swift4 will be released soon.
  3. Language Features • No Virtual Machine and Interpreter • Adopting

    Reference counting to manage memory • Safe • Faster • Expressive
  4. 1. Web Development with Swift 2. Serverless × Swift 3.

    Introduction of Hexaville 4. Create Practical Web App with Hexaville 5. Conclusion Objectives
  5. That’s mean, We can create web apps with Swift It’s

    no longer only for the Apple Platforms 1. Web Development with Swift
  6. Why do we web development with Swift? • Isomorphic development

    (iOS and Backend) • Faster/Easy to deploy (No VM and Interpreter) • Can reduce run-time errors with Strong Static Typing and Null Safety • Easy to handle parallelism and asynchronously (GCD) 1. Web Development with Swift
  7. 1. Development Environment 2. Hello World on Linux 3. Web

    Servers/Frameworks 4. About Deployment How do we web development with Swift? 1. Web Development with Swift
  8. Swift Package Manager (SPM) • SPM is a official package

    management system for Swift. • Post Carthage and CocoaPods • Use to configure application settings and manage dependencies (It’s like npm, sbt) • Also includes cross platform build system (llbuild) > 1. Development Environment 1. Web Development with Swift
  9. HexavilleBench$ swift package OVERVIEW: Perform operations on Swift packages USAGE:

    swift package [options] subcommand SUBCOMMANDS: clean Delete build artifacts describe Describe the current package dump-package Print parsed Package.swift as JSON edit Put a package in editable mode generate-xcodeproj Generates an Xcode project init Initialize a new package reset Reset the complete cache/build directory resolve Resolve package dependencies show-dependencies Print the resolved dependency graph tools-version Manipulate tools version of the current package unedit Remove a package from editable mode update Update package dependencies swift package command > 1. Development Environment 1. Web Development with Swift
  10. HexavilleBench$ swift build -h OVERVIEW: Build sources into binary products

    USAGE: swift build [options] OPTIONS: --build-path Specify build/cache directory [default: ./.build] --build-tests Build the both source and test targets --configuration, -c Build with configuration (debug|release) [default: debug] --disable-prefetching --disable-sandbox Disable using the sandbox when executing subprocesses --enable-prefetching --package-path Change working directory before any other operation --show-bin-path Print the binary output path --verbose, -v Increase verbosity of informational output -Xcc Pass flag through to all C compiler invocations -Xcxx Pass flag through to all C++ compiler invocations -Xlinker Pass flag through to all linker invocations -Xswiftc Pass flag through to all Swift compiler invocations --help Display available options > 1. Development Environment 1. Web Development with Swift swift build command
  11. Generate Initial CLI(Server) Project with SPM /Users/yuki/Desktop % mkdir HelloSPM

    && cd HelloSPM /Users/yuki/Desktop % swift package init —type executable Creating executable package: HelloSPM Creating Package.swift Creating .gitignore Creating Sources/ Creating Sources/main.swift Creating Tests/ /Users/yuki/Desktop/HelloSPM % tree . ├── Package.swift ├── Sources │ └── main.swift └── Tests 2 directories, 2 files > 1. Development Environment 1. Web Development with Swift
  12. swift build and execute yuki$ cd HelloSPM HelloSPM$ swift build

    HelloSPM$ ./.build/debug/HelloSPM Hello, world! > 1. Development Environment 1. Web Development with Swift
  13. Linux version of Swift can download from swift.org > 2.

    Hello World on Linux 1. Web Development with Swift
  14. Build and Run /Users/yuki/Desktop/HelloSPM % docker build -t hello-spm .

    ……… Removing intermediate container 2a1e07e3bb29 Successfully built 2b7cd7f22a1b Successfully tagged hello-spm: Build > 2. Hello World on Linux 1. Web Development with Swift
  15. Build and Run /Users/yuki/Desktop/HelloSPM % docker build -t hello-spm .

    ……… Removing intermediate container 2a1e07e3bb29 Successfully built 2b7cd7f22a1b Successfully tagged hello-spm: Build Run /Users/yuki/Desktop/HelloSPM % docker run -it -t hello-spm Hello, world! > 2. Hello World on Linux 1. Web Development with Swift
  16. • There are several Frameworks (Fullstack - Minimal) • HTTP

    Server is not provided in Swift Standard Lib • So almost WebFrameworks has Web Server • Covering ORM / Database Driver, JWT and Template engines etc… 3. Web Frameworks > 3. Web Frameworks 1. Web Development with Swift
  17. - Vapor - > 3. Web Frameworks 1. Web Development

    with Swift • Developer: Vapor • Type: HTTPServer and FullStack MVC Framework • GitHub Star: 10K • Documentation: Very GOOD > 3. Vapor
  18. - Kitura - 1. Web Development with Swift • Developer:

    IBM • Type: HTTPServer and Lightweight Web Framework • GitHub Star: 6K • Documentation: GOOD? > 3. Kitura
  19. > 3. Web Frameworks 1. Web Development with Swift Actually…

    I tried to create WAF for Swift a year ago…
  20. • Swift doesn’t require VM and Interpreter • Just uploading

    ELF (required runtime libraries or static link) • Working well with Ubuntu and RedHat Distributions • Working well with ECS, Kubernetes and other Container Services 4. About Deployment 1. Web Development with Swift
  21. ”Serverless computing is a cloud computing execution model in which

    the cloud provider dynamically manages the allocation of machine resources, and bills based on the actual amount of resources consumed by an application, rather than billing based on pre- purchased units of capacity.” Serverless computing from wikipedia ~Serverless computing~ 2. Serverless × Swift
  22. Advantages of Serverless Computing • Lower cost • Free from

    Server Management / Scaling • Mobile engineers can easily develop server side • Faster innovation 2. Serverless × Swift
  23. 2. Serverless × Swift This time I’ll focus on AWS's

    Lambda and API Gateway to tell you about Serverless and compatibility of Swift
  24. Lambda 2. Serverless × Swift • AWS’s Serverless Service •

    Triggered by Internal Events • Doesn’t have HTTP Endpoint
  25. 2. Serverless × Swift APIGateway • Gateway and Frontend for

    the AWS Services • Provides HTTPS Endpoint • Can choose lambda as backend service • Client -> API Gateway -> Lambda
  26. 2. Serverless × Swift Environment of AWS Lambda • Amazon

    Linux • Only supports 64Bit Binary • Supporting Languages: Node.js, Python, Java and .NET Core
  27. 2. Serverless × Swift Environment of AWS Lambda • Amazon

    Linux • Only supports 64Bit Binary • Supporting Languages: Node.js, Python, Java and .NET Core
  28. Swift code ELF Lambda Upload Build Swift works on Lambda

    with following flow 2. Serverless × Swift
  29. • Need Linux to build swift • Different development and

    work flow from conventional web development • Must execute Swift from supported languages • Anyways, It’s bother to deploy APIGateway + Lambda But… 2. Serverless × Swift
  30. Deployment Flow of Lambda + ApiGateway Code Zip S3 Lambda

    APIGateway ᶄ Upload ᶃ Archive ᶅ Create Function with S3 URL ᶆ Resource(Path) and Function Mapping ᶇ Deploy 2. Serverless × Swift ᶆ’ mapping
  31. Deployment Flow of Lambda + ApiGateway Code Zip S3 Lambda

    APIGateway ᶄ Upload ᶃ Archive ᶅ Create Function with Uploaded Zip URL ᶆ Resource and Function Mapping Request ᶇ Deploy Request Mapping Function and Resource Serverless × Swift It’s hard….
  32. I thought we can automate such bother things and focus

    to develop web apps with Swift 2. Serverless × Swift
  33. • 1. What is Hexaville? • 2. Installation • 3.

    Deploy to AWS • 4. How is Swift executed on Lambda? • 5. How to Debug? 3. Introduction of Hexaville Objectives
  34. • Framework and libraries for building serverless applications that is

    written in Swift • Currently only supports AWS APIGateway+Lambda • Don’t need to deep knowledge for AWS • Cloud Native 1. What is Hexaville? 3. Introduction of Hexaville
  35. What Hexaville solved / will solve • Fully automate deployment

    of Swift program to serverless environment (Routing, code deploy etc..) • Automatically create Swift execution environment on Lambda • Normal Development/Deployment work flow • iOS Developer can write backend and client with single language. can share code. > 1. What is Hexaville? 3. Introduction of Hexaville
  36. Hexaville has Two layers > 1. What is Hexaville? 3.

    Introduction of Hexaville Hexaville Framework (Web Application Framework) Hexaville (Command Line Interface) Your Application Code Deploy
  37. HexavilleFramework • Web Application Framework layer for Hexaville • Executed

    on Lambda • Provides Middleware and Router features • Extendable with Middlewares and plugins > 1. What is Hexaville? 3. Introduction of Hexaville
  38. /Users/yuki/WorkingsPlace/program/polyrhythm/FlieriOS % hexaville Available commands: - generate Generate initial project

    - deploy Deploy your application to the specified cloud provider - routes Show routes and endpoint for the API - help Prints this help information - version Prints the current version of this app hexaville command > 2. Installation 3. Introduction of Hexaville
  39. /Users/yuki/WorkingsPlace/program/polyrhythm/FlieriOS % hexaville Available commands: - generate Generate initial project

    - deploy Deploy your application to the specified cloud provider - routes Show routes and endpoint for the API - help Prints this help information - version Prints the current version of this app hexaville command > 2. Installation 3. Introduction of Hexaville
  40. /Users/yuki/Desktop % hexaville generate Hello /Users/yuki/Desktop % tree Hello Hello

    ├── Hexavillefile.yml ├── Package.swift └── Sources ├── RandomNumberGenerateMiddleware.swift └── main.swift 1 directory, 4 files The initial directory structure > 2. Installation 3. Introduction of Hexaville
  41. Hexavillefile.yml - Configuration file for the Hexaville Package.swift - SPM

    manifest file that have dependencies, app settings Sources/main.swift - entrypoint Sources/RandomNumberGenerateMiddleware.swift - An example code of Middleware > 2. Installation 3. Introduction of Hexaville
  42. A Project created with hexaville generate is ready to be

    deployed to lambda immediately. > 2. Installation 3. Introduction of Hexaville
  43. /Users/yuki/Desktop/Hello % hexaville deploy Hello Hexavillefile: /Users/yuki/Desktop/Hello/Hexavillefile.yml Start to build

    swift... Building application.... Generating Routing Manifest file.... Generated Routing Manifest file Sending build context to Docker daemon 351.2kB Step 1/11 : FROM ubuntu:14.04 ---> 4a2820e686c4 Step 2/11 : RUN apt-get update -y ---> Using cache ---> 00c2bd676a0a 3. Introduction of Hexaville > 3. Deploy to AWS ‘hexaville deploy’ command
  44. Deployment Flow of Lambda + ApiGateway Code Zip S3 Lambda

    APIGateway ᶄ Upload ᶃ Archive ᶅ Create Function with S3 URL ᶆ Resource(Path) and Function Mapping ᶇ Deploy ᶆ’ mapping > 4. How is Swift executed on lambda? 3. Introduction of Hexaville
  45. Deployment Flow of Lambda + ApiGateway Code Zip S3 Lambda

    APIGateway ᶄ Upload ᶃ Archive ᶅ Create Function with S3 URL ᶆ Resource(Path) and Function Mapping ᶇ Deploy ᶆ’ mapping > 4. How is Swift executed on lambda? 3. Introduction of Hexaville Totally Automated!!
  46. it could deploy swift web application in only 5 minutes

    without Signing in AWS 3. Introduction of Hexaville > 3. Deploy to AWS
  47. Hexaville ᶃ Build (call hexaville deploy) Hexaville Framework Web App

    Code Your Web App Ubuntu Deployment Flow of Hexaville > 4. How is Swift executed on lambda? 3. Introduction of Hexaville
  48. Hexaville ELF Hexaville Framework Web App Code Your Web App

    Ubuntu ᶄ Put the ELF into the Host Machine Deployment Flow of Hexaville > 4. How is Swift executed on lambda? 3. Introduction of Hexaville ᶃ Build (call hexaville deploy)
  49. Hexaville ELF - index.js - *.so ᶅArchive Hexaville Framework Web

    App Code Your Web App Ubuntu ᶄ Put the ELF into the Host Machine Deployment Flow of Hexaville Zip file > 4. How is Swift executed on lambda? 3. Introduction of Hexaville ᶃ Build (call hexaville deploy)
  50. Hexaville ELF AWS - index.js - *.so ᶅArchive ᶆ Deploy

    Hexaville Framework Web App Code Your Web App Ubuntu ᶄ Put the ELF into the Host Machine Deployment Flow of Hexaville Zip file > 4. How is Swift executed on lambda? 3. Introduction of Hexaville ᶃ Build (call hexaville deploy)
  51. Hexaville ELF ᶃ Build AWS - index.js - *.so ᶅArchive

    ᶆ Deploy Hexaville Framework Web App Code Your Web App Ubuntu ᶄ Put the ELF into the Host Machine Deployment Flow of Hexaville Zip file Introduction of Hexaville > 2. How is Swift executed on lambda?
  52. The Exposure of a trick > 4. How is Swift

    executed on lambda? 3. Introduction of Hexaville
  53. 5. How to Debug? 3. Introduction of Hexaville • Local

    Debug • Remote Debug There are two types of debugging
  54. Local Debug > 5. How to Debug? /Users/yuki/Desktop/ % cd

    Hello /Users/yuki/Desktop/Hello % swift build /Users/yuki/Desktop/Hello % ./.build/debug/Hello Available commands: - gen-routing-manif Generate routing manifest file - execute Execute the specified resource. ex. execute GET / - serve Start Hexaville Builtin Server - help Prints this help information - version Prints the current version of this app 3. Introduction of Hexaville
  55. Local Debug /Users/yuki/Desktop/ % cd Hello /Users/yuki/Desktop/Hello % swift build

    /Users/yuki/Desktop/Hello % ./.build/debug/Hello Available commands: - gen-routing-manif Generate routing manifest file - execute Execute the specified resource. ex. execute GET / - serve Start Hexaville Builtin Server - help Prints this help information - version Prints the current version of this app > 5. How to Debug? 3. Introduction of Hexaville
  56. Local Debug /Users/yuki/Desktop/Hello % ./.build/debug/Hello serve Hexaville Builtin Server started

    at 0.0.0.0:3000 Serve builtin web server > 5. How to Debug? 3. Introduction of Hexaville
  57. Local Debug Access the resource with curl /Users/yuki/Desktop/Hello % curl

    localhost:3000 <html><head><title>Hexaville</title></head><body>Welcome to Hexaville!</body></html> > 5. How to Debug? 3. Introduction of Hexaville /Users/yuki/Desktop/Hello % ./.build/debug/Hello serve Hexaville Builtin Server started at 0.0.0.0:3000 Serve builtin web server
  58. $ hexaville tail -f your-project-name Now I’m considering to implement…

    > 5. How to Debug? 3. Introduction of Hexaville
  59. Customize Error handling and Logging let app = HexavilleFramework() app.catch

    { error in print(error) // stdout will be output to Cloud Watch Logs // or // SendLog to Fluentd or something switch error { case FooError.notFound: return Response(status: .notFound) case JWTAuthenticationMiddleware.authrozationHeaderIsMissing: return Response(status: .unauthorized) default: return Response(status: .internalServerError) } } try app.run() > 5. How to Debug? 3. Introduction of Hexaville
  60. OAuth2 Overview Callback 4. Create Practical Web App with Hexaville

    TODO Page(Hexaville) login page (Hexaville)
  61. Stacks 4. Create Practical Web App with Hexaville • Hexaville:

    v0.2.0 • HexavilleFramework: v0.1.12 • Session Storage: DynamoDB • Main Data Base: DynamoDB
  62. HexavilleAuth • OAuth1 • OAuth2 Authorization Methods Authentication Methods •

    Email and Password (not implemented yet) 4. Create Practical Web App with Hexaville
  63. Supported SNS Platforms • Facebook • Github • Google •

    Instagram OAuth1 OAuth2 • Twitter 4. Create Practical Web App with Hexaville
  64. import HexavilleAuth let facebookProvider = FacebookAuthorizationProvider( path: "/auth/facebook", consumerKey: ProcessInfo.processInfo.environment[“FACEBOOK_APP_ID"]!

    ?? "", consumerSecret: ProcessInfo.processInfo.environment[“FACEBOOK_APP_SECRET"]!, callbackURL: CallbackURL(baseURL: “http://foo.bar”, path: "/auth/facebook/callback"), scope: "public_profile,email" ) { credentianl, user, req, ctx in return Response(body: "access token is: \(credentianl.accessToken)") } var auth = HexavilleAuth() auth.add(facebookProvider) app.use(session) app.use(HexavilleAuth.AuthenticationMiddleware()) app.use(auth) // Available /auth/facebook and /auth/facebook /callback app.use(auth) { request, context in print(context.isAuthenticated()) } Code Example 4. Create Practical Web App with Hexaville
  65. • DynamoDB (noppoMan) DataBase Clients 4. Create Practical Web App

    with Hexaville Using in TODO Example • Redis (IBM-Swift, Vapor) • MySQL (noppoMan, IBM-Swift) • Postgres (IBM-Swift) • MongoDB (OpenKitten) Other Available DB Clients
  66. • RDBMS and Serverless • Continuous Integration 4. Create Practical

    Web App with Hexaville Hexaville on Production > Hexaville on production
  67. • Almost developers loves RDBMS even now • Our program

    is not a daemon on Serverless • —> Can’t keep connection for RDBMS • 1 HTTP Request = 1 Connection of RDBMS • —> Your RDBMS will hang up RDBMS and Serverless 4. Create Practical Web App with Hexaville > Hexaville on production
  68. • Hexaville can easily do CI • Provides Prebuilt Binary

    and Installable just one line command • Doesn’t need to install Swift Language and Runtime • just 3 steps: git push -> test -> hexaville deploy Continuous Integration and Hexaville >Continuous Integration and Hexaville 4. Create Practical Web App with Hexaville
  69. Swift Server Performance 5. Conclusion Processed Num of Requests in

    30s 0 17500 35000 52500 70000 Prorsum(Swift 3.1) Kitura(Swift 3.1) Go 1.7 HTTP Server Node.js4.3 http 14,769 64,768 17,144 29,436 ɾwrk -d 30s -t 4 -c 20 ɾResponded with the 10 length of random JSON array ɾMachine: MacOS Sierra, 8 logical cores, 8GB RAM
  70. 5. Conclusion wrk -d 30s -t 4 -c 20 Request/sec

    0 25 50 75 100 Simple JSON Hexaville(Swift4 -O) Node.js 4.3 Node.js4.3 vs Hexaville on Lambda(512 RAM)
  71. 5. Conclusion wrk -d 30s -t 4 -c 20 Request/sec

    0 25 50 75 100 Simple JSON SHA256(1720 length string) Hexaville(Swift4 -O) Node.js 4.3 Node.js4.3 vs Hexaville on Lambda(512 RAM)