Upgrade to Pro — share decks privately, control downloads, hide ads and more …

アルプのEff独自エフェクト集 / Alp-original ’Eff’ pearls

machu
March 19, 2022

アルプのEff独自エフェクト集 / Alp-original ’Eff’ pearls

Now in its fourth year at Alp, Inc, Eff has been in continuous use since the startup's foundation.
In this talk, I will introduce our original Effects that originated from practical needs, and those we envision in the future.

machu

March 19, 2022
Tweet

More Decks by machu

Other Decks in Technology

Transcript

  1. About me > Tsubasa Matsukawa - SWE at Alp, Inc.

    @wing_007 ࣗݾ঺հ @ma2k8 https://ma2k8.hateblo.jp/
  2. Our Products > Scalebase This is software for launch up

    and managing and revenue management their subscription- based services. ΞϧϓͰ͸4DBMFCBTFͱ͍͏αϒεΫϦϓγϣϯ؅ཧɺܦӦ෼ ੳͷ4BB4ϓϩμΫτΛఏڙ͍ͯ͠·͢
  3. ಠࣗఆٛ& ff FDUͷϝϦοτͱͯ͠ɺࡉ͔͍&SSPS)BOEMJOH΍ɺ & ff FDUͷӨڹൣғͷہॴԽͳͲ͕͋Γ·͕͢ɾɾɾ > Detailed error handling

    for each effect. > Modularity of effect by ‘Eff’. > Split expression and execution pros of proprietary definition effects
  4. I'll explain the problems that can arise when you stack

    too many effects in ‘Eff'. & ff Ͱ͸ޮՌΛͨ͘͞ΜੵΉͱ ໰୊ʹͳΔ఺͕͋Γ·͢
  5. First, ‘Eff’ separates the definition of an expression from its

    execution. ·ͣલఏͱͯ͠ɺ& ff Ͱ͸ࣜͷఆٛͱ࣮ߦ͕ ෼͔Ε͍ͯΔͱ͍͏ੑ࣭͕͋Γ·͢
  6. When executing an Effect-A, you may want to use the

    properties of another Effect-B. *OUFSQSFUFSʹΑΔ& ff FDUͷ࣮ߦ࣌ʹɺ ผͷ& ff FDUͷಛੑΛར༻ͨ͘͠ͳΔ͜ͱ͕Α͋͘Γ·͢ Effect-A Use Effect-B
  7. ͋Δ& ff FDU"͕& ff FDU#ͷੑ࣭΋ར༻͢Δ৔߹ɺ & ff FDU4UBDL͔ΒऔΓग़͢ॱ൪͕ݻఆ͞Ε·͢ When an

    effect-A also uses effect-B, the order of extraction from the EffectStack is fixed. Effect-Stack Effect-A Effect-B Deps Effect-A Interpreter Effect-B Interpreter 1. Pick effect-A 2. Run effect-A and use effect-B 3. Stack effet-B 4. Pick effect-B 5. Run effect-B
  8. This constraint becomes more complex as the number of effects

    increases. ͜ͷ੍໿͸& ff FDUͷ਺͕ଟ͘ͳΔʹͭΕ΍΍͘͜͠ͳΓɺ ৗʹ& ff FDUͷґଘؔ܎Λҙࣝͯ͠ίʔυΛॻ͘ඞཁ͕͋Γ·͢
  9. Alp, Inc. addresses this problem by always running all effects.

    ΞϧϓͰ͸ৗʹશͯͷF ff FDUΛSVO͢Δख๏Ͱ ͜ͷ഑ઢ໰୊Λରॲ͍ͯ͠·͢
  10. ೉͍͠ͷ͸ࣜΛධՁ͢Δॱ൪Ͱ͢ Effect-A Effect-B Effect-C The hard part is the order

    in which the 'expressions' are evaluated. Effect-B Effect-C Effect-C Run A Run B
  11. SVOBMMͰ͸ࣜʹશͯͷ& ff FDU͕ೖͬͨલఏʹ͢Δ͜ͱͰɺ ࣮ߦͷॱ൪Λݻఆ͍ͯ͠·͢ Effect-A Effect-C In run-all, all effects

    are assumed to be included in the expression. The order of execution is fixed. Run A Effect-C Run B Effect-C B doesn't exist, but it runs. Expression without e ff ect B
  12. & ff FDU4UBDLͷਪҠΛ໌ࣔతʹॻ͘͜ͱͰ࣮ݱ͍ͯ͠·͢ Write the EffectStack transitions explicitly. implicit class

    AllOps[R, A](effects: Eff[R, A]) { def runAll[U1, U2, U3, U4, U5, U6, U7, U8, U9, U10](withTx: Boolean)( implicit clockmAux: Member.Aux[ClockM, R, U1], idGenmAux: Member.Aux[IdGenM, U1, U2], authzIOAux: Member.Aux[AuthzIO, U2, U3], authzIOMs: _authzScope[U3], authzIOM1: _ppErrorEither[U3], authzIOM2: _task[U3], auditLogMAux: Member.Aux[AuditLogM, U3, U4], auditLogM1: _task[U4], auditLogM2: _ppErrorEither[U4], auditLogM3: _kfirehoseio[U4], kinesisFirehoseIOAux: Member.Aux[KinesisFirehoseIO, U4, U5], kinesisFirehoseIOM1: _task[U5], kinesisFirehoseIOM2: _ppErrorEither[U5], ɹ ……
  13. ͭΤϑΣΫτ͕ੵ·Ε͍ͯΔࣜΛɺͭͷSVOͰ࣮ߦ͢ΔKVTU ͱͭͷSVOͰ࣮ߦ͢ΔBMMͰϕϯνϚʔΫΛऔΓ·ͨ͠ Benchmark def justBench() = { type R =

    Fx.fx3[Task, IdGenM, PpErrorEither] def program[R: _task: _idgenm: _ppErrorEither]: Eff[R, Unit] = { for { _ <- fromTask[R, Unit](Task.delay((): Unit)) _ <- OperatorId.generate[R] _ <- fromPpError[R, Unit](Right((): Unit)) } yield () } val s: Scheduler = Scheduler.global Await.result(program[R].runIdGenM().runPpError.runAsync.runToFuture(s), Duration.Inf) } def allBench() = { type R = Fx.fx6[Task, IdGenM, ClockM, PpErrorEither, Option, List] def program[R: _task: _idgenm: _ppErrorEither: _option: _list]: Eff[R, Unit] = { for { _ <- fromTask[R, Unit](Task.delay((): Unit)) _ <- OperatorId.generate[R] _ <- fromPpError[R, Unit](Right((): Unit)) } yield () } val s: Scheduler = Scheduler.global Await.result(program[R].runIdGenM().runClockM().runOption.runList.runPpError.runAsync.runToFuture(s), Duration.Inf) } •Benchmarked an expression with 3 effects stacked •'justBench' with 3 runs •‘allBench' with 6 runs
  14. [info] Benchmark Mode Cnt Score Error Units [info] AllEffectBench.all sample

    1091700 3.557 ± 0.045 us/op [info] AllEffectBench.all:all·p0.00 sample 3.000 us/op [info] AllEffectBench.all:all·p0.50 sample 3.208 us/op [info] AllEffectBench.all:all·p0.90 sample 3.540 us/op [info] AllEffectBench.all:all·p0.95 sample 3.748 us/op [info] AllEffectBench.all:all·p0.99 sample 4.872 us/op [info] AllEffectBench.all:all·p0.999 sample 9.408 us/op [info] AllEffectBench.all:all·p0.9999 sample 844.452 us/op [info] AllEffectBench.all:all·p1.00 sample 1859.584 us/op [info] AllEffectBench.just sample 822171 2.677 ± 0.150 us/op [info] AllEffectBench.just:just·p0.00 sample 1.790 us/op [info] AllEffectBench.just:just·p0.50 sample 1.958 us/op [info] AllEffectBench.just:just·p0.90 sample 2.164 us/op [info] AllEffectBench.just:just·p0.95 sample 2.416 us/op [info] AllEffectBench.just:just·p0.99 sample 7.288 us/op [info] AllEffectBench.just:just·p0.999 sample 65.386 us/op [info] AllEffectBench.just:just·p0.9999 sample 913.987 us/op [info] AllEffectBench.just:just·p1.00 sample 18612.224 us/op Φʔόʔϔου͸খ͍͞ Little overhead Small overhead because the interpreter is not processed, only the effects are extracted
  15. CryptoM > execute crypto & decrypt. > use library ‘tink’.

    > use kms to store the crypto key. ҉߸Խɺ෮߸Λߦ͏& ff FDUͰ͢
  16. AuditLogM > It is always evaluated at the end of

    the process. > Publish system audit log to s3 via kinesis. > Use the SafeEffect of atnos-eff. ɹ> Since we also need to log processing failures, we can use SafeEffect, to run "finally processing”. γεςϜ؂ࠪϩάͷ& ff FDUͰ͢ ,JOFTJTܦ༝Ͱ4΁อଘ͠·͢
  17. TransactionTask > Manage DB Access & Transaction. > Use Scalikejdbc.

    > Extracts the Either effect from the EffectStack and controls the rollback if Left is included. > There is dryrun mechanism that performs a rollback after the query is executed. %#"DDFTT 5SBOTBDUJPOΛ؅ཧ͠·͢ &JIUFSͷ݁ՌΛ֬ೝͯ͠SPMMCBDL͢Δػೳ΍ɺESZSVO͢Δ࢓૊Έ͕͋Γ·͢ Case-study materials: https://speakerdeck.com/kaelaela/api-design-by-clean-architecture-and-effɹ
  18. Authorization is complex and coverage is wide-ranging Presenter Controller Repository(DB

    etc..) UseCase Domain Masking item Execute endpoint Filter resource read/write auhorization Execute UseCase Ramification domainLogic Execute domainLogic ೝՄ͸ద༻ൣғ͕޿͘ɺཁ݅࣍ୈͰ͸1SFTFOUFSd%PNBJO·Ͱ Өڹ͠͏Δɺͱͯ΋ෳࡶͳ֓೦Ͱ͢
  19. Here is a good implementation that takes advantage of ’Eff’

    property of separating expression and execution. ೝՄͷ࢓૊ΈΛ& ff ͷࣜͱ࣮ߦΛ෼཭͢Δੑ࣭Λར༻͠ɺ ྑ͍ײ͡ʹ࣮૷Ͱ͖ͨͷͰ͝঺հ͠·͢
  20. Principal? Resource? 1SJODJQBMೝՄର৅ 3FTPVSDFೝՄର৅ͷૢ࡞ʹӨڹ͢Δର৅ > Principal > Authorization target ex.

    User, AcessToken etc.. > Resource > object that affect principal operation. ex.Repository,domainLogic,UseCase etc.. > What is treated as Principal may also be treated as Resource.
  21. Policy ? Scope ? 1PMJDZ͸1SJODJQBMʹ෇༩͞ΕΔೝՄଐੑ 4DPQF͸3FTPVSDFʹ෇༩͞ΕΔೝՄ৘ใ > Policy - Authorization

    attribute assigned to Principal - In addition to authorization information, it has resoruce id and other information used for id-based filters > Scope - Authorization attribute assigned to Resource.
  22. Resource exists in various places, making it difficult to handle

    in a unified handling. ೝՄͷର৅Ͱ͋Δ3FTPVSDFͷ4DPQF͕͞·͟·ͳ৔ॴʹ ଘࡏ͢ΔͨΊɺ౷ҰతͳϋϯυϦϯά͕೉͍͠໰୊͕͋Γ·͢
  23. State monad είʔϓ͸ࢦఆͷܕʹแΊ͹ ࣗಈͰΤϑΣΫτελοΫʹੵ·ΕΔΑ͏ʹ͠·ͨ͠ 
 TDBMB fi YͰܕͷࢦఆ͕࿙Ε͍ͯΕ͹ίϯύΠϧΤϥʔʹ͠·͢ > By

    setting the return value of the interface to the specified type, the scope is automatically loaded onto the effects stack. > If the type specification is omitted, ’scalafix’ will generate a compile error.
  24. State monad ͜ΕʹΑ֤ͬͯ૚ʹލͬͨೝՄείʔϓΛ ͭͷ4UBUFͰ؅ཧͰ͖·͢ > This allows for the management

    of authorization scopes across all tiers in a single State. Presenter Controller Repository(DB etc..) UseCase Domain Set Scope A Set Scope B Set Scope C,D Set Scope E Set Scope F State[List[A,B,C,D,E,F], X]
  25. Enforce authorization checks at interpreter execute & ff FDUΛΠϯλϓϦλʹ౉͢खલͰɺ 1SJODJQBMͷݖݶϦΫΤετΛڬΈ·͢

    Before executing your own effects Interrupt the authorization request of the Principal
  26. Enforce authorization flow ೝՄͷద༻ϑϩʔ 1. Expression Assembly Phase Define a

    command to set the required scope to State at any point in the code base. You will have an expression with the scope you need to execute it. 2. Call interpreter Phase 3. Run interpreter Phase program.run() authzInterpreter.run( program <<requestAuthzScope(principalId) ) val program: Eff[R, A] Stack State[List[ActionCompose], A] & others(DB,Task,Either, etc..) Passes an expression with scope to the interpreter. Incorporate authorization request commands when passing. This allows us to enforce the authorization request when obtaining ’Eff[R,A] => A’. When the ‘requestAuthzScope' command is evaluated, a allow/ deny decision is made based on the scope set and permissions associated with the principal. If the decision is "deny", an Either.left is added to the effect stack and the computation is stopped. for { state <- get[U, List[ActionComposing]] attachedPolicy <- showPolicy(principalId) attachedAction = attachedPolicy.policies.map(_.action) …
  27. ͜ΕΒͷϝϦοτΛڗडͰ͖ΔΑ͏ʹͳΓ·ͨ͠ Pros > Scopes set up in various places can

    be collected by State monad. > Can create a situation where an authorization check is mandatory to evaluate an expression. > By placing the permission check before execution, it is no longer necessary to separate implementations based on whether permission checks are required or not.
  28. Compensation Transaction > Implementing the ‘Saga Pattern’. > Could be

    used to notify multiple elements of a transaction status update. ิঈτϥϯβΫγϣϯ͸& ff FDUͱͷ૬ੑ͕ྑͦ͞͏Ͱ͢
  29. Akka-Stream > Use in paging control. > Use for chunk

    control in batch. "LLB4USFBNͷ& ff FDU͸ ϖʔδϯάॲཧ΍όονͷνϟϯΫ੍ޚͳͲʹ࢖͑ͦ͏Ͱ͢
  30. Macro-string > Macro-string generation and parsing effects are being implemented.

    > Provide users with a way to embed expressions and variables > Unlike the effects introduced so far, these effects are closer to the domain. Ϣʔβʔ͕จࣈྻʹࣜ΍ม਺ΛຒΊࠐΊΔػೳͷఏڙ΋ 
 & ff FDUͰߦ͏Α͏։ൃதͰ͢
  31. Logic switch per provider > If the SaaS needs to

    behave slightly differently for a particular customer, the interpreter can be used to switch to a specific logic. 4BB4ͷ༷ͳαʔϏεܗଶͰ͸ɺސ٬ຖͷݻ༗ϩδοΫͷ࣮ݱ ͸೉͍͕͠ɺΠϯλϓϦλ੾Γସ͑Ͱ͋Δఔ౓ରԠͰ͖Δ