Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How LINE Securities Integrates with External Se...

How LINE Securities Integrates with External Services

Di Ai (LINE Fukuoka / Securities Trade Dev2 / Server-Side Engineer )

https://tech-verse.me/ja/sessions/8
https://tech-verse.me/en/sessions/8
https://tech-verse.me/ko/sessions/8

Tech-Verse2022

November 18, 2022
Tweet

More Decks by Tech-Verse2022

Other Decks in Technology

Transcript

  1. Agenda - Self introduction - About LINE Securities - Project:

    Money Forward ME ☓ LINE Securities └ How the security issue is resolved. - Project: LINE app wallet tab integration └ How the test issue is resolved. - Future challenges and prospects
  2. Self Introduction 2015/04 ~ ⎯ Name: Ai Di ⎯ From

    China・Guizhou (famous for hot chili) ⎯ Got master degree from Kyushu University, Information Intelligence Engineering. ⎯ Joined LINE Fukuoka as a server-side engineer. ⎯ Living in Fukuoka. Favourite food is とんこつRamen Profile GUIZHOU FUKUOKA My dog named “Java” ⎯ Border collie is the smartest dog in the world, I think he may become a Java programmer like me.
  3. 2022/04 2021/02 2020/02 2021/06 2020/06 2019/11 About LINE Securities Timeline

    related to me LINE Securities service start Recurring fund investment service start IPO service start Project join MoneyForward ME integration start STO (security token offering) service start
  4. JSDA (Japan Securities Dealers Association) JASDEC (Japan Securities Depository Center)

    Intertrade JPX (Japan Exchange Group) Refinitiv (Renamed from Reuter) LINE pay QUICK LINE Securities Internal network Internet Buy/Sell LINE Securities server pool NRI system Price, market situation Buy/Sell Price information News Price Big data traffic, No data loss, less delay execution data Securities buy/sell records Off-exchange report
  5. +4%" ೔ຊূ݊ۀڠձ +"4%&΄;Γ ূ݊อ؅ৼସػߏ Intertrade +19 ೔ຊऔҾॴάϧʔϓ 3FGJOJUJW 3FVUFS͔Β໊ࣾมߋ LINE

    pay QUICK LINE Securities Internal network Internet Buy/Sell LINE Securities server pool NRI system Price, market situation Buy/Sell Price information News Price Big data traffic, No data loss, less delay execution data Securities buy/sell records Off-exchange report Various external services - Big transaction among systems. - Big data traffic between Refinitiv(rename from Reuter) and LINE Securities. - Combine data sources to show.
  6. Big Data Traffic Over 30K/s peak data traffic(user requests 5k/s)

    at the beginning of morning session(9:00) ⎯ Use Spring WebFlux to do non-blocking I/O processing. ⎯ Use Kafka as a hub to get data. Please refer to LINE Engineering Blog for more detail: ⎯ https://engineering.linecorp.com/ja/blog/on-the-use-of-stock-price-information-from-feeds/ No data lose, less delay
  7. About LINE Securities Current system structure, an example of getting

    securities asset LINE Securities cosmos NRI system LINE Securities microservices internal network LINE Securities front-end LINE Securities old server pool get asset info get fund/stock detail
  8. LINE Securities Cosmos Origin of name exosphere The services which

    users can touch directly, e.g., stock buy/sell API Services control the core logic and business logic, e.g., master data management system asteroid
  9. Money Forward ME Integration Code name: comet LINE Securities cosmos

    NRI system LINE Securities microservices internal network LINE Securities old server pool Money Forward ME
  10. LINE Securities Cosmos Code name: comet exosphere asteroid The part

    we are going to add. For integrating external system. comet
  11. Money Forward ME Integration Issues Fintech service common issue Issue

    1 Separation of internal network and external network The first time to expose API to external. Issue 2 The authentication and authorization
  12. Solutions to Issue 1 Separation of securities network and external

    network DMZ ngnix internal network LINE Securities cosmos NRI system LINE Securities microservices LINE Securities old server pool Money Forward ME Server
  13. internal network DeMilitarized Zone A network between public network and

    internal network, to protect the internal network. Enable Money Forward servers to access LINE securities internal network . DMZ DMZ firewall public network Comet-web Use ngnix in DMZ named comet-web. comet-web (nginx)
  14. Solutions to Issue 1 Separation of internal network and public

    network DMZ comet-web (ngnix) internal network ACL LINE Securities cosmos NRI system LINE Securities microservices LINE Securities old server pool Money Forward ME Server
  15. Plan A Create a new authentication system. Use the API

    in the existing web server. Plan B Create a new authentication system. Create an API in a new microservice. Plan C Use LINE login to authenticate. Create an API in a new microservice. Solutions to Issue 2 Authentication
  16. Why LINE Login The benefit - Security ensured. - Many

    services in LINE use LINE Login including LINE securities. - Rich document for client to implement easily.
  17. Goal To integrate external services by providing asset API. wallet-handler

    Core microservice of integration A simple and flexible scheduler and orchestrator to deploy and manage containers and non- containerized applications. Spring Boot (Kotlin) Microservice Nomad Recently most used in LINE Securities Easy for multiple teams to take responsibility. Tech stack wallet-handler get /asset
  18. Solutions to Issue 2 Authorization Provide authorization API for client

    to access via access token. curl –H “Authorization: Bearer {access_token}” ¥ -X POST ¥ 'https://{domain}/{path-to-wallet-handler}/authorize { ”result": ”OK" } We can know canceling association from LINE by calling access token verification API. curl -X GET ¥ 'https://api.line.me/oauth2/v2.1/verify?access_token =eyJhbGciOiJIUzI1NiJ9.UnQ_o-GP0VtnwDjbK0C8E_NvK...’ { "error": "invalid_request", "error_description": "access token expired" } Also auto delete authorization when user has no access for a specified period.
  19. Money Forward ME Integration System structure DMZ comet-web (ngnix) LINE

    Securities cosmos internal network ACL comet-router wallet-handler NRI system LINE Securities old server pool LINE Login Money Forward ME Server
  20. LINE app Wallet Tab Integration To become a hub to

    fintech DPOUFOUT securities asset FX asset
  21. LINE app Wallet Tab Integration Issues Less then 10 req/s

    Over 400 req/s 0 50 100 150 200 250 300 350 400 450 Issue 2 How to test with FX service while it is still in developing. Issue 1 40 times increasing in traffic securities wallet-handler FX
  22. Cache Create account info API Solutions to Issue 1 Increasing

    request amount ⎯ In Wallet tab side, cache 1 min by FX/securities. ⎯ Call account info API first to reduce unnecessary API call. Global rate limit & API rate limit Global rate limit API rate limit API rate limit FX securities wallet tab wallet-handler account info
  23. Use WireMock to mock FX API server WireMock is a

    flexible tool for building mock APIs. Solutions to Issue 2 How to test with FX service while it is still in developing securities wallet tab wallet-handler FX FX mock server
  24. Motivation Do you have same experience? - Only need to

    mock 1 API from a system. - Need to wait some APIs which is being developed. - Mock server is always not easy to build or use. ? ? ? ? ? ?
  25. Why WireMock 1 json file for 1 API. Easy to

    use We only need to mock 1 API, it means 1 json file in WireMock. Modify mock API without restarting. Dynamic management Use admin APIs to manage mock APIs. Matches many use cases. Suitable for testing For 1 API, we can use request mapping to define different response patterns.
  26. WireMock Example #1 Easy to use. Modify mock API without

    restarting. { "request": { "method": "GET", "url": "/something" }, "response": { "body": "Hello world!", "headers": { "Content-Type": "text/plain" }, "status": 200 } } > java -jar wiremock-jre8-standalone-2.34.0.jar --port 8080 > curl -X POST --data {json String} http://localhost:8080/__admin/mappings > curl -X POST http://localhost:8080/__admin/mappings/save > curl http://localhost:8080/__admin/ { "id" : "746e0a7c-84b3-4837-830a-f228ce381a41", "request" : { "url" : "/something", "method" : "GET" }, ... > curl http://localhost:8080/something Hello world! mockAPI.json
  27. WireMock Example #2 Use WireMock Studio UI 1 1. For

    1 API, use request mapping to define different response patterns.
  28. WireMock Example #2 Use WireMock Studio UI 2 1. For

    1 API, use request parameters mapping to define different response patterns. 2. Mock response status to check error case.
  29. WireMock Example #2 Use WireMock Studio UI 3 1. For

    1 API, use request parameters mapping to define different response patterns. 2. Mock response status to check error case. 3. Mock response body to test UI.
  30. WireMock Example #2 Use WireMock Studio UI 1. For 1

    API, use request parameters mapping to define different response patterns. 2. Mock response status to check error case. 3. Mock response body to test UI. 4. Proxy all requests to the real system, only simulate APIs that don't exist yet. 4
  31. wallet-handler Challenges FX securities cache How to improve rate limit?

    - Need to config rate limit by clients. - Otherwise, if one service reaches rate limit, other services will be unavailable. Where to cache and how long we can cache? - Currently only wallet tab server has cache. - When provide to more services, cache can reduce the access to NRI system. wallet tab cache
  32. Prospects Easy to expand Easily and safely add new asset

    sources as well as new clients. Built a foundation for integration with external services FX securities ??? wallet-handler
  33. Summary Money Forward ME Integration LINE App Wallet Tab Integration

    Future Challenges and Prospects About LINE Securities WireMock