Client-side JS for infeed layout native ad at fluct SSP

I told at Tokyo Node Gakuen 23: http://nodejs.connpass.com/event/42200/

Transcript

1. Client-side JS for infeed layout native ad at fluct SSP Tetsuharu OHZEKI
2. This slide is for i18N • For Japanese speaker –俺の話を聞け (Listen to me :) • For English speaker –Sorry, I’ll talk Japanese.
3. About me • Name: Tetsuharu OHZEKI – github: @saneyuki • Software Engineer of “fluct” div. in VOYAGE GROYP, Inc. – Almost coding JavaScript for advertisement system or management system. • Architecting today’s talking area. • Architecting React based internal applications. • Open Source Hacker – Mozilla committer level 3 (Servo reviewer, but inactive now) – Contributing to rxjs – or etc...
4. Today, I don’t talk about… • Adblock & future of Web • Legacy display advertisement –Or “document.write() must be died”. • You should use my proposed perfect special greatest flux architecture & framework. • Are you ok?
5. Introduction
6. Today’s online advertisement Your Device SSP (Our stance) DSP DSP DSP RTB auction Adnetwork Adnetwork Adnetwork • Manage ad request • The behavior is like a “tag manager” as warehouser
7. Infeed layout web advertisement • Construct Ad from: – Design template • html or App’s native view – Ad content • represented as json or other data format. • 2 development style – SDK style – Consulting style (our service style)
8. OpenRTB protocol • De-facto standard – Google does not use it as “Tier 1” – “At least, this is my feeling” • This protocol is defined by iAB – Definitions of response/request format. – An ad company creates super-set or sub-set spec for their system • As a platform, we should (would like to) support full features.
9. js-tracker in OpenRTB protocol • This feature enable that DSP embeds an arbitrary JavaScript. – Purpose: custom metrics by DSP (e.g. checking viewability in viewport) • Of course, this can be a vulnerable point. – DSP can inject malware script via this. – In almost case, we can fight on court. • By the protocol, DSP may use document.write()! – We need to restrict a protocol with patching! – We must kill document.write() for performance!
10. Support various ad networks • So to up filling rate, get a chance to show many ads. – So SSP’s core value is how many DSP/adnetworks a SSP connects. • What ad networks should be deliver? – Adjustment by a pre-defined data. – In almost case, RTB auction would be a top priority.
11. Current Client-Software Design
12. Data flow overview Web Page Our Script Ad network Ad network Ad network Our (SSP) ad server RTB Auction DSP
13. Data flow overview Web Page Our Script Ad network Ad network Ad network Try to request Try to request Try to request Our (SSP) ad server RTB Auction DSP If auction is failed
14. Construct JavaScript in ad server • To reduce RTT and a file size – We can response only codes that will be used “actually”. – JIT compiling in a broad sense. • Concatenate a strings is fast. – It’s slow that executing escodegen every time per request. • But simple concat string is dangerous. • Let’s imagine `var a = ${x}` • If x is ”1; alert(‘I am malware’)”…? • Expand => `var a = 1; alert(‘I am malware’);` // alert on load!
15. Construct JavaScript in ad server safely • Answer => Encode a value to JSON once. – Create the string that will be evaluate as an object literal of JavaScript. • Reimagine:`var a = ${JSON.stringify({ x })}` • Expand `var a = { “x”: “1; alert(‘I am malware’)” };` – This will be evaluated as an object on load time! • After all, we response RTB results + JS code.
16. Support full async loading • Today’s web advertisement is required “performance”. – E.g. iAB’s LEAN: https://www.iab.com/news/lean/ • We cannot use document.write() newly in 2016.
17. Support full async loading • We need to detect “where does this request comes from?” – Same script tag may be inserted into the same page. – document.currentScript is not supported by legacy Android. • Generate an unique key to a request. – The response reply it. Web Page Ad script Ad script Ad script Ad script
18. Prioritized Retry-able queue • To increase filling rate. • We’d like to fallback to the next priority if the top one does not return any ad contents on client.
19. Prioritized Retry-able queue • Two-dimentional array: Array<Array<RequestTask>> – 1d: an order to try sequential. – 2d: a group to try parallel. – Index=0 is max priority. – Items: Network Request Tasking to an ad network.
20. Prioritized Retry-able queue [ [RTB], // The result of RTB [A, B, C], // stage a: the group to try request adnetworks. [D, E, F], // stage b: start to try if all request fails in stage a. [G, H], // stage c: start to try if …. in stage b ] A is most high priority H is most lowest priority
21. Construct DOM safely • In native ad, we construct ad from design template + content. • If we does this in server side, we might cause XSS because there may be a difference with browser’s parser. • Use web browser’s html parser & DOM APIs is most reliable way.
22. Implementation
23. Implementation Style • Runtime (Supervisor) – Manage driver tasks & provide foundations. • Driver Task – Create for each ad networks. – The behavior is like a co-routine. • No use large try-catch – Async at all. – Convert to return an error value: tryBar()=>{ok, val, err,}
24. Limitations • We cannot large 3rd party libraries. – 1kb/res * 1billion request => 1Tb • So we cannot use Promise… – github:stefanpenner/es6-promise/dist/es6-promise.min.js requires 6.2kb. – We need to support Android ~4.x still now for “the long-tail”.
25. Implementations • We use TypeScript – target=ES3, modules=AMD • We write a classic async co-routine without Promise/ES6 generator. – Core Runtime: about 2000 lines – Each of drivers: almost 300 lines~ • Size (minified & minimum case): 10kb / response – This would be increase 6~8kb per driver.
26. Others implementation problem… • navigator.sendBeacon() might not work on a loading critical path. – From the spec, this behavior would be valid. • Unit tests fails in legacy Android because Android browser’s timer queue is overflowed. • Integration with DOM event loop & our co-routine’s event loop. • I can talk you about them in a networking time
27. Future Work
28. Telemetry Reporter (work in progress) • We cannot debug all environment in chaos – Network Conditions – Devices – Web browsers – Ad content – Embed Page – Our program state • Need to detect fatal errors. • Report only our errors, not embedders. • Catch only “expected” errors, not all.
29. Refactoring priority queue (still be in considered) • The current implementation may not be good abstraction. – We “had” many dreams when we designed it. – But now, we abandoned some of them…. • We might be able to make it an one-dimentional array.
30. Any questions in the networking time?