Share
Embed
Looking for a silver bullet to edit videos on Android.
Seungwon Lee
LINE Plus AppDev5 Android Engineer
https://linedevday.linecorp.com/jp/2019/sessions/F2-2
LINE DevDay 2019
November 21, 2019
Transcript
135 slides
-
2019 DevDay Looking for a Silver Bullet To Edit Videos
on Android. > Seungwon Lee > LINE Plus AppDev5 Android Engineer -
Agenda > Why Do We Need a Silver Bullet? >
What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting -
Agenda > Why Do We Need a Silver Bullet? >
What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting -
Blacklist > List of devices that had problems during video
processing. Device A Device B Device C Device D … -
Blacklist > Blacklisted devices were redirected so that client-side video
processing would not occur Bypass Device A Device B Device C Device D … -
Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R****
**** * *H*-E*** V**-* *-0** -
Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R****
**** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H -
Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R****
**** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0 -
Android Wars: A New Hope {Episode 4) > android-transcoder made
by Yuya Tanaka > https://github.com/ypresto/android-transcoder > Written with pure Java without codec licensing issues. > Supports OpenGL renderer -
Android Wars: A New Hope {Episode 4) Legacy Video Processor
in LINE android- transcoder *A**** P* X O *A**** H***** X O *A**** P* X O *A**** H***** X X *A**** M***** O O > android-transcoder made by Yuya Tanaka > https://github.com/ypresto/android-transcoder > Written with pure Java without codec licensing issues. > Supports OpenGL renderer -
Agenda > Why Do We Need a Silver Bullet? >
What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting -
Audio Track (AAC) Video Track (H.264) MP4 Container Video Format(Container)
Reference: - https://en.wikipedia.org/wiki/Video_file_format - https://en.wikipedia.org/wiki/Codec -
Vorbis Audio VP8 Video 3GP Container Friend A Mine Transcoding
Vorbis, VP8 (X) AAC, H.264 (O) -
AAC Audio H.264 Video MP4 Container Vorbis Audio VP8 Video
3GP Container Encoder (AAC) Decoder (Vorbis) Friend A Mine Transcoding Encoder (H.264) Decoder (VP8) Vorbis, VP8 (X) AAC, H.264 (O) -
Agenda > Why Do We Need a Silver Bullet? >
What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting -
android-transcoder Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer
Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder -
android-transcoder Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer
Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder -
android-transcoder Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer
Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Reference: https://github.com/ypresto/android-transcoder -
android-transcoder Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer
Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Reference: https://github.com/ypresto/android-transcoder -
How to Extract/Decode a Video Extractor android.media.MediaExtractor Audio Decoder Video
Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder -
> Query the metadata of each track > Read the
encoded data stream > Demux data streams into each track android.media.MediaExtractor Media Extractor Audio info: AAC Video info: H.264 Audio Track (AAC) Video Track (H.264) MP4 Container -
> Query the metadata of each track > Read the
encoded data stream > Demux data streams into each track android.media.MediaExtractor Media Extractor Audio info: AAC Video info: H.264 Audio Track (AAC) Video Track (H.264) MP4 Container Audio Decoder Video Decoder -
How to Extract Video and Audio Media Extractor Time Audio
Track (AAC) Video Track (H.264) MP4 Container val size = mediaExtractor.readSampleData(byteBuf, offset) val presentationTimeUs = mediaExtractor.getSampleTime() -
How to Extract Video and Audio Data Media Extractor mediaExtractor.advance()
val size = mediaExtractor.readSampleData(byteBuf, offset) End of Stream Time Audio Track (AAC) Video Track (H.264) MP4 Container End of Stream … … -
How to Decode a Video Audio Decoder Video Decoder Audio
Encoder Video Encoder Muxer Video OpenGL Renderer Extractor android.media.MediaCodec Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder -
How to Decode a Video > Wrapper class for media
codecs (i.e. encoder and decoder) > Use together with android.media.MediaExtractor, android.media.MediaMuxer AAC Audio Decoder H.264 Video Decoder -
How to Extract/Decode a Video Extracting, Decoding Rendering Encoding, Muxing
Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder -
How to Extract/Decode a Video Media Extractor Audio Track (AAC)
Video Track (H.264) MP4 Container … -
How to Extract/Decode a Video Video Decoder Media Extractor Audio
Track (AAC) Video Track (H.264) MP4 Container -
How to Extract/Decode a Video Video Decoder Media Extractor Audio
Track (AAC) Video Track (H.264) MP4 Container Empty Input Buffer -
How to Extract/Decode a Video Video Decoder Media Extractor Audio
Track (AAC) Video Track (H.264) MP4 Container Empty Input Buffer Fill Input Buffer -
How to Extract/Decode a Video Video Decoder Output Surface Media
Extractor Audio Track (AAC) Video Track (H.264) MP4 Container Empty Input Buffer Fill Input Buffer Video Renderer -
How to Extract/Decode a Video Video Decoder Output Surface Media
Extractor Audio Track (AAC) Video Track (H.264) MP4 Container Video Renderer onFrameAvailable Empty Input Buffer Fill Input Buffer -
How to Extract/Decode a Video Video Decoder Output Surface Media
Extractor Audio Track (AAC) Video Track (H.264) MP4 Container Video Renderer onFrameAvailable Released Buffer in Surface Empty Input Buffer Fill Input Buffer -
Rendering Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer
Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Reference: https://github.com/ypresto/android-transcoder -
How to Render a Video Audio Decoder Video Decoder Audio
Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Output Surface Input Surface Frame Buffer Reference: https://github.com/ypresto/android-transcoder -
How to Render a Video Audio Decoder Video Decoder Audio
Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Output Surface Input Surface Frame Buffer Render Reference: https://github.com/ypresto/android-transcoder -
How to Render a Video Audio Decoder Video Decoder Audio
Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Output Surface Input Surface Frame Buffer Render Render Reference: https://github.com/ypresto/android-transcoder -
How to Render a Video > OutputSurface and InputSurface will
be bound with the current OpenGL environment Output Surface from Decoder FrameBuffer Input Surface to Encoder -
How to Render a Video > Pixel data of a
video frame is drawn into OutputSurface of the decoder 720 X 1280 Output Surface from Decoder FrameBuffer Input Surface to Encoder -
How to Render a Video Output Surface from Decoder FrameBuffer
Input Surface to Encoder > We can reduce a video frame by drawing it on FrameBuffer in OpenGL 720 X 1280 540 X 960 -
How to Render a Video > We pass the resized
frame to InputSurface of the encoder 540 X 960 720 X 1280 540 X 960 Output Surface from Decoder FrameBuffer Input Surface to Encoder -
How to Encode/Mux a Video Audio Decoder Video Decoder Audio
Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Extracting, Decoding Rendering Encoding, Muxing Reference: https://github.com/ypresto/android-transcoder -
Output Buffer How to Extract/Decode a Video Video OpenGL Renderer
Request Encoding Input Surface Video Encoder -
Muxer How to Extract/Decode a Video Video OpenGL Renderer Request
Encoding Audio Track Video Track Input Surface Output Buffer Release Output Buffer Video Encoder -
How to Encode/Mux a Video Muxer android.media.MediaMuxer Audio Decoder Video
Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder -
android.media.MediaMuxer > Mux encoded streams of audio/video > Supports MP4,
WebM and 3GP files AAC Encoder H.264 Encoder Media Muxer Audio Track (AAC) Video Track (H.264) Video Container Reference: - https://developer.android.com/reference/android/media/MediaMuxer - https://developer.android.com/guide/topics/media/media-formats -
android.media.MediaMuxer Media Muxer mediaMuxer.writeSampleData(trackIndex, byteBuf, bufferInfo) -> Audio Track (AAC)
Video Track (H.264) Video Container Time -
android.media.MediaMuxer Media Muxer mediaMuxer.writeSampleData(trackIndex, byteBuf, bufferInfo) -> Audio Track (AAC)
Video Track (H.264) Video Container End of stream Time End of stream … … -
How Does android-transcoder Work Audio Decoder Video Decoder Audio Encoder
Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container Reference: https://github.com/ypresto/android-transcoder -
Audio Features Didn’t Exist Audio sampler Audio Decoder Video Decoder
Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container -
Overlaying Stickers Didn’t Exist Audio sampler Overlay Sticker Audio Decoder
Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container -
Agenda > Why Do We Need a Silver Bullet? >
What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting -
Added Features to Cruiser v1.0 Container MP4 Video Codec H.264
Bitrate 2 Mbit/s Resolution 720*1280 Trim range 1200 ms ~ 250300 ms Audio Codec AAC Sample Rate 44.1 kHz ... ... > Adjust sampling rate of audio > Add APIs for a trancoding policy including codec, resolution, trim ranges, and etc. -
Audio Sampler Audio Sampler > Adjust sampling rate of audio
Audio Decoder Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container -
Audio Sampler Audio Sampler AAC Decoder AAC Encoder > Based
on libsamplerate made by erikd > https://github.com/erikd/libsamplerate > A library for performing sample rate conversion of audio data 320 kbit/s 96 kbit/s -
Cruiser v1.0 Features > Adjusts video size and sampling audio
with a simple policy for transcoding > Trims video Container MP4 Video Codec H.264 Bitrate 2 Mbit/s Resolution 720*1280 Trim range 1200 ms ~ 250300 ms Audio Codec AAC Samplerate 44.1 kHz -
Cruiser v1.0 Features > Adjusts video size and sampling audio
with a simple policy for transcoding > Trims video > Mutes audio X Container MP4 Video Codec H.264 Bitrate 2 Mbit/s Resolution 720*1280 Trim range 1200 ms ~ 250300 ms Audio Codec AAC Samplerate 44.1 kHz -
Cruiser v1.1 Main Feature Audio sampler Overlay Sticker Audio Decoder
Video Decoder Audio Encoder Video Encoder Muxer Video OpenGL Renderer Extractor Audio Track Video Track Video Container Audio Track Video Track Video Container -
How Does Cruiser Overlay A Sticker > OutputSurface and InputSurface
will bind with the current OpenGL environment Output Surface from Decoder FrameBuffer Input Surface to Encoder -
How Does Cruiser Overlay A Sticker > Pixel data of
a video frame is copied into OutputSurface of the decoder Output Surface from Decoder FrameBuffer Input Surface to Encoder -
How Does Cruiser Overlay A Sticker > We load a
sticker as a Bitmap instance and replace it to a texture that OpenGL can recognize Teuxture Output Surface from Decoder FrameBuffer Input Surface to Encoder -
How Does Cruiser Overlay A Sticker > We can draw
additionally overlay a sticker on FrameBuffer in OpenGL Output Surface from Decoder FrameBuffer Input Surface to Encoder -
How Does Cruiser Overlay A Sticker > We pass this
frame to InputSurface of the encoder Output Surface from Decoder FrameBuffer Input Surface to Encoder -
How Does Cruiser Overlay A Sticker > Like this, We
can overlay serval stickers on a video frame Output Surface from Decoder FrameBuffer Input Surface to Encoder -
Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R****
**** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0 -
*H*-E*** *H*-0** S*-0** S*-0** R**** **** * R**** **** *
*H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0 Blacklist On Cruiser X -
*H*-E*** *H*-0** S*-0** S*-0** R**** **** * R**** **** *
*H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0 Blacklist On Cruiser X ! -
Agenda > Why Do We Need a Silver Bullet? >
What Should We Know in Advance? > How Does android-transcoder Work? > What is Cruiser? > Troubleshooting -
Troubleshooting Video issues Quality issue Orientation issue Audio issue 0
bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue -
Troubleshooting Video Issues > Quality Issue Orientation Issue Audio Issue
0 bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue -
A Video Profile Configuration Video Encoder Video Encoder Cruiser Device
Camera High Profile Video High Profile Video -
A Video Profile Configuration Video Encoder Video Encoder Cruiser Device
Camera High Profile Video Base Profile Video -
Low Quality Due to Video Profiles Video Encoder Cruiser High
Profile Video > Don’t leave a device to choose a video profile High Profile Video Encoder Cruiser Base Profile Video Base Profile -
Troubleshooting Video Issues Quality Issue > Orientation Issue Audio Issue
0 bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue -
Orientation Setting to Encoder Video Encoder Muxer 90 Orientation =
90 Extractor 90 90 -
Orientation Setting to Encoder X 90 Ignored Video Encoder Muxer
90 Orientation = 90 Extractor 90 -
Video Rotated Issue 90 Video Encoder Muxer 90 Orientation =
90 Extractor 90 -
Video Rotated Issue > Don’t set the orientation value of
a video to an encoder Video Encoder Muxer 90 Orientation = 90 Extractor 90 X -
Troubleshooting Video Issues Quality Issue Orientation Issue Audio Issue >
0 bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue -
0 bit/s Issue Extract the bitrate Audio bitrate = 128
kbit/s Extractor 128 kbit/s -
0 bit/s Issue 128 kbit/s 96 kbit/s 96 kbit/s Audio
Format Video Format Container Audio Decoder Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format 96 kbit/s 48 kbit/s 48 kbit/s 48 kbit/s 48 kbit/s Audio Codec AAC Sample Rate 44.1 kHz Bitrate 96 kbit/s -
0 bit/s Issue Audio bitrate = 128 kbit/s Extract the
bitrate Extractor 0 kbit/s -
0 bit/s Issue > Don’t forget to check 0 values
0 kbit/s 0 kbit/s Audio Format Video Format Container Audio Decoder Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format 0 kbit/s 0 kbit/s -
0 bit/s Issue > Don’t forget to check 0 values
0 kbit/s 96 kbit/s 96 kbit/s Audio Format Video Format Container Audio Decoder Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format 96 kbit/s -
Troubleshooting Video Issues Quality Issue Orientation Issue Audio Issue 0
bit/s Issue > Out of Sync Audio and Video Issue Audio Distortion Issue -
Waiting of Muxer Initialization Muxer Audio Track Video Track Output
Buffer Video Encoder -
Waiting of Muxer Initialization Muxer Audio Track Video Track Output
Buffer Release Output Buffer Video Encoder -
Waiting of Muxer Initialization Muxer Audio Track Video Track Output
Buffer Release Output Buffer Video Encoder -
Waiting of Muxer Initialization Muxer Audio Track Video Track Output
Buffer Release Output Buffer Video Encoder -
Waiting of Muxer Initialization Muxer Audio Track Video Track Output
Buffer Video Encoder X MUXER_STATE_UNINITIALIZED -
Waiting of Muxer Initialization Muxer Audio Track Video Track Queue
Video Encoder Output Buffer Release Output Buffer Shallow copy MUXER_STATE_UNINITIALIZED -
Waiting of Muxer Initialization Muxer Audio Track Video Track Video
Encoder MUXER_STATE_STARTED Output Buffer Release Output buffer Pass Output Buffer Queue -
Waiting of Muxer Initialization Muxer Audio Track Video Track Queue
Video Encoder MUXER_STATE_STARTED Output Buffer Release Output buffer -
Waiting of Muxer Initialization Muxer Audio Track Video Track Video
Encoder MUXER_STATE_STARTED Output Buffer Release Output buffer Pass Output Buffer Queue -
Out of Sync Issue Muxer Audio Track Video Track Pass
Output Buffer Queue Video Encoder MUXER_STATE_STARTED Output Buffer Release Output buffer Shallow copy -
Out of Sync Issue Muxer Audio Track Video Track Pass
Output Buffer Queue Video Encoder MUXER_STATE_STARTED Output Buffer Release Output Buffer Deep copy > Don’t rely on release timing of the device -
Troubleshooting Video Issues Quality Issue Orientation Issue Audio Issue 0
bit/s Issue Out of Sync Audio and Video Issue > Audio Distortion Issue -
Audio Distortion Problem After Trimming Extract the bitrate File Analysis
123 kbit/s Extract the bitrate Extractor 24 kbit/s -
Audio Distortion Problem Audio Format Video Format Container Audio Decoder
Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format > We transcoded the audio track and resampled the ByteBuffer of the audio > We don’t need to resample this problematic audio track 24 kbit/s (123 kbit/s) 24 kbit/s (123 kbit/s) 24 kbit/s (123 kbit/s) 24 kbit/s (123 kbit/s) -
Audio Distortion Problem > Cruiser can detect this issue by
extracting 24 kbit/s > Cruiser bypasses the resampling process if it has 24 kbit/s Audio Format Video Format Container Audio Decoder Audio Encoder Muxer Extractor Audio Down Sampler Audio Format Container Video Format X X X Audio Stream Bypass 24 kbit/s (123 kbit/s) 24 kbit/s (123 kbit/s) -
Troubleshooting Video issues Quality issue Orientation issue Audio issue 0
bit/s Issue Out of Sync Audio and Video Issue Audio Distortion Issue -
Blacklisted Devices *H*-E*** *H*-0** S*-0** S*-0** R**** **** * R****
**** * *H*-E*** V**-* *-0** 2***** *-0** *-0** E**-* E**-* F**-L** P*** *L*-* V**-* W**-* *T*-A* *L*-* V**-* ***0* M** **V-***9 **V-**9 **e*** *R *C-**F C**** C**** *R*-U*** *R*-**9 D**-***L *i*** * **2-***Y ****H **F-***0 **F-***0 **F-**9 **F-***0 **A_***0 **R-***0 **Y-*L **R-***0 **E-***0 **E-***2 **R-***0 **Y-* *E-***0 **A-**1 **A-**1 ****W *7-**0 ****W **V-***L **E-**2 **E-**4 **E-**1 **E-**3 **E-***0 X -
Cruiser v1.2 features > Make a video with a static
image Cruiser v1.3 features > Crop and rotate a video What‘s Next? Cruiser v2.0 features > Each component runs on multiple threads