Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
EZIO, Torrent-based Recovery
Search
Date Huang
May 01, 2017
Programming
0
210
EZIO, Torrent-based Recovery
Date Huang
May 01, 2017
Tweet
Share
More Decks by Date Huang
See All by Date Huang
從大學到出社會,這十年的參與和維護的開源專案
tjjh89017
0
1k
開源專案的商業困境
tjjh89017
0
260
Rapidly Deploy NFV with VyOS on Kubernetes
tjjh89017
0
33
設計Kubernetes Controller與CRD的實踐 - 以網路為例
tjjh89017
0
840
維護自己的開源專案EZIO的心路歷程
tjjh89017
1
470
Massive Bare-Metal Operating System Provisioning Improvement - OpenInfra Day Taiwan 2019/11/12
tjjh89017
0
73
Cloud Infrastructure Interconnect with Wireguard and OSPF
tjjh89017
0
450
Build Redundant Gaming Network with Wireguard and BGP
tjjh89017
1
2.3k
CI on OpenStack
tjjh89017
0
280
Other Decks in Programming
See All in Programming
◯◯エンジニアになった理由
gessy0129
PRO
0
610
ECS向けのドリフト検知機構を実装してみた
tkikuc
0
270
[PHPカンファレンス沖縄2024]「無理なくできるだけ安全に」テストもないレガシーコードをリファクタリングするテクニック
ikezoemakoto
3
110
文化が生産性を作る
jimpei
3
500
Flutterアプリを生成AIで生成する勘所
rizumita
0
250
複数プロダクトの技術改善・クラウド移行に向き合うチームのフレキシブルなペア・モブプログラミングの実践 / Flexible Pair Programming And Mob Programming
honyanya
0
150
Removing Corepack
yosuke_furukawa
PRO
9
1.1k
AWS Lambda Web Adapterを活用する新しいサーバーレスの実装パターン
tmokmss
6
5.5k
自分だけの世界を創るクリエイティブコーディング / Creative Coding: Creating Your Own World
chobishiba
2
320
sqlcを利用してsqlに型付けを
kamiyam
0
230
A Journey of Contribution and Collaboration in Open Source
ivargrimstad
0
150
Remix × Cloudflare Pages × Sentry 奮闘記 / remix-pages-sentry
nkzn
1
370
Featured
See All Featured
The Cult of Friendly URLs
andyhume
76
6k
Building Flexible Design Systems
yeseniaperezcruz
327
38k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
42
6.5k
Done Done
chrislema
181
16k
How GitHub Uses GitHub to Build GitHub
holman
473
290k
Six Lessons from altMBA
skipperchong
26
3.4k
The Pragmatic Product Professional
lauravandoore
31
6.2k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
[RailsConf 2023] Rails as a piece of cake
palkan
49
4.7k
Clear Off the Table
cherdarchuk
91
320k
Why Our Code Smells
bkeepers
PRO
334
57k
Debugging Ruby Performance
tmm1
73
12k
Transcript
EZIO BT-based Data Transfer Yu-Chiang Huang
Outline • CloneZilla • EZIO
CloneZilla • 猋犩牏螭ܻ敟誢 • Server Edition • Multicast-based • UDPcast
Multicast • ᗭ砮 • 盄蕦褾牧ℂ㬵䷱睞螂 • 墋ᘒԏ牧觊犲 Broadcast ጱ狶ဩ
UDPcast • Multicast • ݶྍ秚ګ
CloneZilla • 猻玊 • ݢ瓟婘 • ە玊 • 殾ಅ磪ፓ秂伛猋ਠ౮胼樄ত •
殾ݶྍ蒂ቘ牧ݢ胼䨝ݑک絑ह磧眸ጱࢩᔰ段
EZIO • BT-based • 䲆礯羬翄螭ܻ • ᪡螂绚玟璸
BT
BT • BitTorrent • P2P • ֑๐瑊犵毱疝揗斉
BT 礍䯤 • 獮๗ BT • Torrent • Tracker •
盅๗ BT • Magnet • DHT
Tracker • 戢㲘㪔ᔱ玲ٌ犢 Peer ጱ蝫娄虻懱 • Peer 戢㲘 Tracker ᛔ૩ጱ虻懱
Piece • 裾ଶ傶 2^N ጱ蝫媲玟ྦྷ • 疥ಅ磪䲆礯蝫矑牧ྯ 2^N 㴟獤傶Ӟ㮆 Piece
• 虻碘涢挨ጱ磧ੜ㻌֖ • Piece 碍ᰁ蝢ଉ䨝段蝧ଶ • 蝢ଉ Piece 碍ᰁࣁ 1000 ~ 2000 ԏ樌䨝磧֯
Torrent • 蝢圸物圵ৼ • Tracker 虻懱 • 䲆礯虻懱
Torrent Info { 'announce': 'http://tracker.site1.com/announce', 'info': { 'name': 'directoryName', 'piece
length': 262144, 'files': [ {'path': ['111.txt'], 'length': 111}, {'path': ['222.txt'], 'length': 222} ], 'pieces': '6a8af7eda90ba9f851831073c48ea6b7b7e9feeb... 8a43d9d965a47f75488d3fb47d2c586337a20b9f' } }
Tracker { 'announce': 'http://tracker.site1.com/announce', 'info': { 'name': 'directoryName', 'piece length':
262144, 'files': [ {'path': ['111.txt'], 'length': 111}, {'path': ['222.txt'], 'length': 222} ], 'pieces': '6a8af7eda90ba9f851831073c48ea6b7b7e9feeb... 8a43d9d965a47f75488d3fb47d2c586337a20b9f' } }
Files { 'announce': 'http://tracker.site1.com/announce', 'info': { 'name': 'directoryName', 'piece length':
262144, 'files': [ {'path': ['111.txt'], 'length': 111}, {'path': ['222.txt'], 'length': 222} ], 'pieces': '6a8af7eda90ba9f851831073c48ea6b7b7e9feeb... 8a43d9d965a47f75488d3fb47d2c586337a20b9f' } }
Piece Info { 'announce': 'http://tracker.site1.com/announce', 'info': { 'name': 'directoryName', 'piece
length': 262144, 'files': [ {'path': ['111.txt'], 'length': 111}, {'path': ['222.txt'], 'length': 222} ], 'pieces': '6a8af7eda90ba9f851831073c48ea6b7b7e9feeb... 8a43d9d965a47f75488d3fb47d2c586337a20b9f' } }
Filename { 'announce': 'http://tracker.site1.com/announce', 'info': { 'name': 'directoryName', 'piece length':
262144, 'files': [ {'path': ['111.txt'], 'length': 111}, {'path': ['222.txt'], 'length': 222} ], 'pieces': '6a8af7eda90ba9f851831073c48ea6b7b7e9feeb... 8a43d9d965a47f75488d3fb47d2c586337a20b9f' } }
File Hash { 'announce': 'http://tracker.site1.com/announce', 'info': { 'name': 'directoryName', 'piece
length': 262144, 'files': [ {'path': ['111.txt'], 'length': 111}, {'path': ['222.txt'], 'length': 222} ], 'pieces': '6a8af7eda90ba9f851831073c48ea6b7b7e9feeb... 8a43d9d965a47f75488d3fb47d2c586337a20b9f' } }
BT • 猻玊 • 覍ݶྍ蒂ቘ • 虻碘涢挨 • ە玊 •
犋ݢ瓟婘
Implement
Implement • ल䲆礯羬翄蝱 Torrent • ፗ矑䌃蝱Ꮭ繕
Filesystem • Idea • Filesystem = Metadata + Data
Filesystem • Implement • Filesystem = Blocks • inode ∈
Block • data ∈ Block
None
None
Filesystem • Block 傶च器牧Block size ࢴਧ • 磪 Superblock 牧獵ӥ磪ֵአکጱ
Block ᮷ݢ犥Ꭳ • Block offset • Block size
Filename { 'announce': 'http://tracker.site1.com/announce', 'info': { 'name': 'directoryName', 'piece length':
262144, 'files': [ {'path': ['111.txt'], 'length': 111}, {'path': ['222.txt'], 'length': 222} ], 'pieces': '6a8af7eda90ba9f851831073c48ea6b7b7e9feeb... 8a43d9d965a47f75488d3fb47d2c586337a20b9f' } }
Filesystem Block • ल䲆礯羬翄蝱 Torrent • Offset —> Filename •
Length —> File Length
Write Disk Directly • 眤蘛 libtorrent.org • 䋿֢ custom_storage •
ፗ矑 Block 䌃کᏝ繕ྋ嘦ጱ֖ᗝӤ
Benchmark
Benchmark • ፓ獮ጱ䋿涢᮷氥纈䨝穉 Multicast ጱ狶ဩ盠 • To be continue
Optimization
Connection • 蝫娄碍哴ݢ胼牧Ӭ犋胼犵ෝ 2 牧犥ع蝐玕 • 姘瞱犵蝫娄碍牏ṛߝ搡蝫娄
Cache • 盠玲哴ݢ胼य़ • 碉誢蝧ଶ疰犋䨝ݑ褖ෝᏝ繕蝧ଶ • 翕᪠蝧ଶ > Ꮭ繕褰秚ਂ玲蝧ଶ
Origin Server • ܻত秚ጱ蝧ଶ段碉誢蝧ଶ • ܻত秚哴ݢ胼盠
WARNING!
Warning • ݢ胼蝧ଶ盠ک䨝虏֦ጱ Switch 稴吚 • ইຎ Switch 胙礂毱疝磪猂碘 •
森 P2P ጱ翕᪠ݢ胼䨝犋胼ֵአ
How to
How to • Standalone EZIO • EZIO integrated CloneZilla