Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Internals of GitLab and libgit2
Search
Minqi Pan
November 03, 2015
Programming
4
1.8k
Internals of GitLab and libgit2
Minqi Pan
November 03, 2015
Tweet
Share
More Decks by Minqi Pan
See All by Minqi Pan
RubyConf 2017: Packing your Ruby application into a single executable
pmq20
1
670
Node 8 and Friends
pmq20
0
240
Moving Node.js and nodec to v8 Ignition
pmq20
0
360
QCon 2017 Beijing Observations
pmq20
0
270
Node.js Compiler: compiling your Node.js application into a single executable
pmq20
5
25k
Enclose.IO: current cutting-edges and the future work
pmq20
0
240
环境变量为何能让 Ruby 快十倍
pmq20
3
760
Introductions to Node.js internals
pmq20
0
2.1k
How we scaled GitLab for a 30k-employee company
pmq20
6
1.6k
Other Decks in Programming
See All in Programming
AIで開発生産性を上げる個人とチームの取り組み
taniigo
0
130
CSC509 Lecture 02
javiergs
PRO
0
400
CI_CD「健康診断」のススメ。現場でのボトルネック特定から、健康診断を通じた組織的な改善手法
teamlab
PRO
0
150
ポスターセッション: 「まっすぐ行って、右!」って言ってラズパイカーを動かしたい 〜生成AI × Raspberry Pi Pico × Gradioの試作メモ〜
komofr
0
880
Playwrightはどのようにクロスブラウザをサポートしているのか
yotahada3
7
2.2k
CSC305 Lecture 04
javiergs
PRO
0
230
プロダクト開発をAI 1stに変革する〜SaaS is dead時代で生き残るために〜 / AI 1st Product Development
kobakei
0
440
iOS 17で追加されたSubscriptionStoreView を利用して5分でサブスク実装チャレンジ
natmark
0
430
CSC305 Lecture 03
javiergs
PRO
0
230
dynamic!
moro
9
5.1k
TokyoR#119 bignners session2 Visualization
kotatyamtema
0
130
uniqueパッケージの内部実装を支えるweak pointerの話
magavel
0
870
Featured
See All Featured
Producing Creativity
orderedlist
PRO
347
40k
Documentation Writing (for coders)
carmenintech
75
5k
Fireside Chat
paigeccino
40
3.7k
Six Lessons from altMBA
skipperchong
28
4k
Automating Front-end Workflow
addyosmani
1371
200k
Docker and Python
trallard
46
3.6k
How STYLIGHT went responsive
nonsquared
100
5.8k
Making the Leap to Tech Lead
cromwellryan
135
9.5k
Testing 201, or: Great Expectations
jmmastey
45
7.7k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
570
The Straight Up "How To Draw Better" Workshop
denniskardys
237
140k
How to Ace a Technical Interview
jacobian
280
23k
Transcript
GitLab 与 libgit2 原理理 Minqi Pan
I’m Minqi Pan github.com/pmq20 twitter @psvr
GitLab 的励志故事 • 2011年年:第⼀一⾏行行代码,基于 Rails 3 • 2012年年:发布 2.1.0~4.0.0 •
2013年年:升级到 Rails 4,发布 6.5.0 在阿姆斯特丹丹注册公司 GitLab B.V. • 2014年年:申请 YC 孵化器成功,发布 7.7.0 获得天使融资12万美⾦金金
GitLab 的励志故事 • 2015年年1⽉月~3⽉月:进驻旧⾦金金⼭山湾区⼭山景城 • 2015年年7⽉月:完成 150万 美元 pre-A 轮融资
• 2015年年9⽉月:完成 570万 美元 A 轮融资 • 2015年年10⽉月:发布 8.1.2
⿊黑盒分析 GitLab
⿊黑盒分析 GitLab HTTP 80 SSH 22 SMTP 25/587
⿊黑盒分析 GitLab Redis Postgre SQL File System
⽩白盒分析 GitLab
⽩白盒分析 NGINX OpenSSH Server Postfix Unicorn Sidekiq Gitlab Workhorse libgit2
gitlab-ci gitlab-rails gitlab-shell mattermost omnibus-ctl git
具体场景分析 • 创建项⽬目 • 通过 HTTP 提交代码 • 通过 SSH
克隆隆项⽬目 • 在⻚页⾯面上查看提交历史
创建项⽬目 浏览器 POST /projects ProjectsController#create CreateService#execute Project#save Postgre SQL NGINX
Unicorn
浏览器 POST /projects … Gitlab::Shell#add_repository GitlabProjects#exec GitlabProjects#add_project $ git \
--git-dir=/home/git/repositories/psvr/repo903.git \ init \ --bare $ /opt/gitlab/embedded/ service/gitlab-shell/ bin/gitlab-projects \ add-project \ psvr/repo903.git FileUtils.ln_sf(“/home/git/gitlab-shell/hooks", "/home/git/repositories/psvr/repo903.git/hooks")
浏览器 POST /projects CreateService#after_create_actions Gitlab::Shell#add_repository ProjectsController#create CreateService#execute Project#save 浏览器 302
重定向
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack NGINX Gitlab
Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) GET http://.../pmq2001/repo903.git/ info/refs?service=git-receive-pack 去掉 body
通过 HTTP 提交代码 Unicorn git push … Grack::Auth#auth! Gitlab Workhorse
validated the client request additional request metadata in JSON GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack … Gitlab
Workhorse $ git \ receive-pack \ --stateless-rpc \ --advertise-refs \ /home/git/repositories/psvr/repo903.git func handleGetInfoRefs (w http.ResponseWriter, r *gitRequest, _ string)
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack … Gitlab
Workhorse
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) POST /pmq2001/repo903.git/git-receive- pack 去掉 body
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git $ hooks/pre-receive gitlab-shell
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack $ hooks/pre-receive
GitlabAccess#exec … Gitlab Workhorse Unicorn POST "/api/v3/internal/allowed"
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn
$ hooks/post-receive Postgre SQL GitlabPostReceive#update_redis Sidekiq PostReceive
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn
$ hooks/post-receive GitlabNet#broadcast_message GET /api/v3/internal/broadcast_message Unicorn ProjectCacheWorker Postgre SQL Sidekiq
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse
通过 SSH 克隆隆项⽬目 ssh
[email protected]
OpenSSH Server gitlab-shell GitlabShell#exec
通过 SSH 克隆隆项⽬目 ssh
[email protected]
GitlabShell#exec [root@i-jl15ph2j ~]# ssh
[email protected]
Warning: Permanently added 'gitlab.server,1.2.3.4' (RSA) to the list of known hosts. PTY allocation request failed on channel 0 Welcome to GitLab, psvr! Connection to gitlab.server closed. gitlab-shell OpenSSH Server
通过 SSH 克隆隆项⽬目 git clone GitlabShell#verify_access gitlab-shell OpenSSH Server
[email protected]
:psvr/repo903.git
Unicorn POST "/api/v3/internal/allowed"
通过 SSH 克隆隆项⽬目 git clone GitlabShell#verify_access gitlab-shell OpenSSH Server
[email protected]
:psvr/repo903.git
$ git-upload-pack \ /var/opt/gitlab/git-data/repositories/psvr/repo903.git 009bd9be3108e6e60b9954fec57f6ca00d4f74 95170d HEADmulti_ack thin-pack side- band side-band-64k ofs-delta shallow no-progress include-tag multi_ack_detailed 003fd9be3108e6e60b9954fec57f6ca00d4f74 95170d refs/heads/master 0000
在⻚页⾯面上查看提交历史 浏览器 GET Projects::CommitsController#show Repository#commits NGINX Unicorn /psvr/repo903/commits/master gem -
gitlab_git
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $
git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $
git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - rugged … Rugged::Commit#message rb_git_commit_message_GET
git_commit_message libgit2
libgit2 的特点 portable, pure C, re-entrant, linkable, solid API 提⽰示:对比
ruby 与 mruby
libgit2 是封装良好的 • ⼤大部分头⽂文件位于 src/*.h,是不公开的 • 像最基本的 git_repository 结构体都只有外部声明 •
include/git2/*.h 分类良好,快速定位接⼝口,包含 丰富的注释(对比 mruby =.=) • [ demo ]
libgit2 是抽象良好的 • git2/sys/*_backend.h 解耦, 但是 index 没有解耦 • 通过这些函数挂接新的后端
None
Thank you https://github.com/pmq20/