Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Internals of GitLab and libgit2
Search
Minqi Pan
November 03, 2015
Programming
4
1.8k
Internals of GitLab and libgit2
Minqi Pan
November 03, 2015
Tweet
Share
More Decks by Minqi Pan
See All by Minqi Pan
RubyConf 2017: Packing your Ruby application into a single executable
pmq20
1
680
Node 8 and Friends
pmq20
0
250
Moving Node.js and nodec to v8 Ignition
pmq20
0
360
QCon 2017 Beijing Observations
pmq20
0
270
Node.js Compiler: compiling your Node.js application into a single executable
pmq20
5
25k
Enclose.IO: current cutting-edges and the future work
pmq20
0
250
环境变量为何能让 Ruby 快十倍
pmq20
3
760
Introductions to Node.js internals
pmq20
0
2.1k
How we scaled GitLab for a 30k-employee company
pmq20
6
1.6k
Other Decks in Programming
See All in Programming
O Que É e Como Funciona o PHP-FPM?
marcelgsantos
0
180
AI 駆動開発におけるコミュニティと AWS CDK の価値
konokenj
4
150
「ちょっと古いから」って避けてた技術書、今だからこそ読もう
mottyzzz
12
7.1k
他言語経験者が Golangci-lint を最初のコーディングメンターにした話 / How Golangci-lint Became My First Coding Mentor: A Story from a Polyglot Programmer
uma31
0
340
Goで実践するドメイン駆動開発 AIと歩み始めた新規プロダクト開発の現在地
imkaoru
4
880
monorepo の Go テストをはやくした〜い!~最小の依存解決への道のり~ / faster-testing-of-monorepos
convto
2
530
3年ぶりにコードを書いた元CTOが Claude Codeと30分でMVPを作った話
maikokojima
0
620
CSC305 Lecture 10
javiergs
PRO
0
220
ALL CODE BASE ARE BELONG TO STUDY
uzulla
27
6.6k
TransformerからMCPまで(現代AIを理解するための羅針盤)
mickey_kubo
6
3.1k
Google Opalで使える37のライブラリ
mickey_kubo
3
130
CSC305 Lecture 06
javiergs
PRO
0
270
Featured
See All Featured
Navigating Team Friction
lara
190
15k
How STYLIGHT went responsive
nonsquared
100
5.8k
Why Our Code Smells
bkeepers
PRO
340
57k
Producing Creativity
orderedlist
PRO
347
40k
StorybookのUI Testing Handbookを読んだ
zakiyama
31
6.2k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
990
Raft: Consensus for Rubyists
vanstee
140
7.2k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
253
22k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.2k
The Invisible Side of Design
smashingmag
302
51k
Building a Scalable Design System with Sketch
lauravandoore
463
33k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
46
2.5k
Transcript
GitLab 与 libgit2 原理理 Minqi Pan
I’m Minqi Pan github.com/pmq20 twitter @psvr
GitLab 的励志故事 • 2011年年:第⼀一⾏行行代码,基于 Rails 3 • 2012年年:发布 2.1.0~4.0.0 •
2013年年:升级到 Rails 4,发布 6.5.0 在阿姆斯特丹丹注册公司 GitLab B.V. • 2014年年:申请 YC 孵化器成功,发布 7.7.0 获得天使融资12万美⾦金金
GitLab 的励志故事 • 2015年年1⽉月~3⽉月:进驻旧⾦金金⼭山湾区⼭山景城 • 2015年年7⽉月:完成 150万 美元 pre-A 轮融资
• 2015年年9⽉月:完成 570万 美元 A 轮融资 • 2015年年10⽉月:发布 8.1.2
⿊黑盒分析 GitLab
⿊黑盒分析 GitLab HTTP 80 SSH 22 SMTP 25/587
⿊黑盒分析 GitLab Redis Postgre SQL File System
⽩白盒分析 GitLab
⽩白盒分析 NGINX OpenSSH Server Postfix Unicorn Sidekiq Gitlab Workhorse libgit2
gitlab-ci gitlab-rails gitlab-shell mattermost omnibus-ctl git
具体场景分析 • 创建项⽬目 • 通过 HTTP 提交代码 • 通过 SSH
克隆隆项⽬目 • 在⻚页⾯面上查看提交历史
创建项⽬目 浏览器 POST /projects ProjectsController#create CreateService#execute Project#save Postgre SQL NGINX
Unicorn
浏览器 POST /projects … Gitlab::Shell#add_repository GitlabProjects#exec GitlabProjects#add_project $ git \
--git-dir=/home/git/repositories/psvr/repo903.git \ init \ --bare $ /opt/gitlab/embedded/ service/gitlab-shell/ bin/gitlab-projects \ add-project \ psvr/repo903.git FileUtils.ln_sf(“/home/git/gitlab-shell/hooks", "/home/git/repositories/psvr/repo903.git/hooks")
浏览器 POST /projects CreateService#after_create_actions Gitlab::Shell#add_repository ProjectsController#create CreateService#execute Project#save 浏览器 302
重定向
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack NGINX Gitlab
Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) GET http://.../pmq2001/repo903.git/ info/refs?service=git-receive-pack 去掉 body
通过 HTTP 提交代码 Unicorn git push … Grack::Auth#auth! Gitlab Workhorse
validated the client request additional request metadata in JSON GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack … Gitlab
Workhorse $ git \ receive-pack \ --stateless-rpc \ --advertise-refs \ /home/git/repositories/psvr/repo903.git func handleGetInfoRefs (w http.ResponseWriter, r *gitRequest, _ string)
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack … Gitlab
Workhorse
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) POST /pmq2001/repo903.git/git-receive- pack 去掉 body
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git $ hooks/pre-receive gitlab-shell
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack $ hooks/pre-receive
GitlabAccess#exec … Gitlab Workhorse Unicorn POST "/api/v3/internal/allowed"
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn
$ hooks/post-receive Postgre SQL GitlabPostReceive#update_redis Sidekiq PostReceive
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn
$ hooks/post-receive GitlabNet#broadcast_message GET /api/v3/internal/broadcast_message Unicorn ProjectCacheWorker Postgre SQL Sidekiq
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse
通过 SSH 克隆隆项⽬目 ssh
[email protected]
OpenSSH Server gitlab-shell GitlabShell#exec
通过 SSH 克隆隆项⽬目 ssh
[email protected]
GitlabShell#exec [root@i-jl15ph2j ~]# ssh
[email protected]
Warning: Permanently added 'gitlab.server,1.2.3.4' (RSA) to the list of known hosts. PTY allocation request failed on channel 0 Welcome to GitLab, psvr! Connection to gitlab.server closed. gitlab-shell OpenSSH Server
通过 SSH 克隆隆项⽬目 git clone GitlabShell#verify_access gitlab-shell OpenSSH Server
[email protected]
:psvr/repo903.git
Unicorn POST "/api/v3/internal/allowed"
通过 SSH 克隆隆项⽬目 git clone GitlabShell#verify_access gitlab-shell OpenSSH Server
[email protected]
:psvr/repo903.git
$ git-upload-pack \ /var/opt/gitlab/git-data/repositories/psvr/repo903.git 009bd9be3108e6e60b9954fec57f6ca00d4f74 95170d HEADmulti_ack thin-pack side- band side-band-64k ofs-delta shallow no-progress include-tag multi_ack_detailed 003fd9be3108e6e60b9954fec57f6ca00d4f74 95170d refs/heads/master 0000
在⻚页⾯面上查看提交历史 浏览器 GET Projects::CommitsController#show Repository#commits NGINX Unicorn /psvr/repo903/commits/master gem -
gitlab_git
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $
git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $
git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - rugged … Rugged::Commit#message rb_git_commit_message_GET
git_commit_message libgit2
libgit2 的特点 portable, pure C, re-entrant, linkable, solid API 提⽰示:对比
ruby 与 mruby
libgit2 是封装良好的 • ⼤大部分头⽂文件位于 src/*.h,是不公开的 • 像最基本的 git_repository 结构体都只有外部声明 •
include/git2/*.h 分类良好,快速定位接⼝口,包含 丰富的注释(对比 mruby =.=) • [ demo ]
libgit2 是抽象良好的 • git2/sys/*_backend.h 解耦, 但是 index 没有解耦 • 通过这些函数挂接新的后端
None
Thank you https://github.com/pmq20/