Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Internals of GitLab and libgit2

Minqi Pan
November 03, 2015

Internals of GitLab and libgit2

Minqi Pan

November 03, 2015

Transcript

37 slides
  1. GitLab 的励志故事 • 2011年年:第⼀一⾏行行代码,基于 Rails 3 • 2012年年:发布 2.1.0~4.0.0 •

    2013年年:升级到 Rails 4,发布 6.5.0
 在阿姆斯特丹丹注册公司 GitLab B.V. • 2014年年:申请 YC 孵化器成功,发布 7.7.0
 获得天使融资12万美⾦金金
  2. ⽩白盒分析 NGINX OpenSSH Server Postfix Unicorn Sidekiq Gitlab Workhorse libgit2

    gitlab-ci gitlab-rails gitlab-shell mattermost omnibus-ctl git
  3. 具体场景分析 • 创建项⽬目 • 通过 HTTP 提交代码 • 通过 SSH

    克隆隆项⽬目 • 在⻚页⾯面上查看提交历史
  4. 浏览器 POST /projects … Gitlab::Shell#add_repository GitlabProjects#exec GitlabProjects#add_project $ git \

    --git-dir=/home/git/repositories/psvr/repo903.git \ init \ --bare $ /opt/gitlab/embedded/ service/gitlab-shell/ bin/gitlab-projects \ add-project \ psvr/repo903.git FileUtils.ln_sf(“/home/git/gitlab-shell/hooks", "/home/git/repositories/psvr/repo903.git/hooks")
  5. 通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack NGINX Gitlab

    Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) GET http://.../pmq2001/repo903.git/ info/refs?service=git-receive-pack 去掉 body
  6. 通过 HTTP 提交代码 Unicorn git push … Grack::Auth#auth! Gitlab Workhorse

    validated the client request additional request metadata in JSON GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack
  7. 通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack … Gitlab

    Workhorse $ git \ receive-pack \ --stateless-rpc \ --advertise-refs \ /home/git/repositories/psvr/repo903.git func handleGetInfoRefs (w http.ResponseWriter, r *gitRequest, _ string)
  8. 通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab

    Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) POST /pmq2001/repo903.git/git-receive- pack 去掉 body
  9. 通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab

    Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git
  10. 通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab

    Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git $ hooks/pre-receive gitlab-shell
  11. 通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack $ hooks/pre-receive

    GitlabAccess#exec … Gitlab Workhorse Unicorn POST "/api/v3/internal/allowed"
  12. 通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn

    $ hooks/post-receive Postgre SQL GitlabPostReceive#update_redis Sidekiq PostReceive
  13. 通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn

    $ hooks/post-receive GitlabNet#broadcast_message GET /api/v3/internal/broadcast_message Unicorn ProjectCacheWorker Postgre SQL Sidekiq
  14. 通过 SSH 克隆隆项⽬目 ssh [email protected] GitlabShell#exec [root@i-jl15ph2j ~]# ssh [email protected]

    Warning: Permanently added 'gitlab.server,1.2.3.4' (RSA) to the list of known hosts. PTY allocation request failed on channel 0 Welcome to GitLab, psvr! Connection to gitlab.server closed. gitlab-shell OpenSSH Server
  15. 通过 SSH 克隆隆项⽬目 git clone GitlabShell#verify_access gitlab-shell OpenSSH Server [email protected]:psvr/repo903.git

    $ git-upload-pack \ /var/opt/gitlab/git-data/repositories/psvr/repo903.git 009bd9be3108e6e60b9954fec57f6ca00d4f74 95170d HEADmulti_ack thin-pack side- band side-band-64k ofs-delta shallow no-progress include-tag multi_ack_detailed 003fd9be3108e6e60b9954fec57f6ca00d4f74 95170d refs/heads/master 0000
  16. 在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $

    git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
  17. 在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $

    git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
  18. libgit2 是封装良好的 • ⼤大部分头⽂文件位于 src/*.h,是不公开的 • 像最基本的 git_repository 结构体都只有外部声明 •

    include/git2/*.h 分类良好,快速定位接⼝口,包含 丰富的注释(对比 mruby =.=) • [ demo ]

More Decks by Minqi Pan

Other Decks in Programming