Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Manage Private Internal Dependencies With Compo...

Manage Private Internal Dependencies With Composer and Satis

In this talk we'll discuss how to set up and use Composer to pull open-source packages from public repositories on packagist.org. Then we'll learn how to create your own private/corporate package management server using Satis. And we'll go through an example of how to integrate your own private code packaging server into your daily development workflow.

Note: This presentation contained a fair amount of video which is lost on Speaker Deck

Andrew Cassell

May 20, 2015
Tweet

More Decks by Andrew Cassell

Other Decks in Programming

Transcript

  1. GOOD ARTISTS COPY GOOD ARTISTS COPY GREAT ARTISTS STEAL GREAT

    ARTISTS STEAL https://nancyprager.wordpress.com/2007/05/08/good-poets-borrow-great-poets-steal/
  2. GOOD DEVELOPERS COPY GOOD DEVELOPERS COPY GREAT DEVELOPERS ! DEVELOPERS!

    DEVELOPERS! USE COMPOSER GREAT DEVELOPERS ! DEVELOPERS! DEVELOPERS! USE COMPOSER
  3. Dependency Manager 1. Download Project Libraries 2. Download Dependent Libraries

    3. Verify Compatibility 4. Autoloader for ALL THE THINGS
  4. cassell$ composer require aws/aws-sdk-php Using version ^2.8 for aws/aws-sdk-php ./composer.json

    has been created Loading composer repositories with package information Updating dependencies (including require-dev) - Installing symfony/event-dispatcher (v2.6.7) Loading from cache - Installing guzzle/guzzle (v3.9.3) Downloading: 100% - Installing aws/aws-sdk-php (2.8.5) Downloading: 100%
  5. ^

  6. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” },

    "require-dev": {
 "phpunit/phpunit": "^4.1.0"
 } } Dev Dependencies in composer.json
  7. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” },

    "autoload": {
 "psr-4": {"ProjectName\\": "classes/"}
 } } Autoloader in composer.json
  8. $ composer install Loading composer repositories with package information Installing

    dependencies (including require-dev) - Installing react/promise (v2.2.0) Downloading: 100% - Installing guzzlehttp/streams (3.0.0) Downloading: 100% - Installing guzzlehttp/ringphp (1.0.7) Downloading: 100% - Installing guzzlehttp/guzzle (5.2.0) Downloading: 100% - Installing symfony/dom-crawler (v2.6.7) Downloading: 100% - Installing symfony/css-selector (v2.6.7) Downloading: 100% - Installing symfony/browser-kit (v2.6.7) Downloading: 100% - Installing fabpot/goutte (v2.0.4) Downloading: 100% - Installing symfony/yaml (v2.6.7) Downloading: 100% symfony/browser-kit suggests installing symfony/process () Writing lock file Generating autoload files
  9. $ composer install Loading composer repositories with package information Installing

    dependencies (including require-dev) - Installing react/promise (v2.2.0) Downloading: 100% - Installing guzzlehttp/streams (3.0.0) Downloading: 100% - Installing guzzlehttp/ringphp (1.0.7) Downloading: 100% - Installing guzzlehttp/guzzle (5.2.0) Downloading: 100%
  10. 
 <?php
 
 require_once __DIR__ . “/vendor/autoload.php”; $client = new

    \Goutte\Client();
 
 $crawler = $client->request('GET', 'http:// tek.phparch.com/speakers/'); scraper.php
  11. 
 <?php
 
 require_once __DIR__ . “/vendor/autoload.php”; $client = new

    \Goutte\Client();
 
 $crawler = $client->request('GET', 'http:// tek.phparch.com/speakers/'); scraper.php
  12. <?php
 
 require_once __DIR__ . "/vendor/autoload.php";
 use \Symfony\Component\DomCrawler\Crawler;
 
 $client

    = new \Goutte\Client();
 
 $crawler = $client->request('GET', 'http://tek.phparch.com/speakers/');
 
 $speakers = $crawler->filter('#speakerlist > div')->each(function (Crawler $node) {
 
 $speaker = [];
 $speaker["name"] = $node->filter('div.headshot > img')->attr("alt");
 $speaker["gravatar"] = $node->filter('div.headshot > img')->attr("src");
 $speaker["company"] = $node->filter('div.info > h4')->text();
 try {
 $speaker["twitter"] = $node->filter('div.info > h3 > a')->text();
 } catch (\Exception $e) {
 // might fail
 $speaker["twitter"] = "";
 }
 
 $speaker["talks"] = $node->filter('div.info > dl')->first()->siblings()->filter('dl')->each(function (Crawler $talkNode) {
 
 $talk = [];
 $talk['type'] = $talkNode->filter('dt > div')->eq(0)->text();
 $talk['level'] = $talkNode->filter('dt > div')->eq(1)->text();
 $talk['title'] = $talkNode->filter('dd > h5')->text();
 $texts = [];
 foreach ($talkNode->filter('dd')->getNode(0)->childNodes as $child) {
 if ($child instanceof DOMText) {
 $texts[] = trim($child->textContent);
 }
 
 }
 $talk['room'] = $texts[2];
 $talk['when'] = $texts[3];
 
 return $talk;
 
 });
 
 return $speaker;
 
 });
 
 
 $dumper = new \Symfony\Component\Yaml\Dumper();
 
 echo $dumper->dump(["speakers" => $speakers],999);

  13. { "require": { “company/secret-sauce“: “^4.0” }, "repositories": [ { "type":

    "vcs", "url": “[email protected]:company/secret-sauce.git” } ] } composer.json
  14. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^1.0.0"
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": “https://packageserver.example.com/“
 }
 ]
 }
  15. {
 "name": "Secret Corporation Package Server",
 "homepage": "https://packageserver.example.com/",
 "repositories": [


    {
 "type": "vcs",
 "url": "[email protected]:cassell/example-phptek-scraper-package.git"
 },
 {
 "type": "vcs",
 "url": "[email protected]:cassell/example-cache-package.git"
 },
 {
 "type": "vcs",
 "url": "[email protected]:cassell/example-s3-wrapper-package.git"
 }
 ],
 
 "require-all": true,
 
 "archive": {
 "directory": "dist",
 "format": "tar",
 "prefix-url": “https://packageserver.example.com/“, "skip-dev": true }
 }
  16. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^1.0.0"
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
  17. <?php
 
 require_once '../vendor/autoload.php';
 
 $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client());


    
 // Create new Plates engine
 $templates = new League\Plates\Engine(__DIR__ . '/../templates');
 
 // Create a new template
 echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);

  18. <?php
 
 require_once '../vendor/autoload.php';
 
 $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client());


    
 // Create new Plates engine
 $templates = new League\Plates\Engine(__DIR__ . '/../templates');
 
 // Create a new template
 echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);

  19. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^1.0.0"
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
  20. Loading composer repositories with package information Updating dependencies (including require-dev)

    - Removing secret-company-example/phptek-scraper (1.0.2) - Installing secret-company-example/phptek-scraper (dev-master ab88f08) Cloning ab88f0877781202929049bf3516b23ddb6c0fa12 Writing lock file Generating autoload files cassell:example cassell$ composer update
  21. cassell:phptek-scraper cassell$ pwd ~/example/vendor/secret-company-example/phptek-scraper cassell:phptek-scraper cassell$ git status On branch

    master Your branch is up-to-date with 'origin/master'. nothing to commit, working directory clean cassell:phptek-scraper cassell$
  22. cassell:phptek-scraper cassell$ git status On branch master Your branch is

    up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$
  23. cassell:phptek-scraper cassell$ git status On branch master Your branch is

    up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$ git add .
  24. cassell:phptek-scraper cassell$ git status On branch master Your branch is

    up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$ git add . cassell:phptek-scraper cassell$ git commit "added caching to Scraper"
  25. cassell:phptek-scraper cassell$ git status On branch master Your branch is

    up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$ git add . cassell:phptek-scraper cassell$ git commit "added caching to Scraper" [master 003abe2] added caching to Scraper 2 files changed, 42 insertions(+), 2 deletions(-) cassell:phptek-scraper cassell$
  26. cassell:phptek-scraper cassell$ git push origin master Counting objects: 5, done.

    Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done. Total 5 (delta 2), reused 0 (delta 0) To [email protected]:cassell/example-phptek-scraper-package.git ab88f08..003abe2 master -> master
  27. cassell:phptek-scraper cassell$ git push origin master Counting objects: 5, done.

    Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done. Total 5 (delta 2), reused 0 (delta 0) To [email protected]:cassell/example-phptek-scraper-package.git ab88f08..003abe2 master -> master cassell:phptek-scraper cassell$ git tag -a 2.0.0 -m "2.0.0"
  28. cassell:phptek-scraper cassell$ git push origin master Counting objects: 5, done.

    Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done. Total 5 (delta 2), reused 0 (delta 0) To [email protected]:cassell/example-phptek-scraper-package.git ab88f08..003abe2 master -> master cassell:phptek-scraper cassell$ git tag -a 2.0.0 -m “2.0.0" cassell:phptek-scraper cassell$ git push origin --tags Counting objects: 1, done. Writing objects: 100% (1/1), 166 bytes | 0 bytes/s, done. Total 1 (delta 0), reused 0 (delta 0) To [email protected]:cassell/example-phptek-scraper-package.git * [new tag] 2.0.0 -> 2.0.0
  29. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “dev-master“
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
  30. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^2.0.0“
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
  31. <?php
 
 require_once '../vendor/autoload.php';
 
 class SpeakerCache extends SecretCorporation\ExampleCache\Cache
 {


    
 }
 
 $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client(), new SpeakerCache());
 
 // Create new Plates engine
 $templates = new League\Plates\Engine(__DIR__ . '/../templates');
 
 // Create a new template
 echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);
  32. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass Private Package Update Worlflow
  33. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass 4. Tag new version Private Package Update Worlflow
  34. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass 4. Tag new version 5. Push changes and tags Private Package Update Worlflow
  35. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass 4. Tag new version 5. Push changes and tags 6. Satis rebuild* Private Package Update Worlflow
  36. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass 4. Tag new version 5. Push changes and tags 6. Satis rebuild* 7. Composer update Private Package Update Worlflow
  37. • Cron the rebuild • Git hook for rebuild •

    Github Webhook Satis Improvements
  38. * * * * * cd /path/to/satis && test -f

    web/rebuild && rm web/ rebuild && php bin/satis build -q satis.json ./web <?php // web/postcommit.php touch(__DIR__.'/rebuild'); Jérôme Tamarelle @GromNaN
  39. Satis Improvements • Cron the rebuild • Git hook for

    rebuild • Github Webhook • Mirror Github
  40. {
 "name": "Secret Corporation Package Server",
 "homepage": "https://packageserver.example.com/",
 "repositories": [


    {
 "type": "vcs",
 "url": "[email protected]:cassell/example-phptek-scraper-package.git"
 },
 {
 "type": "vcs",
 "url": "[email protected]:cassell/example-cache-package.git"
 },
 { "type": “composer”, "url": “https://packagist.org" }
 ], "require": { "monolog/monolog": “*", "aws/aws-sdk-php": "*", "phpunit/phpunit": "*" }, "require-dependencies": true
 
 "require-all": true,
 

  41. • No Credentials in Code • Firewall • Use SSH

    • Use HTTPS • Generate to Secret Folder Securing Satis
  42. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “2.0.0“
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/6PeTZwIhRBOiSm
 }
 ]
 }