Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Server Survival

Server Survival

A guide through the annoying parts of servers. This includes slides I couldn't make it through in the actual presentation.

Chris Fidao

July 27, 2016
Tweet

More Decks by Chris Fidao

Other Decks in Technology

Transcript

  1. user & access new user new ssh key $ sudo

    adduser fideloper $ sudo usermod -a -G sudo fideloper $ ssh-keygen -t rsa -b 4096
  2. $ ssh-keygen -t rsa -b 4096 \ -f id_whatever $

    ssh-copy-id -i ~/.ssh/id_whatever \ fideloper@<server-ip> (added to ~/.ssh/authorized_keys file) user & access
  3. firewall sudo iptables -A INPUT -i lo \ -j ACCEPT

    sudo iptables -A INPUT -m conntrack \ —ctstate RELATED,ESTABLISHED -j ACCEPT
  4. firewall sudo iptables -A INPUT -p tcp --dport 22 \

    -j ACCEPT sudo iptables -A INPUT -p tcp --dport 80 \ -j ACCEPT sudo iptables -A INPUT -p tcp --dport 443 \ -j ACCEPT sudo iptables -A INPUT -j DROP
  5. firewall drop v reject default policy $ iptables … -j

    REJECT $ iptables \ --policy INPUT DROP
  6. auto upgrades APT::Periodic::Unattended-Upgrade "1"; Unattended-Upgrade::Allowed-Origins { "${distro_id}:${distro_codename}-security"; }; Unattended-Upgrade::InstallOnShutdown "false";

    Unattended-Upgrade::Automatic-Reboot "false"; $ sudo apt-get install -y \ unattended-upgrades files: /etc/apt/apt.conf.d
  7. ¡more! There’s always more SELinux / AppArmor 2FA for SSH

    Securing “secrets” (.env) Strong PW Enforcement (But don’t freak out about it)
  8. fid@host:~# sudo systemctl status ssh systemctl status <service> systemctl start

    <service> systemctl stop <service> systemctl enable <service> systemctl disable <service> systemd
  9. fid@host:~# sudo service ssh status • ssh.service - OpenBSD Secure

    Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled; \ vendor preset: enabled) Active: active (running) since Fri 2016-07-22 19:46:40 EDT; 1h 27min ago Main PID: 2493 (sshd) CGroup: /system.slice/ssh.service ├─ 2493 /usr/sbin/sshd -D ├─14218 sshd: root [priv] └─14219 sshd: root [net] Jul 22 21:13:28 host sshd[14114]: Accepted password for root from 76.185.167.253 port 56786 ssh2 Jul 22 21:13:28 host sshd[14114]: pam_unix(sshd:session): session opened for user root by (uid=0) systemd
  10. systemd [Unit] Description=OpenBSD Secure Shell server After=network.target auditd.service ConditionPathExists=!/etc/ssh/sshd_not_to_be_run [Service]

    EnvironmentFile=-/etc/default/ssh ExecStart=/usr/sbin/sshd -D $SSHD_OPTS ExecReload=/bin/kill -HUP $MAINPID KillMode=process Restart=on-failure RestartPreventExitStatus=255 Type=notify [Install] WantedBy=multi-user.target Alias=sshd.service /lib/systemd/system/ssh.service
  11. supervisord fid@spr:~# sudo systemctl status supervisor • supervisor.service - Supervisor

    process control system for UNIX Loaded: loaded (/lib/systemd/system/supervisor.service; disabled; vendor preset: enabled) Active: active (running) since Tue 2016-07-26 17:13:54 EDT; 3s ago Docs: http://supervisord.org Main PID: 3712 (supervisord) Tasks: 1 Memory: 11.1M CPU: 216ms CGroup: /system.slice/supervisor.service !"3712 /usr/bin/python /usr/bin/supervisord -n -c / etc/supervisor/supervisord.conf Jul 26 17:13:54 spr systemd[1]: Started Supervisor process control system for UNIX.
  12. [program:lara_queue] command=php artisan queue:work --daemon directory=/home/forge/app.com/current autostart=true autorestart=true startretries=3 redirect_stderr=true

    stdout_logfile=/home/forge/…/logs/queue.log user=forge numproc=4 supervisord /etc/supervisor/conf.d/lara_q.conf
  13. ifconfig private network f@db:~$ ifconfig eth0 Link encap:Ethernet HWaddr 04:01:31:20:63:01

    inet addr:162.243.164.216 Bcast:162.243.164.255 Mask:255.255.255.0 inet6 addr: fe80::601:31ff:fe20:6301/64 Scope:Link … eth1 Link encap:Ethernet HWaddr 04:01:31:20:63:02 inet addr:10.136.11.155 Bcast:10.136.255.255 Mask:255.255.0.0 inet6 addr: fe80::601:31ff:fe20:6302/64 Scope:Link … lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host …
  14. network binding forge@site:~$ netstat -ap | grep http tcp 0

    0 *:http *:* LISTEN 3797/nginx: worker tcp 0 0 *:https *:* LISTEN 3797/nginx: worker
  15. network: mysql # # Instead of skip-networking the default is

    now to listen only on # localhost which is more compatible and is not less secure. bind-address = 10.136.11.155 f@db:~$ mysql -h localhost -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. f@db:~$ mysql -h 127.0.0.1 -u root -p Enter password: ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111) ✅
  16. network: mysql f@db:~$ mysql -h 10.136.11.155 -u root -p Enter

    password: ERROR 1130 (HY000): Host '10.136.11.155' is not allowed to connect to this MySQL server f@db:~$ mysql -u root -p -e "create user root@'10.136.11.155' identified by 'root';" Enter password: f@db:~$ mysql -h 10.136.11.155 -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. ✅
  17. 1.Find networks (interfaces) 2.Learn about socket types 3.See examples of

    mysql 4.Future: Permissions in Forge network review
  18. who can do things user - file/dir owner group -

    file/dir group - shared permissions! other - anyone else
  19. what can they do read - read file, list directory

    write - write to file, add new file/dir execute - execute command, cd into
  20. usr@hst:~$ chown -R www-data:www-data \ /var/www/example.com usr@hst:~$ chmod -R u=rwx,g=rx,o=rx

    \ /var/www/example.com usr@hst:~$ chmod -R u=rwx,go=rx \ /var/www/example.com usr@hst:$ chmod ug+x,o-x \ /var/www/example.com/artisan setting permissions
  21. user@host:/var/www$ ls -lAh total 4.0K drwxrwxr-x 2 deploy www-data 4.0K

    Jul 10 21:43 example.com example d rwx rwx r-x deploy : www-data
  22. usr@host:/var/www$ ps axf o pid,user,group,comm \ | grep -i '[n]ginx\|[p]hp'

    4290 root root nginx 4291 www-data www-data \_ nginx 2887 root root php-fpm7.0 2889 www-data www-data \_ php-fpm7.0 2890 www-data www-data \_ php-fpm7.0 not just files
  23. user@host:/var/www$ ls -lAh total 4.0K drwxrwxr-x 2 deploy www-data 4.0K

    Jul 10 21:43 example.com php + web files <?php // run as user and group www-data file_put_contents( '/var/www/example.com/new-file.txt', 'Here is a new line' ); // ✅
  24. remember files owned by www-data then run php as www-data

    $ sudo -u www-data php artisan foo:bar
  25. web files there’s no place like forge@host:~/store.helpspot.com/current$ ls -lAh drwxrwxr-x

    15 forge forge app -rwxrwxr-x 1 forge forge artisan drwxrwxr-x 3 forge forge bootstrap -rw-rw-r-- 1 forge forge composer.json -rw-rw-r-- 1 forge forge composer.lock
  26. // File /etc/php5/fpm/pool.d/www.conf listen = /var/run/php5-fpm.sock listen.owner = www-data listen.group

    = www-data listen.mode = 0666 user = forge group = forge -rw-rw-rw php-fpm
  27. // File /etc/php5/fpm/pool.d/www.conf listen = 127.0.0.1:9000 listen.owner = www-data listen.group

    = www-data listen.mode = 0666 user = forge group = forge php-fpm
  28. web files there’s no place like forge@host:~/store.helpspot.com/current$ ls -lAh drwxrwxr-x

    15 forge forge app -rwxrwxr-x 1 forge forge artisan drwxrwxr-x 3 forge forge bootstrap -rw-rw-r-- 1 forge forge composer.json -rw-rw-r-- 1 forge forge composer.lock
  29. ACL

  30. pkg managers apt-get & apt sudo apt-get update sudo apt

    update sudo apt-get install whatever sudo apt install whatever
  31. pkg managers search sudo apt search mysql-server ubuntu@host:~$ apt search

    mysql-server mysql-server/trusty-updates,trusty-security 5.5.49-0… mysql-server-5.5/trusty-updates,trusty-security MySQL database server binaries and system database setup mysql-server-5.6/trusty-updates,trusty-security MySQL database server binaries and system database setup
  32. pkg managers show sudo apt show -a \ mysql-server-5.6 Package:

    mysql-server-5.6 Version: 5.6.30-0ubuntu0.14.04.1 Package: mysql-server-5.6 Version: 5.6.16-1~exp1
  33. pkg managers policy sudo apt-cache policy \ mysql-server-5.6 mysql-server-5.6: Installed:

    (none) Candidate: 5.6.30-0ubuntu0.14.04.1 Version table: 5.6.30-0ubuntu0.14.04.1 0 500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty-updates/universe amd64 Packages 500 http://security.ubuntu.com/ubuntu/ trusty-security/universe amd64 Packages 5.6.16-1~exp1 0 500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages
  34. pkg managers ubuntu@host: /etc/apt/sources.list.d $ ls -lah -rw-r--r-- 1 root

    root ondrej-ubuntu-php-xenial.list ubuntu@host: /etc/apt/sources.list.d $ cat \ ondrej-ubuntu-php-xenial.list deb http://ppa.launchpad.net/ondrej/php/ubuntu xenial main # deb-src http://ppa.launchpad.net/ondrej/php/ubuntu xenial main repositories
  35. pkg managers ubuntu@host: ~ sudo apt-key adv --recv-keys --keyserver \

    hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8 ubuntu@host: ~ echo 'deb http://ftp.utexas.edu/mariadb/ repo/10.1/ubuntu xenial main' \ | sudo tee /etc/apt/sources.list.d/mariadb.list manual install
  36. pkg managers ubuntu@host: /etc/apt $ vim sources.list # See http://help.ubuntu.com/community/UpgradeNotes

    for how to upgrade to # newer versions of the distribution. deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial main restricted deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial main restricted # # Major bug fix updates produced after the final release of the # # distribution. deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-updates main restricted deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-updates main \ restricted # # N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu # # team. Also, please note that software in universe WILL NOT receive any # # review or updates from the Ubuntu security team. deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial universe deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial universe included repositories
  37. DNS: ¯\_(ツ)_/¯ ~~ just *TRY* ~~ to coherently explain controlling

    domains to the average [non-tech-client-whoever] challenge:
  38. dig

  39. fideloper@host  ~  vagrant box Usage: vagrant box <subcommand>

    [<args>] Available subcommands: add list outdated remove repackage update boxes (servers)
  40. • ssh by default • can add your own (but

    we’ll do better) port forwarding SSH Default
  41. port forwarding (aside: It’s common to forward to port 80)

    config.vm.network "forwarded_port", guest: 80, host: 8000 $> curl -I localhost:8000 HTTP/1.1 302 Found Server: nginx/1.9.9 Content-Type: text/html; charset=UTF-8 Date: Sat, 02 Jul 2016 17:57:49 GMT Location: http://localhost:8000/login
  42. port forwarding But, two boxes can’t forward to same port!

    config.vm.network "forwarded_port", guest: 80, host: 8000 config.vm.network "forwarded_port", guest: 80, host: 8888 ✅ first box: second box:
  43. sequel pro 2 - SSH Tunnel $> ssh -p 2222

    \ -i /Users/fideloper/…/virtualbox/private_key \ -L 3306:localhost:3306 vagrant@localhost
  44. • 1. Port forwarding (homestead way - easy) • 2.

    Manual SSH tunnel • 3. Sequel Pro SSH Tunnel
  45. sequel pro Remember the SSH Tunnel! You can use it

    in production to view a database.
  46. file sharing config.vm.synced_folder “~/Sites", "/home/vagrant/Sites", id: "core", :nfs => true,

    :mount_options => [‘nolock,vers=3,udp,noatime,actimeo=2,fsc'] network file share handles large # files better
  47. file sharing (I’ve actually used Docker for this instead) docker

    run --rm \ -v ~/Sites/some-project:/opt \ some_node_img:latest \ gulp watch
  48. adding projects 2. Create another server config vagrant@vagrant:/etc/nginx/sites-available$ sudo cp

    \ laravel-a laravel-b vagrant@vagrant:/etc/nginx/sites-available$ sudo vim \ laravel-b server { listen 80; server_name laravel-b.dev; …
  49. adding projects 3. Edit /etc/hosts: 1 ## 2 # Host

    Database 3 # 4 # localhost is used to configure the loopback interface 5 # when the system is booting. Do not change this entry. 6 ## 7 127.0.0.1 localhost 8 255.255.255.255 broadcasthost 9 ::1 localhost 10 11 192.168.33.10 laravel-a.dev laravel-b.dev
  50. adding projects 2. Install DNSMasq brew install dnsmasq cd $(brew

    —prefix) # /usr/local echo 'address=/.dev/192.168.33.10' > etc/dnsmasq.conf sudo cp -v $(brew --prefix dnsmasq) \ homebrew.mxcl.dnsmasq.plist /Library/LaunchDaemons sudo launchctl load -w /Library/LaunchDaemons/ \ homebrew.mxcl.dnsmasq.plist sudo mkdir -p /etc/resolver echo "nameserver 127.0.0.1" | sudo tee /etc/resolver/dev
  51. adding projects 2. DNSMasq continued fideloper@Christophers-iMac  ~  dig

    whatever-i-want.dev \ @127.0.0.1 ;; QUESTION SECTION: ;whatever-i-want.dev. IN A ;; ANSWER SECTION: whatever-i-want.dev. 0 IN A 192.168.33.10
  52. adding projects 3. Magic Nginx Config server { listen 80;

    server_name ~^(.*)\.dev$; set $file_path $1; root /home/vagrant/Sites/$file_path/public; index index.html index.htm index.php; # And so on …
  53. [Bonus] Philosophy • Be ready to throw out a server

    (Ansible) • Docker is not your first answer without ops people