Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Christian Heimes - File descriptors, Unix socke...

Christian Heimes - File descriptors, Unix sockets and other POSIX wizardry

Have you ever wondered how the OS manages open files and network connections, what this 'file descriptor' thing actually is all about, or what's so special about Unix sockets? In my talk I will give you a quick tour into the I/O layer and process model of Unix-like operating systems. You will learn how to securely identify and efficiently share resources between processes.

https://us.pycon.org/2016/schedule/presentation/2019/

PyCon 2016

May 29, 2016
Tweet

More Decks by PyCon 2016

Other Decks in Programming

Transcript

  1. File descriptors, PyCon US 2016 2 Who am I •

    from Hamburg / Germany • Python core contributor since 2008 • PEP 379, 452, 456 • ssl, hashlib • Python Security Response Team • I put bytes and b'' into Python 2.6
  2. File descriptors, PyCon US 2016 3 Professional Life • Senior

    Software Engineer with Red Hat • Security Engineering & Identity Management • FreeIPA IdM • Dogtag PKI
  3. File descriptors, PyCon US 2016 6 Agenda • File descriptor

    • Operating System 101 • File descriptors and processes • Networking and sockets • Unix sockets, containers and sandboxing • Bonus track
  4. File descriptors, PyCon US 2016 11 File descriptors are used

    for • reading / writing files • directories • devices • inter-process communication • network communication • I/O multiplexing • file system monitoring • …
  5. File descriptors, PyCon US 2016 13 Standard numbers • 0:

    standard input (sys.stdin) • 1: standard output (sys.stdout) • 2: standard error output (sys.stderr) • -1: error (Python raises an exception)
  6. File descriptors, PyCon US 2016 14 Simple example with open('example.txt')

    as f: print(f.read()) with open('example.txt') as f: print(f.read())
  7. File descriptors, PyCon US 2016 15 Reference file by descriptor

    with open('example.txt') as f: stat = os.stat(f.fileno()) os.chmod(f.fileno(), 0o640) with open('example.txt') as f: stat = os.stat(f.fileno()) os.chmod(f.fileno(), 0o640)
  8. File descriptors, PyCon US 2016 16 Directory file descriptor flags

    = os.O_RDONLY | os.O_DIRECTORY etcdir = os.open('/etc', flags) # read /etc/passwd stat = os.stat('passwd', dir_fd=etcdir)) os.close(etcdir) flags = os.O_RDONLY | os.O_DIRECTORY etcdir = os.open('/etc', flags) # read /etc/passwd stat = os.stat('passwd', dir_fd=etcdir)) os.close(etcdir)
  9. File descriptors, PyCon US 2016 19 Open CD drive with

    Python import os, fcntl CDROMEJECT = 0x5309 # Linux specific! fd = os.open('/dev/cdrom', os.O_RDONLY | os.O_NONBLOCK) fcntl.ioctl(fd, CDROMEJECT, 0) os.close(fd) import os, fcntl CDROMEJECT = 0x5309 # Linux specific! fd = os.open('/dev/cdrom', os.O_RDONLY | os.O_NONBLOCK) fcntl.ioctl(fd, CDROMEJECT, 0) os.close(fd)
  10. File descriptors, PyCon US 2016 21 Dark Ages application application

    application application disk disk memory memory network network
  11. File descriptors, PyCon US 2016 22 Modern Operating Systems application

    application application application disk disk memory memory network network Kernel Kernel
  12. File descriptors, PyCon US 2016 24 Simple example # read.py

    with open('example.txt') as f: print(f.read()) # read.py with open('example.txt') as f: print(f.read())
  13. File descriptors, PyCon US 2016 25 Simple example $ strace

    ./read.py open("example.txt", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0664, st_size=19, …}) = 0 lseek(3, 0, SEEK_CUR) = 0 read(3, "Python is awesome!\n", 20) = 19 read(3, "", 1) = 0 write(1, "Python is awesome!\n", 19) = 19 write(1, "\n", 1) = 1 close(3) = 0 $ strace ./read.py open("example.txt", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0664, st_size=19, …}) = 0 lseek(3, 0, SEEK_CUR) = 0 read(3, "Python is awesome!\n", 20) = 19 read(3, "", 1) = 0 write(1, "Python is awesome!\n", 19) = 19 write(1, "\n", 1) = 1 close(3) = 0
  14. File descriptors, PyCon US 2016 28 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 open file table entry 23 ... ... ... ... process 2 file descriptor table open 'example.txt'
  15. File descriptors, PyCon US 2016 29 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 fd 4 open file table entry 23 ... entry 117 ... ... process 2 file descriptor table open 'example.txt' a second time
  16. File descriptors, PyCon US 2016 30 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 fd 4 fd 5 open file table entry 23 ... entry 117 ... ... process 2 file descriptor table duplicate file descriptor 4
  17. File descriptors, PyCon US 2016 31 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 fd 4 fd 5 open file table ... ... entry 117 ... ... process 2 file descriptor table rename and close
  18. File descriptors, PyCon US 2016 32 open file / file

    descriptor tables example.txt example.txt process 1 file descriptor table fd 3 fd 4 fd 5 open file table entry 23 ... entry 117 ... ... process 2 file descriptor table fd 7 send file descriptor to another process
  19. File descriptors, PyCon US 2016 33 process and global state

    file descriptor table maps to open file table cloexec flags open file table position mode owner locks credentials reference count ...
  20. File descriptors, PyCon US 2016 36 fork() • creates a

    clone of the current process • inherits copy of file descriptor table • inherits copy of memory • shallow copy with Copy-on-Write
  21. File descriptors, PyCon US 2016 37 fork() example import os

    f = open('example.txt', mode='rb', buffering=0) pid = os.fork() print("getpid {:>5}, fork {:>5}, fd {}: {}".format( os.getpid(), pid, f.fileno(), f.read(6))) f.close() $ python3 fork.py getpid 17364, fork 17365, fd 3: b'Python' getpid 17365, fork 0, fd 3: b' is aw' import os f = open('example.txt', mode='rb', buffering=0) pid = os.fork() print("getpid {:>5}, fork {:>5}, fd {}: {}".format( os.getpid(), pid, f.fileno(), f.read(6))) f.close() $ python3 fork.py getpid 17364, fork 17365, fd 3: b'Python' getpid 17365, fork 0, fd 3: b' is aw'
  22. File descriptors, PyCon US 2016 38 exec() • replaces current

    program code • file descriptors are inherited by default • unless cloexec flag is set • potential security risk • PEP 446 -- Make newly created file descriptors non-inheritable (Victor Stinner)
  23. File descriptors, PyCon US 2016 39 Summary • applications must

    go through the Kernel • syscalls call into kernel space • file descriptors point to global table • global tables contains mode and position • new process with fork() and exec()
  24. File descriptors, PyCon US 2016 43 pipe example import os

    readend, writeend = os.pipe() pid = os.fork() if pid != 0: # parent process os.close(writeend) with open(readend) as f: print(f.read()) else: # child process os.close(readend) os.dup2(writeend, 1) # 1: stdout os.execl('/bin/ls', # programm 'ls', '-l', 'example.txt') # argv 0, 1, 2 import os readend, writeend = os.pipe() pid = os.fork() if pid != 0: # parent process os.close(writeend) with open(readend) as f: print(f.read()) else: # child process os.close(readend) os.dup2(writeend, 1) # 1: stdout os.execl('/bin/ls', # programm 'ls', '-l', 'example.txt') # argv 0, 1, 2 1 2 3 4 6 5 7
  25. File descriptors, PyCon US 2016 47 socket parameters • addressing

    / routing • IPv4: AF_INET • IPv6: AF_INET6 • flow control • TCP: SOCK_STREAM • UDP: SOCK_DGRAM
  26. File descriptors, PyCon US 2016 48 socket server from socket

    import socket, SOCK_STREAM,AF_INET server = socket(AF_INET, SOCK_STREAM) server.bind(('0.0.0.0', 443)) server.listen(1) while True: conn, addr = server.accept() from socket import socket, SOCK_STREAM,AF_INET server = socket(AF_INET, SOCK_STREAM) server.bind(('0.0.0.0', 443)) server.listen(1) while True: conn, addr = server.accept()
  27. File descriptors, PyCon US 2016 49 socket client from socket

    import (socket, SOCK_STREAM, AF_INET, AF_INET6) cl4 = socket(AF_INET, SOCK_STREAM) cl4.connect(('104.130.43.121', 443)) cl6 = socket(AF_INET6, SOCK_STREAM) cl6.connect(('2001:4802:7901:0:e60a:1375:0:5', 443)) from socket import (socket, SOCK_STREAM, AF_INET, AF_INET6) cl4 = socket(AF_INET, SOCK_STREAM) cl4.connect(('104.130.43.121', 443)) cl6 = socket(AF_INET6, SOCK_STREAM) cl6.connect(('2001:4802:7901:0:e60a:1375:0:5', 443))
  28. File descriptors, PyCon US 2016 53 Unix sockets from socket

    import (socket, socketpair, SOCK_STREAM, AF_UNIX) server = socket(AF_UNIX, SOCK_STREAM) server.bind('/path/to/file') client = socket(AF_UNIX, SOCK_STREAM) client.connect('/path/to/file') a, b = socketpair() from socket import (socket, socketpair, SOCK_STREAM, AF_UNIX) server = socket(AF_UNIX, SOCK_STREAM) server.bind('/path/to/file') client = socket(AF_UNIX, SOCK_STREAM) client.connect('/path/to/file') a, b = socketpair()
  29. File descriptors, PyCon US 2016 54 peer credentials import socket,

    struct def getpeercred(sock): fmt = "iII" size = struct.calcsize(fmt) raw = sock.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, size) pid, uid, gid = struct.unpack(fmt, raw) return pid, uid, gid >>> getpeercred(uds) (31362, 0, 0) import socket, struct def getpeercred(sock): fmt = "iII" size = struct.calcsize(fmt) raw = sock.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, size) pid, uid, gid = struct.unpack(fmt, raw) return pid, uid, gid >>> getpeercred(uds) (31362, 0, 0)
  30. File descriptors, PyCon US 2016 55 peer security context import

    socket SO_PEERSEC = getattr(socket, 'SO_PEERSEC', 31) def getpeersec(sock): raw = sock.getsockopt(socket.SOL_SOCKET, SO_PEERSEC, 256) return raw.rstrip(b'\x00').decode('utf-8') >>> getpeersec(uds) 'system_u:system_r:svirt_lxc_net_t:s0:c560,c872' import socket SO_PEERSEC = getattr(socket, 'SO_PEERSEC', 31) def getpeersec(sock): raw = sock.getsockopt(socket.SOL_SOCKET, SO_PEERSEC, 256) return raw.rstrip(b'\x00').decode('utf-8') >>> getpeersec(uds) 'system_u:system_r:svirt_lxc_net_t:s0:c560,c872'
  31. File descriptors, PyCon US 2016 56 Unix sockets and containers

    • Unix sockets work between containers • SELinux! • Kernel translates PID name space • pid 31362 • Multi-Category Security (MCS) Separation • system_u:system_r:svirt_lxc_net_t:s0:c560,c872
  32. File descriptors, PyCon US 2016 57 Docker # cat /proc/31336/cgroup

    7:pids:/system.slice/docker-de20f4cb....scope # docker inspect de20f4cb... ... "ProcessLabel": "system_u:system_r:svirt_lxc_net_t:s0:c560,c872", ... "Config": { "Labels": { "io.kubernetes.container.name": "custodia-pwmgr", "io.kubernetes.pod.name": "custodia-pwmgr-tqxtz", "io.kubernetes.pod.namespace": "default", ... }} # cat /proc/31336/cgroup 7:pids:/system.slice/docker-de20f4cb....scope # docker inspect de20f4cb... ... "ProcessLabel": "system_u:system_r:svirt_lxc_net_t:s0:c560,c872", ... "Config": { "Labels": { "io.kubernetes.container.name": "custodia-pwmgr", "io.kubernetes.pod.name": "custodia-pwmgr-tqxtz", "io.kubernetes.pod.namespace": "default", ... }}
  33. File descriptors, PyCon US 2016 58 sending file descriptors #

    sender with open(filename) as f: send_fds(sender, b'\x01', [f.fileno()]) # receiver msg, fds = recv_fds(rcv, msglen=1, maxfds=1) with open(fds[0]) as f: f.read() # sender with open(filename) as f: send_fds(sender, b'\x01', [f.fileno()]) # receiver msg, fds = recv_fds(rcv, msglen=1, maxfds=1) with open(fds[0]) as f: f.read()
  34. File descriptors, PyCon US 2016 59 seccomp sandboxing syscall filter

    OpenSSH, Tor, Firefox media plugins, Google Chrome sandbox disk disk network network broker broker • read, write, close • open, socket, unlink • fork, exec
  35. File descriptors, PyCon US 2016 63 Efficient I/O with zero-copy

    • sendfile() • copy_file_range() • splice() / vmsplice() / tee() • AF_KTLS (Kernel space TLS)