Terminal Awareness with notes

Just a general talk on being aware of how to work in the terminal, some basic scripting knowledge, and some tips, tricks and tools I picked up in about a year and a half of working nearly exclusively in a terminal

Transcript

1. Shell Scripting Lessons from a year administering remote machines
2. Basic Concepts
3. Shell Anatomy 101 • time php test.php 2>&1 | grep $NEEDLE > results.txt Shell Builtin Executables Arguments Redirection Pipe Environment Variable output builtin: a regular command, except provided by your shell instead of an executable executables: a file in your path marked as executable arguments: you all know what this means Redirection: Swapping around the standard streams
4. Process Pipes, a review Process stdin stdout stderr 2 1
5. 2>&1 Process stdin stdout stderr 2 1
6. Shell Anatomy 101 • time php test.php 2>&1 | grep $NEEDLE > results.txt Shell Builtin Executables Arguments Redirection Pipe Environment Variable output Pipe: Special case of redirection, connects stdout to stdin of the other side Environment variable: a variable that gets the value substituted by the shell
7. Tools to Know About
8. man • read the manual page for a given command • $ man grep info also exists, but that means using emacs-like bindings… ugh!
9. cat • Puts the contents of a file onto stdout • $ cat “hello.txt”
 Hello, world
10. head/tail • Puts the beginning or end of a file on stdout • great for dealing with logs, especially with -f • $ tail -f /var/log/system.log
 Hello, world
11. ps • list information about processes • $ ps axu | grep grep
 bnicholas 4521 0.0 0.0 2453264 692 s000 S+ 10:26PM 0:00.00 grep grep
12. kill/killall • send a signal to processes by pid or name • $ kill -SIGUSR1 2048 • $ killall perl
13. sort • Takes lines from stdin and sorts them on stdout • $ cat file.txt | sort • $ cat numbers.txt | sort -n
14. uniq • Removes duplicate lines from stdin to stdout • $ cat file.txt | sort | uniq • $ cat ~/history | sort | uniq -c | sort -n
15. grep • Search for a regular expression • can be used on stdin or files • $ cat /var/log/product.log | grep -i “error”
16. tr • “trim” text, replacing characters with others • $ cat sentence.txt | tr ‘ ‘ ‘\n’
 My
 name
 is
 ben.
17. sed • stream editor: a mini text editor • Similar to vim find+replace syntax, but so much more too • $ echo “hello world!” | sed -e “s/ w.*d/bn.u/“
 hello bn.u!
18. find • A file finder with extra • $ find . -name “*.orig” -print - delete • $ find . -type f —mtime -1 -print This will find all the files ending in .orig (merge files from git), print them out, and then delete them exec also allows for running any command selectors let you pick files very concisely
19. awk • a small programming language for dealing with columns • ps aux | grep grep | awk ‘{print $2}’
 6845 So much more complicated and powerful than I have time for I’ve only scratched the surface, but it’s really helpful for certain things this example prints the pid of any grep processes
20. Control Flow
21. conditionals • if [ $VAR = “foo” ]
 then
 echo “bar”
 else
 echo “baz”
 fi explain test, and how dumb it can be sometimes -eq for integer equality, -gt -lt -ne…
22. Better conditionals • Use && and || when possible • cat a.txt | grep “hello” && echo “world”
23. For loops • for FILE in `ls`
 do
 cat $FILE
 done
24. SSH The magic glue that holds things together
25. Key-based auth • More secure and convenient than passwords • ssh-copy-id <username>@<host> • Last time you’ll need to type a password • Also applies to git, scp, sftp, and others
26. ~/.ssh/config • Set up usernames across computers • Keep connections alive longer • Set up tunnels tunnels: local ports actually go through to the remote server good things like accessing servers that are firewall’d out
27. Great Tricks/Tools
28. autocomplete stop typing and hit tab. 95% of the time it’s gonna do what you want. trust your shell.
29. fish If you’re still using bash, stop it, install fish, make all of your config 100000x easier autocomplete by default, web config, syntax highlighting and checking, sane scripting, small config files (instead of one monolithic _THING_), man page completion
30. ack http://beyondgrep.com it’s faster grep that ignores the files you don’t want. Simple as that.
31. tmux If you’re working on a remote machine, this will save you so many times (worked on a laptop, connection on and off, long running tasks that never died)
32. vim This is going to start a war, I know, but knowing a command line editor helps a lot vim is the one of the most powerful, nano, pico, emacs all work, but why?
33. dsh Elliot and James swear by it, I don’t know if it’s worth the config time, but it depends on how you work
34. Some example problems