How the Command-Line Can Give You Superpowers (GDG Toronto Android)

I want to show you how the command line can simplify, and even automate some of the things Android developers need to do. Learning how to use command line tools will give you more ways to get your work done.

Using Mac OS X as the platform, I’ll demonstrate real-world examples of how you can boost your productivity when navigating, searching, working with JSON and Web responses, manipulating images, automating interaction with devices, scripting Java and Kotlin code generation, and more. All from a terminal, without using a GUI

Note that the slides don't include the videos/animations.

Video recording: https://www.youtube.com/watch?v=YMS_AplpmZg

Transcript

1. How The Command-Line Can Give You Superpowers Eric Fung GDG Toronto Android, 2018-08-09
2. Both screenshots [Public domain], via Wikimedia Commons
3. Linux screenshot © 2011, Anderson Silva. Released under the Open Publication License — gcc screenshot [Public domain], via Wikimedia Commons
4. Why Learn the Command-Line? — Gives you more ways to solve problems — Learn tools to get results quickly — Makes you more valuable — Feel like a superhero How The Command-Line Can Give You Superpowers — @gnufmuffin
5. Overview — Setup — Navigation — Search — I/O — JSON — Web — Images — Moving Things Around — Scripting How The Command-Line Can Give You Superpowers — @gnufmuffin
6. Overview (of superpowers) — Setup — Navigation Teleportation — Search Locality Tracking — I/O Information Transferal — JSON Knowledge Manipulation — Web Arachnid Physiology — Images Skill Replication — Moving Things Around Telekinesis — Scripting Automatic Tasking How The Command-Line Can Give You Superpowers — @gnufmuffin
7. Setup How The Command-Line Can Give You Superpowers — @gnufmuffin
8. Terminal Emulator — Get it: https://iterm2.com/ How The Command-Line Can Give You Superpowers — @gnufmuffin
9. ⌘-click URL to open it Click on filenames too How The Command-Line Can Give You Superpowers — @gnufmuffin
10. Highlight all matches Regex support How The Command-Line Can Give You Superpowers — @gnufmuffin
11. ⌘-; to autocomplete From anything that has ever appeared How The Command-Line Can Give You Superpowers — @gnufmuffin
12. Command line shell — macOS default shell — may want to install newer version How The Command-Line Can Give You Superpowers — @gnufmuffin
13. Package manager — GNU Linux/UNIX software — Get it: https://brew.sh/ How The Command-Line Can Give You Superpowers — @gnufmuffin
14. Homebrew — brew install — Read caveats and post-install notes! — brew update — brew upgrade How The Command-Line Can Give You Superpowers — @gnufmuffin
15. Homebrew — brew uninstall — brew cleanup — Remove old versions to free up disk How The Command-Line Can Give You Superpowers — @gnufmuffin
16. Completion — Press <Tab> instead of typing — Context sensitive: command, subcommand, options — Get it: $ brew install bash_completion How The Command-Line Can Give You Superpowers — @gnufmuffin
17. adb completion — Completes install with APKs — Completes uninstall with application IDs — Get it: — https://android.googlesource.com/ platform/system/core/+/master/adb/ adb.bash — https://android.googlesource.com/ platform/build/+/master/envsetup.sh How The Command-Line Can Give You Superpowers — @gnufmuffin
18. adb install completion How The Command-Line Can Give You Superpowers — @gnufmuffin
19. adb uninstall completion How The Command-Line Can Give You Superpowers — @gnufmuffin
20. Gradle completion — Works with standalone and wrapper — Maintains a cache — Implicit tasks disabled by default — Get it: brew install gradle-completion How The Command-Line Can Give You Superpowers — @gnufmuffin
21. Gradle completion How The Command-Line Can Give You Superpowers — @gnufmuffin
22. To Finder — open <thing> — Equivalent to double-clicking on <thing> — open image.png — Opens image in Preview — open . — Opens Finder with current directory How The Command-Line Can Give You Superpowers — @gnufmuffin
23. open -R <file> — Opens enclosing folder in Finder, and highlights file — Handy for dragging files to upload, e.g. Slack How The Command-Line Can Give You Superpowers — @gnufmuffin
24. open -R <file> How The Command-Line Can Give You Superpowers — @gnufmuffin
25. open -a <app> <file> — Open file with a specific application — Useful if file does not have default handler — Tip: define a function or alias asopen() { open -a "/Applications/Android Studio.app" "$1" } How The Command-Line Can Give You Superpowers — @gnufmuffin
26. From Finder — Drag-and-drop file(s) onto Terminal — Pastes their filename(s), space-separated How The Command-Line Can Give You Superpowers — @gnufmuffin
27. Drag files to shell → paste filename How The Command-Line Can Give You Superpowers — @gnufmuffin
28. Navigation How The Command-Line Can Give You Superpowers — @gnufmuffin
29. cd path/to/file — Instead of cd path1㾑 cd path2 㾑 cd path3㾑 — use path separators cd path1/path2/path3 How The Command-Line Can Give You Superpowers — @gnufmuffin
30. cd - — Instead of $ pwd /Users/efung/src/github.com/example/project/app/src/main/java $ cd $ANDROID_SDK $ <do some stuff> $ cd ~/src/github.com/example/project/app/src/main/java — jump to previous directory $ cd $ANDROID_SDK $ <do some stuff> $ cd - How The Command-Line Can Give You Superpowers — @gnufmuffin
31. Beyond cd — Reduce typing — Provide quicker access to folders further away — Intelligently select folders based on "frecency" How The Command-Line Can Give You Superpowers — @gnufmuffin
32. Beyond cd — autojump: https://github.com/wting/autojump — z: https://github.com/rupa/z — fasd: https://github.com/clvv/fasd — Get it: brew install {autojump,fasd,z} How The Command-Line Can Give You Superpowers — @gnufmuffin
33. autojump — j name — Go to most-used folder containing name — jc name — Go to a subdirectory containing name — j name <Tab> — Offers choices when there are multiple matches How The Command-Line Can Give You Superpowers — @gnufmuffin
34. autojump How The Command-Line Can Give You Superpowers — @gnufmuffin
35. Searching How The Command-Line Can Give You Superpowers — @gnufmuffin
36. Beyond grep — The Silver Searcher: https://github.com/ggreer/ the_silver_searcher — ripgrep: https://github.com/BurntSushi/ripgrep — Get it: brew install {the_silver_searcher,ripgrep} How The Command-Line Can Give You Superpowers — @gnufmuffin
37. The Silver Searcher — Really fast — Builtin file types — Ignores files from .gitignore How The Command-Line Can Give You Superpowers — @gnufmuffin
38. ag <term> How The Command-Line Can Give You Superpowers — @gnufmuffin
39. ag <term> --<filetype> -w How The Command-Line Can Give You Superpowers — @gnufmuffin
40. Search Android documentation offline ag <term> $ANDROID_SDK/docs How The Command-Line Can Give You Superpowers — @gnufmuffin
41. Spotlight (⌘-Space) — mdfind: Search metadata — mdls: List known attributes — e.g. show where an APK was downloaded from $ mdls -name kMDItemWhereFroms 1850000.apk kMDItemWhereFroms = ( "https://play.google.com/apps/publish/artifact/download?artifact=12 34737584826475678", "https://play.google.com/apps/publish/?account=9876232438554103210" ) How The Command-Line Can Give You Superpowers — @gnufmuffin
42. I/O How The Command-Line Can Give You Superpowers — @gnufmuffin
43. I/O Review — Streams: stdin and stdout — Redirection: > and < — Pipelines: | How The Command-Line Can Give You Superpowers — @gnufmuffin
44. Pasteboard (⌘-C, ⌘-V) — pbpaste — Copies contents of pasteboard, writes to stdout — pbcopy — Reads from stdin, copies to pasteboard How The Command-Line Can Give You Superpowers — @gnufmuffin
45. Copying and pasting from StackOverflow — Format code snippet — Write to file $ pbpaste | tidy -xml > file.xml How The Command-Line Can Give You Superpowers — @gnufmuffin
46. Copying and pasting from StackOverflow How The Command-Line Can Give You Superpowers — @gnufmuffin
47. JSON How The Command-Line Can Give You Superpowers — @gnufmuffin
48. gron — Turns JSON data into line-oriented data — Then, search with line-oriented tools, like ag — Identify individual array elements — Get it: brew install gron How The Command-Line Can Give You Superpowers — @gnufmuffin
49. Sample flickr.photos.search response { "photos": { "page": 1, "pages": "25074", "perpage": 7, "total": "250739", "photo": [ { "id": "29012739647", "owner": "159732495@N03", "secret": "39dd96b21d", "server": "1840", "farm": 2, "title": "BombSquad MOD (Unlocked)", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43231929164", "owner": "159732495@N03", "secret": "56fd0acc30", "server": "1836", "farm": 2, "title": "MARVEL Strike Force MOD (Unlimited Energy)", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43950331011", "owner": "7171588@N05", "secret": "c45ba149c1", "server": "1797", "farm": 2, "title": "New photo added to gallery", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43950279251", "owner": "7171588@N05", "secret": "b9dc92ff2b", "server": "1836", "farm": 2, "title": "New photo added to gallery", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43231327754", "owner": "150238411@N07", "secret": "dc3af72ee3", "server": "851", "farm": 1, "title": "New photo added to gallery", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43901777582", "owner": "125141703@N02", "secret": "e76813d131", "server": "1775", "farm": 2, "title": "Samsung Galaxy Note 9, así encaja en el catálogo completo de smartphones Samsung en 2018", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "42140715770", "owner": "159732495@N03", "secret": "6b7156103c", "server": "1837", "farm": 2, "title": "WeTorrent Premium – Torrent Downloader (Cracked)", "ispublic": 1, "isfriend": 0, "isfamily": 0 } ] }, "stat": "ok" } How The Command-Line Can Give You Superpowers — @gnufmuffin
50. Example gron output How The Command-Line Can Give You Superpowers — @gnufmuffin
51. jq — Powerful, flexible processor — Hard to learn — Massage data into other formats — Get it: brew install jq How The Command-Line Can Give You Superpowers — @gnufmuffin
52. Pretty printing $ jq '.' flickr.json { "photos": { "page": 1, "pages": "25074", "perpage": 7, "total": "250739", "photo": [ { "id": "29012739647", "owner": "159732495@N03", "secret": "39dd96b21d", "server": "1840", "farm": 2, "title": "BombSquad MOD (Unlocked)", }}} How The Command-Line Can Give You Superpowers — @gnufmuffin
53. Extract list of values $ jq -r '.photos.photo | .[] | .title' flickr.json BombSquad MOD (Unlocked) MARVEL Strike Force MOD (Unlimited Energy) New photo added to gallery New photo added to gallery New photo added to gallery Samsung Galaxy Note 9, así encaja en el catálogo … WeTorrent Premium – Torrent Downloader (Cracked) How The Command-Line Can Give You Superpowers — @gnufmuffin
54. Create array of numeric IDs $ jq '.photos.photo | [.[].id | tonumber]' flickr.json [ 29012739647, 43231929164, 43950331011, 43950279251, 43231327754, 43901777582, 42140715770 ] How The Command-Line Can Give You Superpowers — @gnufmuffin
55. Web How The Command-Line Can Give You Superpowers — @gnufmuffin
56. cURL — Comprehensive client for working with URLs — "the number of features will make your head spin!" — Documented in a new book https://bookcurl.haxx.se/ — Has extensive support of protocols: HTTP, SMTP, IMAP, RTSP, etc. — Get it: built-in How The Command-Line Can Give You Superpowers — @gnufmuffin
57. Debugging endpoints: --include $ curl 'https://search.icons8.com/api/iconsets/v4/search' --include HTTP/2 200 server: keycdn-engine date: Thu, 09 Aug 2018 19:39:52 GMT content-type: application/json; charset=utf-8 content-length: 96 cache-control: max-age=604800 expires: Thu, 16 Aug 2018 19:39:52 GMT x-cache: HIT x-shield: active x-edge-location: usch access-control-allow-origin: * accept-ranges: bytes {"parameters":{"amount":"25","offset":"0","impresser_preview":false, "language":"en"},"icons":[]} How The Command-Line Can Give You Superpowers — @gnufmuffin
58. Debugging Firebase Cloud Messaging — Typically, app server initiates push — Console can only send notification messages — Can simulate server from command-line — More details in the Firebase docs How The Command-Line Can Give You Superpowers — @gnufmuffin
59. Sending a push notification $ curl \ --header "Authorization: key=AJzx8oIelephanT4gELATOi99elkbffevOon03F" \ --header "Content-Type: application/json" \ --data payload.json \ https://fcm.googleapis.com/fcm/send { "name": "projects/myproject-b5ae1/messages/0:1500415314455276%31bd1c9631bd1c96" } — Use legacy authentication to avoid OAuth headaches — Exercise: Wrap in function and parameterize How The Command-Line Can Give You Superpowers — @gnufmuffin
60. Generating data model from JSON — quicktype https://app.quicktype.io/ generates code from JSON — Rapidly integrate with an unfamiliar API — Get sample JSON with cURL, then paste into website How The Command-Line Can Give You Superpowers — @gnufmuffin
61. How The Command-Line Can Give You Superpowers — @gnufmuffin
62. Images How The Command-Line Can Give You Superpowers — @gnufmuffin
63. Image Considerations — Supporting different pixel densities — Converting from XML to VectorDrawable — Minimizing APK size How The Command-Line Can Give You Superpowers — @gnufmuffin
64. ImageMagick — Suite of utilities — Read/write 200 image formats — Any kind of transformation you can think of — Best to learn by example — Get it: brew install imagemagick How The Command-Line Can Give You Superpowers — @gnufmuffin
65. ImageMagick — compare — annotate differences — convert — convert file format, apply transformations — identify — describe format and attributes — montage — combine images into another image — Watch out for order of options, punctuation How The Command-Line Can Give You Superpowers — @gnufmuffin
66. Example: Include before/after in pull requests How The Command-Line Can Give You Superpowers — @gnufmuffin
67. Shell script calling IM #!/bin/bash beforeafter() { montage -background lightgray -pointsize 60 -label '%t' $* \ -tile x1 -geometry '+30+30' out.png } — Define beforeafter function — $* is all arguments — %t is IM syntax for each argument's filename in label How The Command-Line Can Give You Superpowers — @gnufmuffin
68. Calling the script $ beforeafter before.png after.png How The Command-Line Can Give You Superpowers — @gnufmuffin
69. Pull Request How The Command-Line Can Give You Superpowers — @gnufmuffin
70. Example: Crop bitmaps across density sizes How The Command-Line Can Give You Superpowers — @gnufmuffin
72. Shell script calling IM's montage $ for r in m h xh xxh; do \ convert -gravity SouthWest -crop 100%x40%+0+0 \ android/onboarding/src/main/res/drawable-${r}dpi/background.jpg \ android/onboarding/src/main/res/drawable-${r}dpi/background_out.jpg; \ done — bash for loop to iterate the density prefixes — Using percentage avoids calculating pixel size per density — "Crop 40% of vertical (100% of horizontal) starting from (0,0)" — Visually check the *_out.jpg files before copying over originals How The Command-Line Can Give You Superpowers — @gnufmuffin
73. Image optimization — Reduce file size — pngcrush https://pmt.sourceforge.io/pngcrush/ — svgo https://github.com/svg/svgo — Get it: brew install {pngcrush,svgo} How The Command-Line Can Give You Superpowers — @gnufmuffin
74. Input SVG How The Command-Line Can Give You Superpowers — @gnufmuffin
75. Apply svgo $ svgo --precision=1 butterfly.svg -o optimized.svg butterfly.svg: Done in 221 ms! 115.713 KiB - 19.2% = 93.482 KiB — Much smaller! — But does it look the same? How The Command-Line Can Give You Superpowers — @gnufmuffin
76. Output SVG How The Command-Line Can Give You Superpowers — @gnufmuffin
77. Comparing images visually with IM's compare $ compare butterfly.svg optimized.svg svgo-compare.png $ compare butterfly.png optimized.png png-diff.png How The Command-Line Can Give You Superpowers — @gnufmuffin
78. Moving Things Around How The Command-Line Can Give You Superpowers — @gnufmuffin
79. Install APKs wirelessly — APKs can be installed over WiFi (see Google's adb over WiFi) — Process seems cumbersome — Why? Some devices block USB loading unless you sign up! How The Command-Line Can Give You Superpowers — @gnufmuffin
80. qr-filetransfer — Starts a local webserver, with randomized URL hosting your file — Generates QR code in terminal that points to that URL — Get it: https://github.com/claudiodangelis/qr- filetransfer How The Command-Line Can Give You Superpowers — @gnufmuffin
81. qr-filetransfer <file> How The Command-Line Can Give You Superpowers — @gnufmuffin
82. Demo How The Command-Line Can Give You Superpowers — @gnufmuffin
83. Scripting How The Command-Line Can Give You Superpowers — @gnufmuffin
84. Automating soft keyboard and touches — adb shell input text <text> — Sends keycodes to device — adb shell input tap <x> <y> — Sends touch events to device — Combine these into a shell script How The Command-Line Can Give You Superpowers — @gnufmuffin
85. Shell script to perform a login function adb_login_admin() { for field in "$@" do adb shell input text "$field" adb shell input keyevent KEYCODE_ENTER done adb shell input tap 760 50 } — Simple username/password — Inputs each argument then presses Enter to advance — Taps on top-right corner (login button) How The Command-Line Can Give You Superpowers — @gnufmuffin
86. Usage $ adb_login_admin username password — Caveats: may need to insert sleep <seconds> calls — Exercise: Correctly handle screen density on taps How The Command-Line Can Give You Superpowers — @gnufmuffin
87. Generating Kotlin/Java code — Automate repetitive editing — Turn CSV, list, etc. into code — Doesn't have to be 100% syntactically correct How The Command-Line Can Give You Superpowers — @gnufmuffin
88. Input CSV CSS font name,Font name,Category,Weight,Style,Friendly name,File name AnonymousPro 400,Anonymous Pro,Block,400,normal,Anonymous,AnonymousPro-Regular AnonymousPro 700,Anonymous Pro,Block,700,normal,Anonymous Bold,AnonymousPro-Bold Cousine 400,Cousine,Block,400,normal,Cousine,Cousine-Regular Cousine 700,Cousine,Block,700,normal,Cousine Bold,Cousine-Bold … How The Command-Line Can Give You Superpowers — @gnufmuffin
89. Parse a CSV using Ruby File.open(font_list_filename, 'r') do |file| font_list = CSV.new(file, { :headers => true, :return_headers => false }) while row = font_list.shift slug = row['CSS font name'].slugify friendly_name = row['Friendly Name'] font_names[slug] = friendly_name font_css_names[slug] = row['CSS font name'] end end — Build up two hashes, mapping their identifier to their friendly name and CSS name — e.g. { "font_cousine400" => "Cousine 400" } How The Command-Line Can Give You Superpowers — @gnufmuffin
90. Generate Kotlin enum def android_kotlin_enum(font_names, font_css_names) val = "enum class FontName(val cssName: String, @StringRes val resId: Int) {\n" val += font_names.keys.map {|k| " #{k.upcase}(\"#{font_css_names[k]}\", R.string.font_#{k})" }.join(",\n") val += ";\n" val += <<END companion object { fun find(cssName: String): FontName? = values().find {it.cssName == cssName} } } END puts val end How The Command-Line Can Give You Superpowers — @gnufmuffin
91. Output enum class FontName(val cssName: String, @StringRes val resId: Int) { ANONYMOUSPRO400("AnonymousPro 400", R.string.font_anonymousPro400), ANONYMOUSPRO700("AnonymousPro 700", R.string.font_anonymousPro700), COUSINE400("Cousine 400", R.string.font_cousine400), COUSINE700("Cousine 700", R.string.font_cousine700), … companion object { fun find(cssName: String): FontName? = values().find {it.cssName == cssName} } } — Not a complete Kotlin class file (but very close) — But the laborious parts were automated How The Command-Line Can Give You Superpowers — @gnufmuffin
92. Idea: Automate generation of Google Play Billing CSV — Get products from "source of truth", e.g. Firebase — Convert or parse data (jq, script) — Generate CSV How The Command-Line Can Give You Superpowers — @gnufmuffin
93. Further Exploration How The Command-Line Can Give You Superpowers — @gnufmuffin
94. Further Exploration — Install more packages from Homebrew (4000+) — Read the documentation of your terminal and shell — Learn a keyboard-oriented text editor — Learn the basics of a high-level scripting language How The Command-Line Can Give You Superpowers — @gnufmuffin
95. End — Email efung@acm.org — Blog code.gnufmuffin.com — Code github.com/efung — Social @gnufmuffin — Slides speakerdeck.com/efung How The Command-Line Can Give You Superpowers — @gnufmuffin