Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CLIFuzzer: Mining Grammars for Command-Line Inv...

CLIFuzzer: Mining Grammars for Command-Line Invocations

ESEC/FSE 2022

Rahul Gopinath

May 23, 2022
Tweet

More Decks by Rahul Gopinath

Other Decks in Research

Transcript

  1. CLIFuzzer: Mining Grammars for Commandline Invocations Rahul Gopinath Abhilash Gupta

    Andreas Zeller CISPA Helmholtz Center for Information Security CISPA Helmholtz Center for Information Security University of Sydney
  2. CLIFuzzer: Mining Grammars for Commandline Invocations Rahul Gopinath Abhilash Gupta

    Andreas Zeller CISPA Helmholtz Center for Information Security CISPA Helmholtz Center for Information Security University of Sydney
  3. 3 Command Line Utility $ ls -a --color=always . Google

    Chrome.app Microsoft Word.app TextMate.app .. Google Docs.app Numbers.app UTM.app .DS_Store Google Drive.app OneDrive.app Utilities .localized Google Sheets.app Opera.app VMware Fusion Tech Pr Barrier.app Google Slides.app Pages.app VMware Fusion.app Cisco Keynote.app Parallels Desktop.app Visual Studio Code.ap Cisco Jabber.app LanguageTool.app Piezo.app Zotero.app Dato.app Microsoft Edge.app Safari.app iMovie.app DiffusionBee.app Microsoft Excel.app Self Service.app scanserver.app Docker.app Microsoft OneNote.app Skype.app zoom.us.app Firefox.app Microsoft Outlook.app Slack.app Free Ruler.app Microsoft PowerPoint.app Stats.app GarageBand.app Microsoft Teams.app TeX
  4. 4 Command Line Utility $ ls -a --color=always . Google

    Chrome.app Microsoft Word.app TextMate.app .. Google Docs.app Numbers.app UTM.app .DS_Store Google Drive.app OneDrive.app Utilities .localized Google Sheets.app Opera.app VMware Fusion Tech Pr Barrier.app Google Slides.app Pages.app VMware Fusion.app Cisco Keynote.app Parallels Desktop.app Visual Studio Code.ap Cisco Jabber.app LanguageTool.app Piezo.app Zotero.app Dato.app Microsoft Edge.app Safari.app iMovie.app DiffusionBee.app Microsoft Excel.app Self Service.app scanserver.app Docker.app Microsoft OneNote.app Skype.app zoom.us.app Firefox.app Microsoft Outlook.app Slack.app Free Ruler.app Microsoft PowerPoint.app Stats.app GarageBand.app Microsoft Teams.app TeX $ command [configuration options][arguments]
  5. $ ls xldjafljdj;jfafiioequreqrin,mnewioqr;e3kekjdfjdafj ls: xldjafljdj: No such file or directory

    zsh: exit 1 ls xldjafljdj zsh: command not found: jfafiioequreqrin,mnewioqr zsh: exit 127 jfafiioequreqrin,mnewioqr zsh: command not found: e3kekjdfjdafj zsh: exit 127 e3kekjdfjdafj
  6. 9 {'<start>': [ '(<option>)*<arguments>'], '<option>': [' -h', ' --help', '

    --version', ' -v', ' --verbose'], '<arguments>': [ ' foo.py'], '<str>': ['<char>+'], '<char>': [ /0-9/, /a-z/, /A-Z/ '[', '\\', ']', '^', '_', '`', '{','|','}', '~'], '<filename>': ['<str>'], '<int>': ['(-)?<digit>+'], '<digit>': [/0-9/], '<n>': ['<int>'], '<globs>': ['<str>'], '<errors>': ['<str>'], '<line>': ['<int>']} Getopt to Context Free Grammar
  7. 12 Evaluation ▪ 44 utilities in Ubuntu 20.04.3 LTS ▪

    Use getopt, getopt_long or getopt_long_only to parse their options ▪ Take in a file or stdin as argument ▪ Test inputs ▪ 60 generated files of large lengths (~ 100KB and 10MB) using different seed values ▪ 3 large publicly available text files* ▪ 3 large publicly available media (1 image and 2 audio) files as bc bison cat col colcrt column colrm comm cmp cut dc diff expand fmt fold gdb grep head join look m4 nl nm od paste pr ptx rev sdiff spell strings strip sort tac tail tee tr troff tsort unexpand uniq wc xargs
  8. 22 37 41 85 11 39 80 32 34 58

    29 17.31 30.89 26.06 83.32 10.51 30.34 56.86 16.49 31.11 49.84 29.70 0.00 22.50 45.00 67.50 90.00 as bison column dc gdb ptx spell tac tee troff tsort % coverage achieved CLI Utility CLIFuzzer AFL++ Fig: Coverage achieved by AFL++ and MyFuzzer on different utilities that report errors* * -> AFL++ ran for 3 hours. CLIFuzzer ran 3000 invocations. CLIFuzzer took 1-1.5 hours to run.
  9. 14 as bc bison cat col colcrt column colrm comm

    cmp cut dc diff expand fmt fold gdb grep head join look m4 nl nm od paste pr ptx rev sdiff spell strings strip sort tac tail tee tr troff tsort unexpand uniq wc xargs column (v2.37.2) tac (v9.0) tee (v9.0) tsort (v9.0) * * = crash, = hang as (v2.37) bison (v3.8) dc (v1.41) gdb (v11.1) ptx (v9.0) spell (v1.1) troff (v1.22.4) * * * Parameter Interactions General Failures
  10. 15

  11. 16