Software Licenses and Interpreted Languages

Given at the Open Science Grid User School 2016
https://twiki.opensciencegrid.org/bin/view/Education/OSGUserSchool2016

Transcript

1. Licenses and Interpreted Languages for DHTC Thursday morning, 10:45 am Christina Koch (ckoch5@wisc.edu) Research Computing Facilitator University of Wisconsin - Madison
2. OSG User School 2016 Expanding Our Horizons •  Our previous examples had two things in common: - Software was unlicensed - Code could be compiled or otherwise installed and then run •  This presentation discusses: - Licensed software - Running interpreted languages (Matlab, Python) on a DHTC system 2
3. OSG User School 2016 LICENSING 3
4. OSG User School 2016 Licensing •  Many scientific softwares are licensed. •  Licenses are restrictive, particularly for high-throughput computing 4
5. OSG User School 2016 License Variations •  Per machine or 'single-install’ •  Per running instance of the software (per “job”) •  Per username / user •  Via a license server - can support 1 - 1000s of concurrently running processes (“seats”) 5
6. OSG User School 2016 Licensing implications for DHTC •  Per machine or 'single-install': can’t be used for DHTC •  Per job: restrictive, limits the number of jobs you can have running, how do you access licenses from execute servers? •  Username: restrictive, could only run jobs on one system where your jobs run as *your username* 6
7. OSG User School 2016 Approaches •  Seek out open source alternatives - Python or R packages that emulate specific software behavior - If you can’t replace entire workflow, substitute free software where you can •  License-free workarounds (Matlab) •  Choose the least restrictive license possible 7
8. OSG User School 2016 INTERPRETED LANGUAGES 8
9. OSG User School 2016 Interpreted code •  Instead of being compiled and then run… •  …interpreted languages are translated into binary code “on the fly” 9
10. OSG User School 2016 Interpretation 10 Script Interpreter Libraries text turns into binary instructions uses
11. OSG User School 2016 On the command line 11
12. OSG User School 2016 Common interpreted languages* •  Python •  R •  Julia •  Ruby •  Matlab •  Perl •  Javascript 12 *Note: the line between interpreted/compiled languages can be fuzzy. Many languages support both options, with one method being more common. The rest of this talk will cover running Matlab and Python jobs on a DHTC system.
13. OSG User School 2016 Running interpreted code in jobs General procedure •  Need to bring along interpreter and script •  Use a wrapper script as the executable •  Wrapper script will: - “Install” the interpreter - Run the script using the local installation 13
14. OSG User School 2016 Matlab •  Wait a minute…isn’t Matlab licensed? •  Yes, when interpreted on your computer using a normal Matlab installation. •  However, Matlab code can also be compiled. •  Once compiled, the code can be run without a license using a (free) set of files called the Matlab runtime. 14
15. OSG User School 2016 Matlab contrast Running Matlab on your computer Uses license per instance Running Matlab on DHTC Uses license once, runs many instances for free 15 Matlab script(s) compiled w/ Matlab compiler (uses license) Compiled binary interpreted by Matlab Runtime (free)
16. OSG User School 2016 Matlab on DHTC 1.  Compile Matlab code using the Matlab compiler (mcc) - requires a license 2.  Prepare a copy of the Matlab runtime - download for free from Mathworks 3.  Write a script that “installs” the runtime - The Matlab compiler actually writes most of this script for you 4.  Use the runtime install to run the compiled Matlab code 16
17. OSG User School 2016 Interpreted Languages •  Matlab is still being compiled before running •  What about programs like Python or R that aren’t usually compiled? 17
18. OSG User School 2016 Python •  Common language used in research computing •  Can incorporate external modules for extra functionality •  Usually interpreted, not easy to compile •  Free! 18
19. OSG User School 2016 Python on DHTC 1.  Bring along: - pre-built installation OR Python source code - your Python code 2.  Use a wrapper script to: - unpack pre-built install OR install from source - run your Python script (Similar to Exercise 1.3 this morning, will also work for R) 19
20. OSG User School 2016 Exercises •  Running Matlab Jobs - Exercise 1.4 •  Running Python Jobs - Exercise 1.5: Pre-building Python and using that installation - Exercise 1.6: Writing a script that installs Python with every job •  Half of the room should start with Matlab, the other with Python 20
21. OSG User School 2016 Questions? •  Feel free to contact me: - ckoch5@wisc.edu •  Now: Hands-on Exercises - 11:00am-12:15pm •  Next: - 12:15-1:15pm: Lunch - 1:15 onward: free time 21