Conda is a cross-platform package manager that lets you quickly and easily build environments containing complicated software stacks. It was built to manage the NumPy stack in Python but can be used to manage any complex software dependencies.
have different dependencies. Often many of the dependencies are shared but a few applications need different versions of some packages. • Not specific to any particular language or ecosystem. Python, Ruby, Node.Js, C/C++, .NET, Java, all have the same problem: How do you manage software life-cycle effectively? • Production deployments need stability. IT managers want ease of deployment and testing. Developers want agility and ease of development.
organization quickly and easily take advantage of the latest software developments yet still have stable production deployments of complex software? You cannot take full advantage of the pace of open-source development if you don’t address this!
and one could make a web-page and put code up on it and people started using it ... Facebook for Hackers I started SciPy in 1999 while I was in grad- school at the Mayo Clinic (it was called Multipack back then)
make file (users had to build) SciPy is basically a bunch of C/C++/Fortran routines with Python interfaces Observation: Popularity of Multipack (Early SciPy) grew significantly when Robert Kern made pre- built binaries for Windows
desire to avoid the dependency chain and lack of broad packaging solutions led to early SciPy being a “distribution” instead of separate inter-related libraries. • There were (and are) too many different projects in SciPy (projects need 1-5 core contributors for communication dynamic reasons related to team-sizes)
I was teaching at BYU (it was a merger of Numeric and Numarray) NumPy ABI has not changed “officially” since 1.0 came out in 2006 Presumably extension modules (SciPy, scikit-learn, matplotlib, etc.) compiled against NumPy 1.0 will still work on NumPy 1.8.1 This was not a design goal!!!
and community difficulty when date-time was added in version 1.4 (impossible without changing the ABI in some way) but not really settled until version 1.7 The fundamental reason was a user-driven obsession with keeping ABI compatibility. Windows users lacked useful packaging solution in face of NumPy-Stack
•Fundamental principle of software engineering is “separation of concerns” (modularity) •Reusability is enhanced when you “do one thing and do it well” •But, to deploy you need to bring the pieces back together. ! •This means you need a good packaging system for binary artifacts — with multiple-environments.
package Premium features: • hosting of private packages (public packages are free) • access to priority build queue • $10 / month (individuals) • 25 private packages • 5 GB disk space • $50 / month (organizations) • 200 private packages • 30 GB disk space • right to have private packages in organizations • $1500 / year • unlimited private packages • 100 GB of disk space binstar.org • Internal mirror of public repositories • Mix private internal packages with public repositories • Build customized versions of Anaconda installers • Environment to .exe and .rpm tools • Comprehensive licensing • Comprehensive support • On-premise version of binstar.org
homebrew Windows chocolatey npackd Cross-platform conda With virtual environments conda provides a modern, cross- platform, system-level packaging and deployment solution
mini VMs but much lighter weight than docker.io) • Minimizes code-copies (uses hard/soft links if possible) • Dependency solver using fast satisfiability solver (SAT solver) • Simple format binary tar-ball + meta-data • Meta-data allows static analysis of dependencies • Easy to create multiple “channels” which are repositories for binary packages • User installable (no root privileges needed) • Can still use tools like pip --- conda fills in where they fail.
for example, /opt/Anaconda or C:\Anaconda! /pkgs! Also referred to as PKGS_DIR. This directory contains exploded packages, ready to be linked in conda environments. Each package resides in a subdirectory corresponding to its canonical name.! /envs! The system location for additional conda environments to be created.! ! the default, or root, environment! /bin! /include! /lib! /share
files index.json bzipped tarfile of all the files comprising the package at the full-paths they would be installed to relative to a “system” install or “chroot jail” an environment is just a “union” of these paths All conda packages have this info directory which contains meta-data for tracked files, dependency information, etc.
Easy to make Easy to throw away Uses: • Testing (python 2.6, 2.7, 3.3) • Development • Trying new packages from PyPI • Separating deployed apps with different dependency needs • Trying new versions of Python • Reproducing someone’s work conda create -h
Run conda index <dirname> • Either use file:///path/to/dir in .condarc or use simple web server on the /path/to/dir Option 1 Option 2 Use binstar.org (also available as on-premise solution with Anaconda Server)
latest pandas from the channels you are subscribed to conda update anaconda change to the latest released anaconda including its specific dependencies this can downgrade packages if they are newer than those in the “released” Anaconda conda update --all To update all the packages in an environment to the latest versions use the --all option
Removing Packages Removing Environment conda remove -n py3k --all Note: packages are just “unlinked” from environment. All the files are still available unpacked in a package cache. Removing unused packages conda clean -t conda clean -p Remove unused tarballs Remove unused directories
Files Easy way to install into an environment using anything (pip, make, setup.py, etc.) and then package up all of it into a binary tar-ball deployable via conda install <pkg-name>.tar.bz2 ! pickle for binary code!
bld.bat CMD build commands (Win) meta.yaml extended yaml declarative meta-data Required Optional run_test.py will be executed during test phase *.patch patch-files for the source * any other resources needed by build but not included in sources described in meta.yaml file
version of package about: home: # home-page license: # license ! # All optional from here.... source: fn: # filename of source url: # url of source md5: # hash of source # or from git: git_url: git_tag: patches: # list of patches to source - fix.patch build: entry_points: # entry-points (binary commands or scripts) - name = module:function number: # defaults to 0 requirements: # lists of requirements build: # requirements for build (as a list) run: # requirements for running (as a list) test: requires: # list of requirements for testing commands: # commands to run for testing (entry-points) imports: # modules to import for testing http://docs.continuum.io/conda/build.html
particular platform. However, if there are no platform- specific binary files in a package, it can be converted automatically to a package that can be installed on another platform. conda convert --output-dir win32 --platform win-32 <package-file> Example
can see this package. Personal Everyone will be able to see this package in your user repository. Publish This package will be published in the global public repository.