Keynote at the OCaml Workshop 2020 (part of the ACM ICFP 2020 series). The talk describes the advances in the OCaml community around tooling and the processes for the construction of the OCaml Platform.
is spent within an IDE. • Traditionally, OCaml has had good support for Emacs, and "ok" support for Vim. • Editor support is complex due to the number of tools and latency requirements.
Server New OCaml project: github.com/ocaml/ocaml-lsp Integrates pieces of merlin, odoc, and omd into a single binary, so works out of the box. Builds on previous efforts, but in pure OCaml.
for OCaml. • Single binary means that complexities of tooling can be hidden behind it. • We also developed a complete Visual Studio Code plugin: • VSCode is popular, extensible and cross-platform. • Install the OCaml Platform extension and off you go. • Supports all the features newcomers expect from a modern IDE. Integrated Development Environments
plugin in the VSCode Marketplace. • 100% free and open-source! Integrated Development Environments The plugin will transition to the "ocaml" organisation on GitHub. Next: how do we accept and maintain such packages?
with support for popular tools. • Combined efforts of Florian Angeletti (OCaml release), Kate Deplaix (opam) and David Allsopp (OCaml core), and all the developers of the tools. • See ocaml/opam-repository#6539 • Requires an extended release cycle for OCaml, with a new alpha phase before the more public beta. The OCaml Platform: Synchronous Release
2.0.2 LGPLv2 active opam packaging 2.0.7 LGPLv2 active dune-release packaging 1.4.0 ISC active odoc documenting 1.5.1 ISC incubate ocamlbuild building 0.14.0 LGPLv2 sustain dune building 2.7.0 MIT active merlin editing 3.3.7 MIT active utop editing 2.6.0 BSD3 active ocp-indent editing 1.8.1 LGPLv2 sustain ocamlformat editing 0.15.0 MIT incubate lsp-server editing 1.0.0 ISC incubate mdx testing 1.7.0 ISC incubate bun testing 0.3.3 MIT incubate omp ast 2.0.0 LGPLv2 sustain ppxlib ast 0.16.0 MIT active
ecosystem, or perform an existing function differently. • Not yet ready for mainstream release and have unreliable backwards compatibility. • Active • The day-to-day workhorse projects, with strong backwards compatibility. • May be adding major features, but not radically changing their workflows. • Sustain • No major new features added, but updated for OCaml releases. • Continue to use these, but be aware that there may be active alternatives. • Can be an extremely important part of the ecosystem and in this mode indefinitely. • Deprecate • Being pushed out of the OCaml distribution and platform. • There will be recommended alternatives, and projects should migrate. The OCaml Platform: Classification
the Platform? • Initially, just tools. Libraries will be supported as "transitive dependencies". (e.g. even ppx compiles into a binary driver) • Principles for acceptance: • Sharing: does this assist with the reuse and publication of OCaml code? • Development: does this improve the OCaml development process, ranging from onboarding new developers to maintaining a legacy codebase? • Evolution: is there a strategy for incremental change and backwards compatibility over several years? • Openness: is the source code liberally licensed, and are there any operational restrictions (such as online services) that the tool depends on? • Structural requirements: • Maintainer resources, establish a community need, demonstrate some adoption. The OCaml Platform: Scope
in the ecosystem: opam: source-based package management dune: composable build system for OCaml merlin: IDE and type-checker integration utop: interactive REPL via a CLI ppxlib: AST manipulation infrastructure opam-publish: release OCaml packages Usage Guidelines: Are the recommendation for new projects. Advanced users can continue to use toolchain directly. Avoid duplication of functionality. Metadata files are versioned reliably.
gap in the tooling ecosystem: odoc: cross-referenced documentation ocamlformat: mechanical code formatting lsp-server: OCaml backend for IDEs mdx: automated testing of code fragments bun: automated fuzz testing in CI dune-release: easier releases to opam How to get promoted? Establish a migration path and need. odoc: replace all major uses of ocamldoc ocamlformat: support partial file indentation (ocp-indent) lsp-server: a release cycle for time and testing dune-release: integrate with opam-publish, avoid duplication
being maintained best-effort: ocamlfind: compilation unit manager ocamlbuild: build system (formerly included with compiler) ocp-indent: as-you-type code formatting Sustain projects are best-effort If your projects use them, then it is safe to continue, but there may be better alternatives. ocamlfind: generally no need to use directly ocamlbuild: recommend a migration to dune, but older projects will still build fine. Newer OCaml options may not be exposed. ocp-indent: works fine and stable, but steadily supplanted by ocamlformat.
camlp4: no longer included in compiler or used as a dependency in tools oasis: no longer supported for modern versions of OCaml Deprecated projects are only deprecated for the Platform In all cases, any community user can step up and take over maintainership. However, the guarantees about being released in sync with the OCaml compiler no longer hold. camlp4: now maintained by ygrek for the community (thanks!) oasis: still builds old code (e.g. Mirage libraries) fine, not used in new packages.
progress and stability: A thought experiment: if an OCaml Platform in 2013 had required ocamlbuild, would dune exist now? So incubated projects with new points of view are vital, but they must progress to maturity before replacing active ones. Improving syntax and metadata is important. Dev meeting notes regularly recorded where practical: https://github.com/ocaml/opam/wiki https://github.com/ocaml/dune/wiki https://github.com/ocaml/odoc/wiki Maintainer guidelines: Active projects need to remain open to contributors. No active OCaml Platform project shall be restricted due to employer, and shall retain a liberal license.
incubate ocamlformat editing Josh Berdine, Guillaume Petiot incubate lsp-server editing Rudi Grinberg incubate mdx testing Guillaume Petiot, Nathan Rebours incubate bun testing Mindy Preston incubate dune-release packaging Nathan Rebours, Thomas Gazagnaire incubate opam packaging Raja Boujbel, Louis Gesbert, David Allsopp active opam-publish packaging Louis Gesbert active dune building Jeremie Dimino, Rudi Grinberg active merlin editing Fred Bour, Thomas Refis active utop editing ZAN DoYe, Jeremie Dimino active ppxlib ast Jeremie Dimino, Nathan Rebours active ocamlfind packaging Gerd Stolpmann sustain ocamlbuild building Gabriel Scherer sustain ocp-indent editing Louis Gesbert sustain omp ast Fred Bour, Jeremie Dimino sustain camlp4 ast ygrek deprecate These maintainers work for a diverse range of organisations (Tarides, Jane Street, OCamlPro, OCaml Labs, Inria, Facebook, OCaml Software Foundation) or act as individual contributors.
private mirror of repositories incubate inbox.ocaml.org email list archives active ci.ocaml.org continuous integration for opam active discuss.ocaml.org forum for threaded discussion active discord.ocaml.org server for Discord chat bot active www.ocaml.org main website active lists.ocaml.org email list management sustain opam.ocaml.org opam package lists sustain forge.ocaml.org source code hosting deprecate Some services will be deprecated over the next 12 months: lists.ocaml.org is quite a maintenance burden forge.ocaml.org will finally have DNS entries removed. Some services will improve over next 12 months: ci.ocaml.org will be replaced with an ocurrent.org-based CI, backed by a multiarch computer cluster (see Thomas Leonard's talk) docs.ocaml.org: odoc is seeing major improvements to support this (see Jon Ludlam's talk) Thanks to Packet, Scaleway, Amazon and Cambridge University for sponsorship
next steps? • merge opam.ocaml.org with the main OCaml website. • add a newsfeed to the OCaml website for platform announcements (from discuss.ocaml.org). • regular newsletter about Platform developments (like the "multicore monthlies"). • add workflow guides to ocaml.org There are still some major missing development gaps, however: Windows support, and better turnkey installers. Onto the development plans for the next 12 months...
and these major new features: - integrated external dependency handling - opam install will intelligently check for apt/rpm/brew depends. - no more need for an external opam depext - opam lock integration - generate fixed version files via "opam lock" - create reproducible switches from these lock files. - opam cli versioning - in scripts, just set OPAM_CLI=2.1 to guarantee 2.1 behaviour, even in future clients. - switch invariants - richer version constraints to specify base packages in a switch, such as upgrading automatically on OCaml patch releases. - significant performance improvements: - faster repository loading - interleaving downloading and installation - more efficient copying during package installation - new "0install solver" optimised for finding from-scratch solutions
workflows: - opam compiler - https://github.com/ocaml-opam/opam-compiler - manage OCaml compiler installations much more easily - opam tools - https://github.com/avsm/opam-tools - install Platform tools within a local switch automatically - opam monorepo - https://github.com/ocamllabs/duniverse - generate dune-compatible monorepos as opam lock files - (the tool formerly known as duniverse, now integrated with opam) These are all under development, and examples of how we can experiment with new workflows from within the familiar opam framework
2.1.0 is released, the maintainers have decided that we need to focus together on end-to-end Windows support. This will integrate fdopen's superb Cygwin fork. Requires some key client features, which also help other platforms: • Native shell integration • Do not assume the existence of a unix environment • Package parameters • Support variable tracking through opam files • Helps rationalise compiler variants as well (500+ now). • Layered switches • allow binaries to be made available across switches. • Build environments • generalise sandboxing to support more toolchains • could potentially use this to build directly in containers too More details: https://github.com/ocaml/opam/wiki/opam-2.2-slides.pdf
avsm
taichinakamura
iselegant
cyberagentdevelopers
huffon
lmi
freee
calm1205
tsujiba
tsumita
fujiwara3
rkaga
bluesmoon
sonatard
speakerdeck
jonyablonski
cromwellryan
eileencodes
yeseniaperezcruz
vanstee
mraible
brettharned
tanoku
tammyeverts
