Tobi Wulff: Control All Your Programs with MIDI Controllers

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Tobi Wulff:
Control All Your Programs with MIDI Controllers
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2014 - Saturday, 13 Sep 2014 - Track 1
http://kiwi.pycon.org/

**Audience level**

Intermediate

**Description**

MIDI devices are commonly used to provide input to audio applications, e.g. to play a synthesizer via a keyboard or to control a mixer in an audio workstation. This talk will show you tools for receiving input and processing messages, and how to control all of your programs with minimal configuration effort, the advantage being that not every single program has to be made natively MIDI aware.

**Abstract**

This talk will give a quick overview of the MIDI protocol and controllers which are commonly used to provide input to audio applications. These controllers can have the form of a musical keyboard, a mixer with knobs, sliders and buttons, or a combination of those. Some other variants use touch-sensitive trackpads or look like a drum kit. Then, the talk will go over the following tools: Python (duh), mididings (to receive and manipulate MIDI messages), xte (send keystrokes and mouse events to the Linux Xorg server) and some other necessary programs such as jackd to virtually connected MIDI inputs and outputs. I'll describe the steps needed to bring all those tools together and make it extensible to (hopefully, eventually) work on other OS as well.

mididings provides a Domain Specific Language (DSL) for MIDI processing which will be introduced as part of this talk. If you are doing anything with MIDI devices (and other external devices apart from keyboards) and feel like they pose some limitations on you (for example not supporting different customisable button assignments on the device), this will be of interest to you. xte will be used in a very basic way but if you haven't heard of it and want to send events to your X server, this is a great program. The possibilities are endless and we haven't even combined all the tools yet. This is about to change in the next step of the talk. A live demo/video will demonstrate how it all comes together into one beautiful piece of ingenuity and software engineering.

Finally, I'll discuss pros and cons of this solution versus other possible methods including native MIDI device support for various programs.

**YouTube**

https://www.youtube.com/watch?v=p2CsRZoatc8

Transcript

1. Control All Your Programs With MIDI Controllers Tobi Wulff KiwiPyCon 2014
2. MIDI Controllers
3. The Good: Standardised Protocol No worries about OS drivers Lots of connections and daisy chaining Reusability Libraries for Arduino, and more Cheap MIDI­to­USB adapters
4. MIDI Controllers Mixer boards Pads Keyboards (Piano) Even drum kits
5. Lighting Console www.alg­uk.com
6. Keyboards www.openmusiclabs.com
7. Keyboards www.flickr.com (Stephen Drake)
8. Drums musicradar.com
9. Drums performing­musician.com
10. MIDI Controller Components Sliders, Buttons, Potis, Rotary Encoders, Keyboard, Trackpads, ... www.bauerindustries.com
11. The Protocol and Theory Controllers generate notes and controlling messages Software can generate or process (play) messages Synthesisers can play MIDI messages => Keyboards are controller + synthesisers
12. MIDI Messages www.sebkhachott.net
13. MIDI Messages Can be sent on specific channels: 0(all) ­ 15 Data is in the range of 0­127 On/off usually done with ≤ 63 and ≥ 64 Control Change [Control, Value] Program Change Note On/Off [Note, Speed] Pitch Bend SysEx Panic! Note Off for all notes
14. MIDI Messages midi­protocol.wikispaces.com
15. Quick Excursion Linux Audio: basic layer (ALSA) Real­time and routing with Jack (jackd) Mididings can connect to both of those
16. Audio Connections
17. MIDI Connections
18. The Target: Darktable
19. The Target: Darktable Alternative: API
20. Introducing: mididings Can receive, process and send MIDI messages das.nasophon.de
21. A small Mididings Example f r o m m i d i d i n g s i m p o r t r u n , P r o c e s s , N O T E O N d e f i n v e r t ( e v e n t ) : i f e v e n t . c h a n n e l = = 1 a n d e v e n t . t y p e = = N O T E O N : e v e n t . v e l o c i t y = 1 2 7 - e v e n t . v e l o c i t y r e t u r n e v e n t r u n ( P r o c e s s ( i n v e r t ) )
22. Domain Specific Languages "A domain­specific language (DSL) is a computer language specialized to a particular application domain."
23. Mididings as a DSL p i a n o = O u t p u t ( ' s a m p l e r ' , 1 , 1 ) # p o r t , c h , p r g s t r i n g s = O u t p u t ( ' s y n t h ' , 2 ) # p o r t , c h , p r g C h a n n e l F i l t e r ( 1 ) > > K e y S p l i t ( ' c 3 ' , p i a n o , s t r i n g s )
24. Mididings as a DSL F i l t e r ( C T R L ) > > C h a n n e l ( 2 ) F i l t e r ( N O T E O N ) > > T r a n s p o s e ( - 1 2 ) > > o r c h e s t r a / / b e l l s { C T R L : C h a n n e l ( 2 ) , N O T E O N : T r a n s p o s e ( - 1 2 ) > > [ o r c h e s t r a , b e l l s ] }
25. Mididings as a DSL ( ( C h a n n e l F i l t e r ( 1 ) & ( C t r l F i l t e r ( 2 3 ) | C t r l F i l t e r ( 4 2 ) ) ) & ~ C t r l V a l u e F i l t e r ( l o w e r = 5 0 , u p p e r = 1 0 0 ) ) > > C h a n n e l ( 2 )
26. Mididings as a DSL ( # o n l y u s e c h a n n e l 1 ( C h a n n e l F i l t e r ( 1 ) & # C C # 2 3 o r 4 2 ( C t r l F i l t e r ( 2 3 ) | C t r l F i l t e r ( 4 2 ) ) ) & # s l i d e r n o t b e t w . 5 0 a n d 1 0 0 ~ C t r l V a l u e F i l t e r ( l o w e r = 5 0 , u p p e r = 1 0 0 ) ) # s e n d T H O S E m e s s a g e s t o c h . 2 > > C h a n n e l ( 2 )
27. das.nasophon.de
28. CTRL specific functions C t r l M a p ( ) # c h a n g e s C C # C t r l R a n g e ( ) # c h a n g e s v a l u e C t r l C u r v e ( ) # a p p l y a c u r v e t o v a l u e s Should use those more
29. Ok, back to controlling GUIs 1. Collect incoming MIDI messages (filtered) 2. Grab collected events and process 3. Turn event into key press (or something else) 4. Send off key press
30. Collect Main process acting as a producer adding items to the event queue. Overwrite if same event happens again. r u n ( P r o c e s s ( c o l l e c t ) ) d e f c o l l e c t ( s e l f , m i d i ) : w i t h s e l f . _ e v e n t s _ l o c k : s e l f . _ c o l l e c t ( m i d i )
31. Grab Second thread acting as a consumer taking items from the event queue. i m p o r t t h r e a d i n g w i t h s e l f . _ l o c k : s e l f . _ e v e n t s . i t e m s ( )
32. Key Press Map MIDI message to shortcut action object: M A P = { ( 0 , C T R L , 1 1 ) : ( ' W i n d o w ' , S h o r t c u t A c t i o n ( N O _ M O D , ' o ' ) ) , . . . } c l a s s S h o r t c u t A c t i o n ( o b j e c t ) : d e f _ _ i n i t _ _ ( s e l f , m o d i f i e r s , k e y , f u n c = N o n e ) : . . . d e f r u n ( s e l f , e x e c u t o r , v a l u e ) : . . . c l a s s S h o r t c u t A c t i o n S e q u e n c e ( o b j e c t ) : . . .
33. Send Will only go to foreground window (unless special global shortcut). f r o m s u b p r o c e s s i m p o r t P o p e n , P I P E k e y _ s e q u e n c e = [ ' k e y d o w n C o n t r o l _ L ' , ' k e y o ' , ' k e y u p C o n t r o l _ L ' , ' s t r 1 . 3 3 ' ] P o p e n ( [ ' x t e ' ] , s t d i n = P I P E ) . c o m m u n i c a t e ( i n p u t = k e y _ s e q u e n c e )
34. The Future OSC will probably replace MIDI: mididings ­> S e n d O S C Extend to send to other OS (not just using xte) More flexibility for other programs (GIMP, ...) Mouse etc macros Feedback (MIDI from PC to controller)
35. References For more theory: www.midi.org This program: github.com/tobiw/midi2gui Mididings: (part of most Linux package managers) das.nasophon.de Xte: xautomation.sf.net
36. Contact Me T w i t t e r : t o b i a s w u l f f G i t h u b a n d I R C : t o b i w J a b b e r a n d E m a i l : t o b i ( a ) s w u l f f . d e