Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Brain-Computer Interfaces - Science Fiction or ...

Piotr Wittchen
November 20, 2018

Brain-Computer Interfaces - Science Fiction or Reality?

Brain-Computer Interface (BCI) is a direct communication pathway between the brain and an external device. It seems to be new technology, but it has its origins in 1924! Today, we have technologies, which lie on the border between biology and computer science like neuroprosthetics, electromyography and of course BCI. During the talk, we will start the journey through different brain imaging techniques, but in the end, we will focus on electroencephalography (EEG). BCI is no longer expensive technology available only in laboratories. Anyone can have an access to it. Such technology gives us the possibility to create human-computer interaction of the future. Additionally, we can develop communication interfaces for people with illnesses like LIS and it may be the only hope for them. During this talk, we will discuss not only a theory but also practical examples, available software and working pieces of code for the Android platform. If you care about transhumanism, you cannot neglect BCI.

This presentation was shown during the Øredev 2018 conference in Malmö, Sweden on 19th of November 2018.

Piotr Wittchen

November 20, 2018
Tweet

More Decks by Piotr Wittchen

Other Decks in Research

Transcript

  1. 3

  2. 4

  3. 5

  4. 6

  5. 7

  6. 10

  7. 12

  8. In 1924 Hans Berger was the first to record human

    brain activity. He was able to identify oscillatory activity in the brain, such as the alpha wave (8–12 Hz). 13
  9. 20

  10. 21

  11. 27

  12. 28

  13. BCI vs. Neuroprosthetics Neuroprosthetics connect the nervous system to a

    device, whereas BCIs connect the brain with a computer system 30
  14. Magnetoencephalography MEG is the technique for mapping brain activity by

    recording magnetic fields produced by electrical currents occurring naturally in the brain, using very sensitive magnetometers. 33
  15. Positron Emission Tomography PET is the nuclear medical imaging technique

    that produces a 3D image or picture of functional processes in the body. 34
  16. Functional Magnetic Resonance Imaging FMRI is the Magnetic Resonance Imaging

    procedure that measures brain activity by detecting associated changes in blood flow. 36
  17. Functional Near Infrared Using FNIR, brain activity is measured through

    hemodynamic responses (blood movement) associated with neuron behavior. Mentioned responses can be detected with Infrared (IR) radiation. 37
  18. How does EEG work? After placing electrodes on the skull,

    we can register changes of electric potential on the surface of the skin. These changes are caused by activity of neurons of the cortex. 39
  19. 40

  20. 42 EEG Voltage recorded from the electrode electrode amplitude frequency

    phase Frequency component time Source: Nunez P., Srinivasan R.: Electroencephalogram, Scholarpedia, 2007
  21. EEG rhythmic activity frequency bands • Delta (up to 4

    Hz) - in babies or slow-wave sleep by adults • Theta (4-8 Hz) - young children or drowsiness by older children and adults, idling • Alpha (8-13 Hz) - relaxed/reflecting, closing eyes, pathologically: coma • Beta (>13-30 Hz) - active, busy, concentration or anxious thinking, pathologically: Benzodiazepines • Gamma (30-100+ Hz) - perception that combines different senses (e.g. sound and sight), short-term memory matching of recognized objects • Mu (8-13 Hz) - shows rest-state motor neurons 43
  22. NeuroSky MindWave Mobile • Bluetooth v. 3.0 class 2 (10

    meters range) • Wireless pairing • Static headset ID for pairing purposes • 8-hours battery runtime (uses one AAA battery - 1.5 V) • iOS and Android support • Measures raw brainwaves • Processing and output of EEG power spectrum (Alpha, Beta, etc.) • Processing and output proprietary eSense meter for mediation, attention and other future meters • Blink detection • Signal quality analysis 45
  23. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 53
  24. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 54
  25. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 55
  26. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 56
  27. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 57
  28. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 58
  29. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 59
  30. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 60
  31. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 61
  32. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 62
  33. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 63
  34. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 64
  35. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 65
  36. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 66
  37. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 67
  38. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 68
  39. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 69
  40. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 70
  41. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 71
  42. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 72
  43. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 73
  44. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 74
  45. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 75
  46. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 76
  47. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 77
  48. 78

  49. Source code, examples and links https://github.com/pwittchen/neurosky-android-sdk (brand new project) https://github.com/pwittchen/EEGReader

    (old project using ThinkGear only) http://developer.neurosky.com/ (official website of the NeuroSky) http://developer.choosemuse.com/sdk (official Muse developer website) http://wittchen.io/tags/bci/ (a few articles about BCI on my blog) 80
  50. Summary • Direct communication between human brain and the computer

    is no longer science-fiction • We have affordable devices using EEG technology today • We are able to write Brain-Computer Interfaces without having scientific knowledge about it • We can develop new ways of communication with BCI • We can create data acquisition tools for doctors and scientists • We can develop tools supporting meditation • We can develop tools for rehabilitation and improving concentration • We can create communication tools for people with illnesses like LIS • BCI can be improved with Machine Learning techniques 81
  51. Brain-Computer Interfaces Science Fiction or Reality? Piotr Wittchen wittchen.io get

    presentation slides at: wittchen.io/talks Thank you for attention! Questions?