Skip to content

Tim Burgess: improving hardware accessibility using MIDI

Introduction

There has been significant progress in providing people with disabilities access to software products over the last few decades, with alternative switching systems being produced for people with any degree of motor function loss, hardware and software magnification systems to assist those with some degree of sight loss, but with a reasonable amount of remaining vision,  and screen readers and optical character recognition systems for people with little or no useable sight.

This progress seems destined to continue as governments, albeit slowly, introduce and enhance legislation that imposes responsibilities on manufacturers and service providers to address issues of equality. Artificial intelligence is also starting to play a part by improving access to images for those with a visual impairment via systems such as Be My Eyes’ “Be My AI” feature. However, hardware has lagged somewhat in many areas, especially in the world of music technology.

I am a totally blind keyboard player and software developer, so my experiences and conclusions are inevitably biased towards that perspective. Having said that, providing solutions for visually impaired users often turns out to be more complex than for any other group of people facing accessibility issues, which usually means that solutions that address visual impairment almost always also provide insights into solving the accessibility challenges of other groups. Inclusive design improves the user experience for everybody.

Background

Only a very small proportion of the musical equipment available today has been designed with accessibility as a key consideration throughout development.

“Traditional” instruments can be broadly thought of as being increasingly problematic as a player’s level of motor function is reduced, but most such instruments don’t necessarily present issues for visually impaired players that can’t be solved with diligent practice. The same could be claimed for early technology-based equipment, as the number and complexity of the controls was limited. If a user is able to turn a knob, move a fader, or flip a switch, then the only real barrier would lie in learning the layout of the device’s panels.

The advent of the microprocessor in the 1970s started to change the situation in terms of accessibility, however, as manufacturers began to reduce the number of physical controls available on their devices, sometimes for aesthetic reasons but more usually to reduce the cost of manufacture. Knob-per-function interfaces became increasingly replaced with many fewer controls and some form of text display that provided a menu system for selecting and modifying parameters. This trend drastically reduced the level of accessibility to any user who does not have access to the display and any ancillary status lamps/LEDs.

To be fair, many musicians who would not consider themselves to be disabled in any way complained that this new design approach made the equipment less intuitive to learn and less fun to use. A reduced physical control count also makes it harder to make adjustments in a live environment – most musicians on a darkened stage or in an orchestra pit could be considered to be visually impaired as a product of their working environment.

Coping strategies for live musicians include the use of any performance settings that might be available, such as preset-dependent routing of a synthesizer modulation wheel so that live changes can be made without having to delve into a complex display-based menu. Strategies for visually impaired users are usually based around the use of fully-sighted helpers to document any menu structure in some accessible format then learn a pattern of control manipulations that will end up with the desired change (for example, pressing the “Main Menu” button, followed by 9 presses of the “Arrow Down” button, a press of the “Enter” button, 7 more “Down Arrow” presses then a final “Enter” might toggle a chorus effect). The limitations of this approach are obvious:

  • The user needs to have access to enough sighted help to create the initial mapping documentation.
  • The user needs to have a highly organised mind and an excellent memory to learn the device’s interface in the required amount of detail.
  • The possibility of error, especially when performing live, is high.
  • All controls need to provide enough tactile feedback to allow for counting. For instance, an encoder that behaves like a traditional potentiometer (i.e. it has a “smooth” action instead of providing a click as it steps through available options) would completely defeat this kind of counting strategy.
  • In addition, any firmware updates applied to the target device have the potential to require re-mapping the menu structure and the user having to re-learn any required pattern of control adjustments.

MIDI and accessibility

MIDI messages have always provided the potential to act as a source of accessibility. The most basic form of access is via the use of continuous controller (CC), registered parameter number (RPN) and non-registered parameter number (NRPN) messages.

Most digital audio workstation (DAW) software that supports MIDI provides mechanisms for the user to incorporate at least some of these message types into a track. Computer-based utilities such as MIDI-Ox for Windows support these message types outside of a full-on production package and some hardware control surfaces, such as Behringer’s BCR2000 and Kenton’s Control Freak, come pre-configured so that their physical controls are mapped to specific CC messages.

CC and RPN messages have defined uses in the MIDI standard, so all the user needs to do is learn which message relates to which function to gain some basic parameter control. NRPN messages have no allocated function in the MIDI standard and are less-well supported by hardware and software, but manufacturers are free to map NRPN messages to device functions then document those mappings in a device manual.

However, the message types described above are limited to a single 7-bit byte of data (for CC messages) or 2-byte values (RPN and NRPN), which limits their usefulness accessibility-wise.

Fortunately, the original MIDI specification made provision for devices that might require transmission and reception of messages of arbitrary length – these are the system exclusive (SysEx) messages.

MIDI accessibility by accident?

A few manufacturers have included MIDI system exclusive messages to provide device state information to other devices and/or as part of the device’s role in controlling software. Two notable examples are:

The Kurzweil K2000, K2500 and K2600 synthesizer ranges

These synthesizers provide nearly complete control panel status data, including the state of all indicators and the text of the main LCD display. Individual elements can be queried, and a dedicated response received by any software implementing the relevant system exclusive messages is possible.

I am unsure of the history of this feature, as I’ve been told that it was incorporated due to Ray Kurzweil’s relationship with Stevie Wonder, but I’ve also heard that it was implemented to provide remote control support of the units and there is certainly software that exploits the feature for this purpose.

A small company produced the Kurzweil Display Reader application for Windows in the 90s to exploit this functionality for accessibility purposes, but the company now seems to be defunct. The Reference Guide for the Kurzweil synthesizers details the system exclusive messages that are required. I have developed a browser-based version of this concept that will work on any browser that supports the Web MIDI standard.

Kurzweil Access video demo

The Mackie Control, Logic Control and HUI protocols

Mackie developed these three protocols to support hardware control surfaces that aimed to provide a more traditional hardware front-end to digital audio workstation (DAW) software packages. Again, indicators and display elements can be interrogated using system exclusive messages and their states reported back using response messages.

Mackie deems the details of their protocols to be proprietary, though they have been willing to release their technical documentation to interested parties under an NDA.

I developed an accessibility tool for the Mackie Control protocol that worked on Windows in conjunction with Cakewalk’s Sonar DAW then expanded the idea into a Windows/Mac cross-platform application called Surface Reader that attempted to be protocol and DAW-agnostic, the work being part-funded by the Elizabeth Eagle-Bott Memorial Fund.

Case study: MIDI accessibility by design

We may consider the Kurzweil and Mackie cases to be accessibility by accident (discounting any possible Stevie Wonder involvement). I’d now like to detail a project that deliberately used MIDI to add accessibility to a MIDI-equipped hardware device: the Expert Sleepers FH-2 MIDI-CV Eurorack module.

Interface description

The FH-2’s front panel contains an OLED display, a “clicky” encoder that also acts as a push button, together with a USB-B socket, a USB-C socket, and several 3.5mm mono jack sockets for incoming CV signals. The final control component is a small push button that can be used as a tap tempo or a clock start/stop, amongst other software-defined options. The FH-2 also supports 5-pin DIN MIDI I/O or 3.5mm MIDI A/B I/O via a choice of small expansion modules. The encoder can be pressed to activate the device’s menu system then turned to select menu options. Pressing the encoder within the menu will activate a selection/change a parameter value. A long press of the encoder progressively backs out of the menu.

Basic principles

I proposed a MIDI system exclusive scheme to make the FH-2’s display accessible based on a simple set of six messages:

  • Request display CONTENTS
  • Send display CONTENTS as text
  • Request current parameter name
  • Send current parameter name
  • Request current parameter value
  • Send current parameter value

Each of the “Send” responses encode the required text as ASCII bytes. Andrew Ostler of Expert Sleepers undertook to add support for these messages into the FH-2 firmware, while I modified Expert Sleepers’ existing Web MIDI-based configuration tool to improve its HTML accessibility with respect to screen readers. Additionally, I developed a new single-page HTML and Web MIDI application (FH-2 Access) to act as an accessibility and remote-control client.

Accessibility mode

In addition to the FH-2 responding to any messages requesting display or parameter information, an Accessibility setting was introduced into the module’s menu system. With this setting enabled, any encoder turn, push or long push operations send accessibility messages, as follows:

  • Pressing the encoder sends the text content of the display.
  • Turning the encoder transmits parameter name and/or value messages, depending on the menu location at the time.
  • Holding down the small push-button during power-up also toggles the Accessibility setting, so no additional help is required to enable or disable the feature. Allowing this setting to be turned on/off (it is off by default) was deemed necessary to prevent the additional MIDI traffic from overloading a potentially crowded MIDI bus.

The accessibility client

This application is distributed as open-source and forms part of Expert Sleepers’ downloadable tools to support the hardware, all of which can be obtained from their GitHub page.

The accessibility client initially processed any of the display, parameter name or parameter messages to populate a facsimile of the hardware’s display text within a browser window, with parameter name and value messages also being written to an Aria live region to ensure automatic announcement by any active screen reader.

Refinements

Initial testing revealed that the basic strategy provided useful feedback but that some refinement was required for when the device’s display was in the default (i.e. non-menu) state. In this mode, the encoder adjusts the device’s internal clock tempo in 1BPM increments, so an additional pair of messages was added to request and send this data.

Additionally, we realised that adding an extra message to provide notification of what panel control had been adjusted helped the responsiveness of the accessibility client. This led to the implementation of a matching message that the client could send to control the FH-2’s front panel remotely. Remote control not only provides a user with additional needs to use the most convenient approach to using the device (i.e. using the front panel physical controls or using virtual encoder controls within the accessibility client), but the feature also provides a convenient alternative means of control for any owner of the module.

Finally, it was realised that naming presets required another pair of messages: one that triggers the appearance of an edit field on the client and a second to return the text representing the preset name to the device.

The whole process of adding the required messages to the module’s firmware and creating the client was completed on a very part-time basis over the course of a couple of weeks.

FH-2 Access video demo

Conclusion

I hope that this article has demonstrated that MIDI provides a useful mechanism for implementing accessibility into any equipment that supports the protocol and the costs of providing such support need not be burdensome to the manufacturer, especially if such support is built-in to the initial design. The obvious limitations of this approach are:

  • The hardware requires support for bidirectional MIDI traffic.
  • The manufacturer must have a registered system exclusive Manufacturer ID.
  • The target device needs to have enough spare capacity in its firmware memory to contain the required additional code.

If Web MIDI is to be used to develop client applications, there are some additional limitations, as this interface is not currently supported by all browsers on all platforms. To the best of my knowledge, the Windows and Mac versions of Chrome, Edge and Opera offer full Web MIDI support, but Firefox does not, and Apple’s Safari browser offers no Web MIDI support on the Mac. iOS is particularly poorly-served as none of the mainstream browsers support Web MIDI, apparently due to there being no support for the protocol in Apple’s WebKit library. There is a 3rd-party Web MIDI Browser for iOS, but is unfortunately inaccessible. I am told that some Android browsers support Web MIDI, but I have had no success with the single tablet device that I have had access to so far.  

MIDI 2.0

The new MIDI 2.0 specification has the potential to maintain and improve access to software and hardware by the following mechanisms:

  • Backward compatibility. A key part of the MIDI 2.0 specification is that it maintains compatibility with all MIDI 1.0 messages, so all of the techniques discussed earlier in this article should continue to function with no modification.
  • MIDI Capability Inquiry (MIDI-CI). This new feature allows compliant hardware and software to discover what facilities each device/application in the network offers, potentially allowing accessibility clients a mechanism to adapt their user interfaces to whatever equipment is discovered.
  • MIDI-CI Property Exchange and Profiles. This is a key part of MIDI-CI and allows applications/devices to interrogate each other to discover what parameters are supported, including parameter names and value ranges. This should allow client applications to auto-configure themselves to provide adaptive user interfaces, without the user/developer having to plough through documentation to figure out what is required to map hardware or software controls to device parameters.

About the author

I played in a number of local bands as a young person in the 1980s and also did a little session work as a keyboard player. I lost my sight in my early 20s so sold my gear, then thinking that my days as a musician were over.

After attending a rehabilitation course that taught me white-cane mobility, braille and touch-typing, I studied computer science to degree level. I then worked as a product trainer and script developer for an IBM Special Needs Services agency. I later moved to Microsoft as a support technician, followed by a role in an accessibility consultancy, where I developed and trained on workplace solutions for visually impaired people.

A chance encounter with Jackie Clifton, Research Fellow at the Royal College of Music, led to me forming my own company, Raised Bar, in 2006 with the aim of providing advice, consultancy, and development services based around accessibility in general, but with a strong emphasis on music-related accessibility barriers. Since then, I have worked with numerous music technology companies, including Native Instruments and Ableton, to improve accessibility within their products.

Further information can be obtained by contacting me via: tim@raisedbar.net

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.