Skip to content
AIFIM
Research & Practice

accessibility softwarecontrol blindaccess

PapoNox

A research project on accessible music production for blind and partially sighted users

Status

With the assistive tools available today, blind people can only scratch the surface of professional music production software.

The video has subtitles; the full transcript is below. It is spoken in German.

Transcript of the video

The spoken text, translated, with timestamps so you can find the passage in the video. Based on YouTube’s automatic transcript; obvious mishearings have been corrected.

0:05
[Musik]
0:15
Well, that’s me — Petar Nikolic.
0:18
About two years ago I lost my sight, and I asked myself: what now, when it comes to producing music? You need a screen, software you have to operate, software you have to see.
0:44
[Musik]
0:46
Here we want to show you some of the features of PapoNox that already work.
0:55
What should the project be called? Which folder is the file in?
1:02
Import Beat MP3. — Audio file “Beat MP3” imported.
1:10
Track 4, Audio 4, solo enabled. Playback started.
1:22
Das Forschungsprojekt Paponox muss jetzt weiterentwickelt werden. We have to standardise it so that another
1:29
blind or partially sighted person can work with it.
1:33
What we have built works. A prototype has come out of volunteer work. What is missing is validation: tests with blind musicians,
1:42
accuracy measurements and an external audit. That is what we need funding for.

Apple’s screen reader reads the menus. The mixer, the interfaces of the audio tools and the editor remain inaccessible. A single productive action can take twenty keyboard shortcuts or more. And continuous values such as volume or pan are practically impossible to set without looking at the screen.

This affects no small group. According to the World Health Organization, around forty-three million people worldwide are blind; a marked increase is expected by mid-century as populations age. Professional creative software has so far largely left this group out of account.

Purpose

PapoNox investigates whether professional music production software can be operated entirely without visual output.

The approach is called intent-based control. Instead of rebuilding a screen you cannot see, you say or type what you want to achieve. A language model translates that intent into actions inside the program — entirely on your own device, with no connection to a cloud service.

This is explicitly research, not a reimplementation of someone else’s software. PapoNox modifies no third-party source code and reproduces nothing. What it examines is how far the publicly documented interfaces of the operating system can be opened up for blind users — methodologically comparable to a screen reader that reaches deeper.

Goal

The goal is an empirically tested answer to a question nobody has asked systematically so far: can professional software be made systematically accessible even when its interface was never designed for it?

The first prototype answers part of it. It shows that the principle holds. That is where the real work begins: establishing which classes of function can be addressed reliably and which cannot, and where the robust limits of the approach lie.

Tasks

The ongoing research tests three assumptions.

  1. One sentence is enough A command in ordinary language suffices if the system correctly grasps the intent behind it. What needs testing is which actions this holds for and where the intent stays ambiguous.
  2. Intent becomes action A language model running on the user’s own device can reliably turn musical intent into actions inside the program. What needs testing is how stable this stays across different tasks and across program updates.
  3. Interfaces become addressable Several techniques combined — the operating system’s accessibility interface, text recognition and a vision model — make elements addressable that offer no official interface. What needs testing is how far this chain holds.

Questions on the horizon

Can this intent-based approach, and the depth of control it offers, be carried over to other closed programs? To all of Apple’s software? That question of universal transferability is the subject of our project “Click — Software Control for macOS”.

Who works on it

PapoNox is a non-commercial research project from Vienna with its own website. It is led by Petar Nikolic, who formulated the research question out of his own experience. The association supports the project as research into assistive technology and into the question of how professional software can be made accessible.

Legally, the work rests on the European Accessibility Act and on the interoperability exception in copyright law.

More about the project