WEBVTT 00:00.000 --> 00:10.000 Welcome to this talk about LibreOffice Accessibility and how it's implemented on the 00:10.000 --> 00:15.280 different platforms. My name is Michael, I'm a LibreOffice developer working for the document 00:15.280 --> 00:20.000 foundation with primary focus and accessibility. 00:20.000 --> 00:27.280 Okay, first let's start with a definition of accessibility. That's from Wikipedia. Accessibility 00:27.280 --> 00:33.120 is the design of products, devices, services, vehicles or environments, so it's 00:33.120 --> 00:39.440 to be usable by people with disabilities. The concept of accessible design and practice of 00:39.440 --> 00:45.840 accessible development ensures both direct access that means unassisted and indirect access, 00:45.840 --> 00:50.480 meaning compatibility with the person's assistive technology, for example computer screen 00:50.480 --> 00:58.160 readers. And that second aspect is what this talk will focus on, how assistive technology 00:58.160 --> 01:03.920 can interfere with LibreOffice or can interact with LibreOffice. So if a person is for example 01:03.920 --> 01:12.320 initially impaired or blind, how they can still use LibreOffice. One sample scenario of using 01:12.320 --> 01:18.160 an assistive technology, a screen reader in this case is for example if we have a dialogue like 01:18.160 --> 01:22.240 the one that shows up when you close a document while still having changes that are not saved, 01:23.040 --> 01:29.680 and the screen reader will read out the text of the dialogue. Do you want to save the changes? 01:30.480 --> 01:35.280 And then the user can interact with the dialogue using the keyboard, for example the tab key 01:35.280 --> 01:41.360 to move between the buttons and then LibreOffice sends an event on the via the accessibility 01:41.360 --> 01:46.880 to protocol of the platform. The screen reader receives that event and then it can ask for additional 01:46.960 --> 01:54.400 information and then allowance for example cancel a button. So the objects have different 01:54.400 --> 02:02.880 attributes like a name and a role and states and so on. So there are certain specifications 02:02.880 --> 02:07.280 that describe what they can have. But the screen reader gets that information can then announce 02:07.280 --> 02:12.800 it for example cancel a button or okay button and so on. So that's on the high level. 02:13.760 --> 02:20.480 The interesting bit is how that works, how does LibreOffice transmit that information to the screen 02:20.480 --> 02:28.000 reader and that's platform specific. For a simple case and overview we have some kind of 02:28.000 --> 02:33.840 application that's using some UI toolkit. The simple application is just built out of 02:34.960 --> 02:42.480 toolkit widgets and the toolkit takes care of everything and makes the application accessible by 02:42.800 --> 02:48.800 translating that to whatever accessibility protocol is used on the next windows, macOS, 02:48.800 --> 02:53.440 and maybe even more platforms and the screen reader assistive technology can interact with that. 02:53.440 --> 02:59.360 So it goes both ways. In that case the application doesn't actually have to 02:59.360 --> 03:05.680 know what the platform does but the toolkit takes care of everything. That's the nice scenario 03:05.680 --> 03:12.000 which we don't have in LibreOffice unfortunately. So in our case it's a bit more complex. 03:12.000 --> 03:19.520 This is simplified diagram. We have the application again as in the previous diagram and we do have 03:19.520 --> 03:27.440 widgets of course and these widgets can be native toolkit widgets. We don't have a single toolkit 03:27.440 --> 03:37.040 but we support different tool kits on Linux. We have GTK3, GTK4 or QTE5, QT6. In this case when we use 03:37.040 --> 03:42.640 native widgets and the toolkit takes care of it and translates it to the platform accessibility 03:42.640 --> 03:50.480 protocol which is ATSBI on Linux. We don't always use native toolkit widgets. If we don't then 03:50.480 --> 03:56.080 we have to do all of that stuff ourselves. So we do not need to know what does the platform do on 03:57.040 --> 04:06.320 windows and macOS. We don't use any UI toolkit but we actually translate everything ourselves. 04:06.320 --> 04:12.720 We support the iAccessible iAccessible 2 protocol and windows and the inaccessibility protocol on macOS. 04:14.240 --> 04:20.080 And we have an internal accessibility API which is the X-Accessible API. So whatever we want to 04:20.080 --> 04:26.000 expose in the platform layer internally uses the accessible API for example our widgets or we see 04:26.000 --> 04:32.800 our widgets. They use that API or support that API and then we have accessibility bridges, 04:32.960 --> 04:39.200 let's translate that to the platform protocol for example iAccessible 2 on windows or an 04:39.200 --> 04:45.600 inaccessibility on macOS. On Linux we use the toolkit again so we have this internal API but 04:45.600 --> 04:52.320 then we use whatever the toolkit provides. For example GTK4 has an API I'll go into that in the next 04:52.320 --> 05:02.240 slides and let's translate it to ATSBI again so the platform protocol. For QTE it's basically the same 05:02.320 --> 05:08.320 for GTK3 as an intermediate layer which is ATK but I'll get into that in a little more soon. 05:10.560 --> 05:18.160 So like about the platforms again so for windows and macOS as I mentioned we do all of the stuff 05:18.160 --> 05:27.520 ourselves we support we implement an accessibility bridge which directly implements iAccessible 2 on 05:27.520 --> 05:35.680 windows. We do not support UIA natively that's a new protocol but there's some basic support by some 05:36.880 --> 05:45.360 MSAAUIA proxy so windows comes with some automatic bridge so we have some basic support for UIA as 05:45.360 --> 05:54.400 well. On macOS we directly implement the inaccessibility protocol as I mentioned so we have a 05:54.480 --> 06:03.040 bridge that us this by itself. On Linux as I mentioned we do have different UIA variants that we support 06:04.000 --> 06:11.680 so depending on what UIA variant is used the ways a little different. In all cases the platform 06:11.680 --> 06:18.640 protocol is the same which is ATSBI as I mentioned before. For GTK3 which is currently the most 06:18.640 --> 06:26.480 accessible UIA variant in Linux we use native widgets so all of this native widget automatically 06:26.480 --> 06:33.840 is translated by the toolkit but the other bit is the document and our document content is not 06:33.840 --> 06:41.120 built out of widgets but that's all custom accessibility implementations so somehow we also need to 06:41.120 --> 06:46.960 get the semantics of the document to the screen reader and the interaction for example if you move 06:46.960 --> 06:52.720 between paragraphs then the screen reader should be notified that another paragraph is focused 06:52.720 --> 07:02.640 now and can read out what's the text there for example and for that for GTK3 use the ATK toolkit 07:02.640 --> 07:09.760 which is also what GTK itself uses internally and that translates it to the platform accessibility 07:09.760 --> 07:19.680 protocol which is ATSBI as I mentioned. For GTK4 it looks a little different because GTK4 07:19.680 --> 07:28.880 dropped all of the accessibility stuff that GTK3 had and they started something new. The intern so 07:28.880 --> 07:37.280 GTK no longer uses ATK, GTK4 no longer uses ATK but it implements the ATSBI protocol directly 07:38.240 --> 07:44.640 or there's also since recently another option to use access kit which is a gun and a library that 07:44.640 --> 07:51.760 does it but that's the translation and also supports macOS and windows but the default is still 07:52.480 --> 08:03.440 the GTK interl implementation right now and the not so nice thing currently is that because ATK 08:03.440 --> 08:09.360 was dropped and there's a new API in GTK we need to do things differently which by itself is not a 08:09.360 --> 08:16.480 problem and there's a GTK accessible API for application so leaf office can implement that API and if 08:16.480 --> 08:23.920 it does the then GTK takes care of translating things to HSPI and Linux the main problem is that currently 08:23.920 --> 08:30.400 this API doesn't provide everything we need so we currently cannot expose all of the document 08:30.400 --> 08:38.800 content the way we would have to. Most important thing or one of the main things is we can't 08:38.800 --> 08:44.000 notify about focus changes so when the user changes from one paragraph to another the screen 08:44.000 --> 08:49.680 read it just won't notice because the developer office can notify because GTK just likes the 08:49.680 --> 08:55.280 IP API to do that there's some discussion with upstream but there's some things that are missing 08:56.000 --> 09:03.680 for that reason we don't use GTK for by default anywhere right now so we still default to GTK free 09:03.680 --> 09:12.560 on the GNOME independent of the version. For QT we also have some accessibility API which is Q 09:12.560 --> 09:20.640 accessible interface and funds which bridges to HSPI directly that's been some recent improvements in QT 09:20.640 --> 09:26.800 so QT6 has most of the API we need and there's some things missing at least I know of high 09:26.800 --> 09:32.320 polling support but I'm quite optimistic we can work together with the QT project to get this in some 09:32.320 --> 09:40.720 someday so GTK free currently is still the most accessible variant but I'd say QT6 is getting closer 09:40.720 --> 09:45.920 and I'm quite optimistic where we can get that pretty accessible if we put some more work into that. 09:46.320 --> 09:55.840 Some more things of interest I mentioned that HSPI is a protocol used in Linux at the moment but 09:55.840 --> 10:03.120 there's also some proof of concept accessibility stack called Newton that's major place HSPI at some point 10:04.320 --> 10:09.040 the main idea is that it's push based approach so the application always pushes and each 10:09.920 --> 10:15.920 and there's no need for the assistive tooling to query and information 10:17.680 --> 10:23.200 what is to be known is that it's also mentioned it doesn't really know yet how to deal with 10:23.200 --> 10:28.960 big documents for example so that's an open point but it's interesting to to keep an eye on that 10:28.960 --> 10:34.960 and see how it goes and one of the main things I want to say is that in order to be really 10:34.960 --> 10:40.000 accessible that's something that deep profits can do by itself so even if we implement all the 10:40.000 --> 10:46.000 protocols very nicely there might be things on the toolkit level or on the assistive technology 10:46.000 --> 10:51.520 level that expect things differently so actually making deep profits accessible is an effort that 10:51.520 --> 10:58.240 needs to involve all of the projects of the components and I really thank for working together 10:58.240 --> 11:02.960 with all the people in the different projects works really well and for all the support I get there 11:02.960 --> 11:09.680 and the input so I really want to say big thanks to everyone involved and both in the tool 11:09.680 --> 11:15.200 kits and libraries and also on the assistive technology side really really thankful for all the people there 11:17.200 --> 11:21.920 that's it so far there's some links if you're interested in learning more about 11:21.920 --> 11:26.000 the accessibility you're getting involved there's some links in the presentation 11:26.720 --> 11:32.560 it's in in the foster system so you can get it there thanks a lot for your attention 11:37.840 --> 11:40.160 and if you have any questions you can ask me afterwards