User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge @dhamlinmusic @lynessence @pixelate @quanin @GamingWithEars @mcourcel Oh right, yeah, the cost of the typical virtual buffer implementation and the lack of automatic scrolling in say-all in web content are also problems.
2
0
1
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt @dhamlinmusic @lynessence @pixelate @quanin @GamingWithEars @mcourcel IMHO those are larger problems than bookmarking. We can get close enough (nearest paragraph) bookmarking in a webview. And that would be fair exchange to have working links, footnotes, endnotes, media, math content, and all the rest. But I can think of no way with current screen readers and API's that we can do autoscroll. Kindle manages it, but they're doing...something custom that I don't know anything about. And I'm just not willing to give that up.
1
1
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge @dhamlinmusic @lynessence @pixelate @quanin @GamingWithEars @mcourcel Ooh, now I want to go look at what Kindle's doing.
3
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge @dhamlinmusic @lynessence @pixelate @quanin @GamingWithEars @mcourcel Or did Kindle for PC punt on the whole problem by doing direct TTS output?
1
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt @dhamlinmusic @lynessence @pixelate @quanin @GamingWithEars @mcourcel Nope. Direct TTS output is available if you want that. But both NVDA and jaws present kindle ebooks in some sort of custom virtual buffer. It supports pressing enter on links, autoscroll during say all, etc.
1
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge Removing everyone else from the thread at this point, so we don't keep spamming their notifications.

What I really want to find out now is how
JAWS activates its virtual buffer-like behavior in Kindle and whether other apps can piggyback on that.
2
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt I've honestly been afraid to look at it too close. Kindle Cloud broke accessibility because people started using it as a way to remove Kindle DRM now that the other popular methods don't work. If I wanted too, it would be pretty easy to create a new synth driver for NVDA that uses say all as a way to extract the text from a Kindle book. Here's the current technique they're using, if you're curious: blog.pixelmelt.dev/kindle-web-drm/
2
1
1
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge But the virtual cursor still works in the Kindle Windows app?
2
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge For the record, my interest wouldn't be in bypassing DRM, but in making it easier for applications and toolkits other than the big two Windows browser engines to trigger the virtual cursor behavior in JAWS and NVDA.
2
0
1
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt When you first open Kindle for PC, your ebook library is shown in what is just a standard web view. So they do have a full browser engine as part of Kindle. But they don't seem, from what I can tell, to be using it to show the actual ebook.
1
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge Can you do a quick NVDA+F1 in the main ebook view and find out what window class it is?
1
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt Developer info for navigator object:
name: 'Dungeon Farm: A Slice of Life Base Building Fantasy Adventure'
role: DOCUMENT
processID: 47676
roleText: None
states: FOCUSABLE, DEFUNCT, FOCUSED
isFocusable: True
hasFocus: True
Python object: <NVDAObjects.Dynamic_BookPageViewPageTurnFocusIgnorerIAccessible object at 0x000001F6863568D0>
Python class mro: (<class 'NVDAObjects.Dynamic_BookPageViewPageTurnFocusIgnorerIAccessible'>, <class 'appModules.kindle.BookPageView'>, <class 'textInfos.DocumentWithPageTurns'>, <class 'appModules.kindle.PageTurnFocusIgnorer'>, <class 'NVDAObjects.IAccessible.IAccessible'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'documentBase.TextContainerObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <class 'garbageHandler.TrackedObject'>, <class 'object'>)
description: None
location: RectLTWH(left=676, top=143, width=640, height=901)
value: None
TextInfo: <class 'appModules.kindle.BookPageViewTextInfo'>
appModule: AppModule(kindle, appName='kindle', processID=47676)
appModule.productName: 'Kindle'
appModule.productVersion: '2.8.0.70980'
appModule.helperLocalBindingHandle: c_void_p(2158309884096)
appModule.appArchitecture: 'x86'
windowHandle: 3804144
windowClassName: 'Qt5QWindowIcon'
windowControlID: 0
windowStyle: 1442840576
extendedWindowStyle: 0
windowThreadID: 23448
windowText: 'Kindle'
displayText: ''
IAccessibleObject: <POINTER(IAccessible2) ptr=0x1fe8d4de878 at 1f685e60f50>
IAccessibleChildID: 0
IAccessible event parameters: windowHandle=3804144, objectID=-4, childID=0
IAccessible accName: 'Dungeon Farm: A Slice of Life Base Building Fantasy Adventure'
IAccessible accRole: ROLE_SYSTEM_DOCUMENT
IAccessible accState: STATE_SYSTEM_FOCUSED, STATE_SYSTEM_FOCUSABLE, STATE_SYSTEM_VALID (1048580)
IAccessible accDescription: exception: (-2147467263, 'Not implemented', (None, None, None, 0, None))
IAccessible accValue: exception: (-2147467263, 'Not implemented', (None, None, None, 0, None))
IAccessible2 windowHandle: 3804144
IAccessible2 uniqueID: -527070692
IAccessible2 role: ROLE_SYSTEM_DOCUMENT
IAccessible2 states: exception: (-2147467263, 'Not implemented', (None, None, None, 0, None))
IAccessible2 attributes: 'kindle-first-visible-physical-page-label:233;kindle-first-visible-physical-page-number:234;kindle-last-visible-physical-page-label:233;kindle-last-visible-physical-page-number:234;kindle-last-physical-pagse-label:316;kindle-last-physical-pagse-number:317;kindle-first-visible-location-number:3185;kindle-last-visible-location-number:3194;kindle-max-location-number:4323;class:KindleBookPageView;'
IAccessible2 relations: exception:
2
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge Thanks. Bottom line: The virtual cursor implementation that NVDA uses for Kindle is hard-coded to the Kindle app, and based on that window class name, the same is probably true for JAWS.
1
0
1
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt Yup. So in short, if I wanted to make another ebook app that worked the way Kindle does, I'd need to write a custom virtual cursor implementation for NVDA to work with it.
1
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge NVDA is modular enough that the implementation work in NVDA, or in an add-on, wouldn't have to be done from scratch, but yes, as far as I can tell, some app-specific or toolkit-specific work would currently have to be done. And for JAWS, that would basically mean being big enough to get onto Vispero's development roadmap.
2
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt As other folks are saying: I could do all that work for free and without pay...or...I could just read my epubs in Kindle and get back to my actual job. :-) But in short, this is why the current set of blindness made ebook readers are the way they are.
1
0
1
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge Fair enough. I have my own reasons for wanting to crack the problem of activating virtual cursor behavior for things that aren't Chromium or Firefox. Of course, I don't expect anyone to work on that unpaid on my behalf.
1
0
1
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt If ladybird and/or servo ever get off the ground this is going to be needed. But they haven't, yet.
1
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge Servo recently got funding specifically to work on accessibility, so I'm wondering how that's going to play out, and if they're going to want to get me involved given my work on Rust GUI accessibility.
3
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge It's possible, of course, that the Servo team will punt on Windows and just implement accessibility for Linux.
1
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt It feels like Windows would be the easier place to start, though, given Wayland and everything else that was still a mess on Linux last I checked.
1
0
0
0

User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge I guess all the platforms are messy in different ways. I'm just thinking, since the funding came from the Sovereign Tech Fund, they might prioritize getting results on Linux, where it's all open-source.
2
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge And I now have direct experience with the accessibility APIs on all the major platforms except iOS.
1
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt Did your work with gnome ever go anywhere? There were good ideas, there. Perhaps too many.
1
0
0
0
User avatar
Matt Campbell @matt@toot.cafe
8mo
@fastfinge I did a prototype in the first half of 2024. It's been sitting on hold because I've been busy with other stuff, and I also don't know if GNOME actually got another round of funding.
1
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt Good to know!
0
0
0
0
User avatar
🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca
8mo
@matt Depends how they define results, I guess. By features that exist? By the number of users that can now access the thing? Etc.
0
0
0
0