Note by @fastfinge

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

Second update: Yes, this is pointless security theatre. Any addon can download and execute remote code. So even if you do all of the work I mentioned, unless you audit the code, you still have no idea if the addon is safe or secure. This just gives users a dangerously false sense of security. The only way it would be meaningful is if NVDA code signs all addons, disables remotely downloading files, and only allows approved addons through the store. And doing this would be far, far worse than the alternative. It would mean, for one, that Eloquence would be completely dead as an NVDA addon. NVDA doesn't allow it in the addon store, and thus wouldn't sign it.

Update: it looks like the outreach to virustotal.com in my firewall logs was unrelated to NVDA. Based on a look at: github.com/nvaccess/nvda/blob/master/source/addonStore/models/scanResults.py

NVDA just accepts whatever the addon manifest says without verification. So instead of a privacy violation, this is just pointless. I can put whatever I want there in my addons, and it'll reassure the user that no viruses were found. To actually know the truth, a user has to:
* visit the URL
* hash there addon
* compare the hashes

And only then can they know if the results in the virustotal URL they visited are the same ones for the addon they installed, and that the information in the manifest is correct based on the actual virustotal.com findings.

By the time you do all that, either Windows Defender has flagged the virus, or you're already screwed.

I guess I'd rather pointless security theatre than privacy violation, if I have to choose. But can't I have neither?

It looks like in the latest #nvdasr alphas, it's now sending all of your addons to be scanned by VirusTotal. I did not give permission for this, and I do not want this. How long until NVDA stops addons it doesn't approve from running at all? For now I have virustotal.com blocked at the router. There seems to be no other way to block this.
'scanResults': {'scanUrl': 'www.virustotal.com/gui/file/2a83b713e38596cfbcb3f98b5eb91530ddfd0e9319907c6119cbbbe08f7acc88', 'malicious': 0, 'undetected': 67, 'harmless': 0, 'suspicious': 0, 'failure': 0, 'timeout': 0, 'confirmedTimeout': 0, 'typeUnsupported': 9}}
Traceback (most recent call last):
File "addonStore\models\scanResults.pyc", line 31, in fromDict
KeyError: 'virusTotal'

#screenreader #a11y #nvda #accessibility

NV Access @NVAccess@fosstodon.org

@fastfinge I think there's a misunderstanding on how the Add-on Store works. Every add-on in the store must keep a consistent hash which the VirusTotal scan reflects, which must be the exact same add-on you download from the Store. NVDA won't keep the download of an add-on without a matching hash. The scan provides useful results from well respected security vendors without every user needing to know how to audit code. This is done when the add-on is uploaded to store, not from your PC. 1/2

NV Access @NVAccess@fosstodon.org

@fastfinge 2/2 The log provided is a file read error, not a network error. We also have many warnings about the potential risks of add-ons. Signing add-ons would only prove the author, not that the add-on is safe. We're working on increased add-on security & would strongly suggest communicating security concerns privately. If any of these concerns were correct, this would be a dangerous example of irresponsible disclosure practices given you have not reached out to us privately with any of this.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@NVAccess As for reaching out privately: the fact that the entire security model here seems completely wrong to me isn’t a vulnerability. It’s endemic to your design. And I have no idea what the store does with it, but I can absolutely put my own virustotal results in a manifest of an addon distributed outside of the store. So NVDA can’t make any assertions based on that data unless it could guarantee the addon was installed from the store. And that would involve…I guess having a local list of the hashes of every single addon? The only way you can secure addons is by restricting the functions available to them, in ways I’m certain I and other users would find unacceptable. So your only workable recourse is to develop a model that strongly relies on author trust, reputation, and audit ability. To be totally honest, this feels like NVDA has not done threat modeling at all, doesn’t understand what threats an “addon security” system is attempting to prevent or what’s realistic, and hasn’t put long and hard thought into balancing the security triangle for users. Instead, from the outside, a virus total scan looks like “we must be seen to be doing something, and this is something.”

Tyler Spivey @tspivey@dragonscave.space

@fastfinge @NVAccess If Virustotal results were delivered with the manifest, I would agree with that. But they're not; I just looked at one of my addons. I think you have a slight misunderstanding about how all this works.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@tspivey @NVAccess My understanding is that NVDA adds them in during its build/test process. They’re then stored on the users machine, fully accessible to addon authors to add our own fake results, or overwrite others, or do whatever we want.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@tspivey @NVAccess I can see the dictionary that contains them in the Python Console. So how they’re delivered really doesn’t matter, if we assume either that I’m malicious or am working for another malicious actor.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@tspivey @NVAccess I also think it’s fair to say that any malicious payload delivered via an NVDA Addon is going to be targeted to NVDA users. So you have to assume it knows about NVDA’s security process. So once again, unless I’ve misunderstood a lot more than I think I have, a Virus Total scan is nothing more than a placebo at best. I’d love to be wrong here.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@tspivey @NVAccess I’ll grant you that I was incorrect about it being delivered in the manifest. But from the perspective of a malicious addon author, I’m not sure how it’s delivered really makes any difference if I can still intercept and modify it at will. It’s still data that the users local copy of NVDA, running on the user’s machine with the user’s addons of unknown providence installed, cannot trust. I admit I’m not the expert either of you are, but I can’t think of a way around this that doesn’t have other awful side effects. It would be safer for everyone for NVAccess to work on developing a trust model for addon developers and a reputation system for addons, rather than pretending it can make any claims at all about the safety of an addon. Again, unless I’m wildly wrong somewhere. And if I am, I’d love to know.

Tyler Spivey @tspivey@dragonscave.space

@fastfinge @NVAccess If I ran a malicious addon, yes, that addon could modify the VirusTotal results of another one. But at that point, the author already has code running on my machine that can do whatever it wants, so there would be no reason to do that. When you open the store for the first time, it warns you: "Add-ons are created by the NVDA community and are not vetted by NV Access. NV Access cannot be held responsible for add-on behavior. The functionality of add-ons is unrestricted and can include accessing your personal data or even the entire system." That warning seems clear enough. I'm sure if there was a malicious addon on the store, NV Access would remove it. But you run them at your own risk.
Advocating for a better security model might not be a bad idea, though I don't know what that would look like. It also takes resources I don't think NV Access has.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@tspivey @NVAccess Right. But what I don’t understand is why they’re spending resources on this. It doesn’t inconvenience any users, sure. But it also doesn’t inconvenience any attackers. And for inexperienced users, a virus total result saying “0 threats sounds the same as saying this addon is safe. What NVDA can do is make assertions about who developed any given addon. Users can then decide if they trust that developer or not. IMHO, that’s where effort should be spent. Not in pretending to play a threat detection game that even the likes of Google and Microsoft can’t win.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@tspivey @NVAccess Also, I have two trusted addons, one in the store. Give me $500 per addon and I’ll make sure everyone running my addons never see that your addons have viruses. And when your malicious addon eventually gets removed, mine will probably remain unnoticed in the store so I can sell my result modification to my next customer. This is a joke! Not a real offer! But someone could easily do this. The only practical defence is for users to know and trust the developers of there addons. Not just to be not evil, but also to be skilled. Once you have a single addon installed, you can no longer trust anything NVDA reports unless you trust that addon author. That’s why I think we need pgp keys and author identities front and center. All a virus total scan can do is, at best nothing, and at worst convince a user that an addon is safe when it isn’t.

NV Access @NVAccess@fosstodon.org

@fastfinge @tspivey I really cannot stress enough how irresponsible it is to have a discussion about what you believe may be a security hole - in ANY software - on a public forum. We, like other software companies, have published ways of responsibly disclosing information like this, as well as readily available email contacts. Even if you're not sure if something is a bug or a security hole or not, or just have a general suggestion for improvement, ask privately & we'll happily check.

Tyler Spivey @tspivey@dragonscave.space

@NVAccess @fastfinge For my part in this, I was only trying to explain what was going on to avoid spreading wrong info around. If there was an actual security issue, I would of course report it privately. I've always gotten a response quickly, even if the issue I report doesn't turn out to be one. Whether Mastodon is a good place to have these kinds of technical discussions about NVDA's design is another question, since I assume the entire team isn't going to see it. Email is probably a better option there.

NV Access @NVAccess@fosstodon.org

@tspivey @fastfinge Thanks Tyler, no your information was spot on and much appreciated!

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@NVAccess @tspivey So the answer to a poorly thought out security model is to demand that discussions of it happen in private? That’s an even worse decision than the one I’m upset about.

MostlyBlindGamer @MostlyBlindGamer@dragonscave.space

@NVAccess @fastfinge responsible disclosure of what? A user read something in their logs that looked like a feature, not a bug. As is expected in FOSS, they didn’t implicitly trust you. I looked at your open source code and assuaged their concerns, one of the key benefits of the FOSS model. What was even disclosed here?

The rest of the conversation is a whole other story: NVDA exists in a complex environment where it needs to satisfy a wide range of users and stakeholders. That NVAccess can’t make every single one completely happy is a foregone conclusion.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@MostlyBlindGamer @NVAccess Public criticism of the security model is disclosure I guess. Other than complain about inaccuracies I’ve gone out of my way to correct, NVaccess has declined to respond to the actual problem here: virus total scanning is ineffective, and gives users a false sense of security, while being easy for malicious addons to work around. That’s not an exploit! That’s the system working as designed!

NV Access @NVAccess@fosstodon.org

@fastfinge @MostlyBlindGamer We actually went out of our way to answer all your initial point, which were mostly incorrect assumptions. The point about responsible disclosure is simple: you found something you believed enabled a malicious actor to spread malware through add-ons - the responsible thing is to share that with us privately so we can fix it if needed. By sharing it publically first, you potentially allow someone to exploit a vulnerability you didn't give us a chance to fix.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@NVAccess @MostlyBlindGamer This is, to be blunt, not true. You corrected assumptions that my post was edited to correct hours before you got involved. The rest of the thread is you complaining that the design of security systems should be conducted completely behind closed doors. I am doing my best to believe you are conducting this discussion in good faith. But I’m starting to struggle. As you are the nv access official account, it’s starting to look like nvaccess just doesn’t want any public discussion or criticism of its design choices.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@NVAccess But this is utterly meaningless. Any addon can download and run any code it wants, from any URL. So there is no guarantee the addon isn’t running malicious code. As long as the addon can download and run its own code, the virus total results can’t mean anything. So they’re security theatre, as I said. Either NVDA blocks addons from downloading arbitrary files to the user’s machine completely, NVDA manually reviews all addon code (and the code of any network servers the addon uses), or the Virus Total results in the store are useless.

Jayson Smith @jaybird110127@dragonscave.space

@fastfinge @NVAccess Sorry for jumping in here, but just pointing out that NVDA reviewing code downloaded from external servers is pointless, as if you're a malicious addon creator, you can just wait for NVDA to give your addon a clean bill of health and add it to the store, then flip a switch on your server so that the good, clean, harmless code you were downloading to satisfy the review process is now replaced with the worst virus/trojan/malware imaginable.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@jaybird110127 @NVAccess I thought that was obvious enough to not need saying. But perhaps it wasn’t, so thank you for saying it. Chrome addons, and even Android and IOS apps, do this constantly. But Google and Apple have to keep fighting the good fight, and ever restricting functions, because there are just so many app developers that expecting users to know who any of them are or who they can trust is impossible. In the much smaller community of addon developers, that’s really not the case.

Tyler Spivey @tspivey@dragonscave.space

@fastfinge You're most likely hitting this issue that was fixed: github.com/nvaccess/nvda/issues/19984
Reading that and the linked PRs, you might need to delete the .json file for some addons for that error to go away.

Tyler Spivey @tspivey@dragonscave.space

@fastfinge As far as I know, the store does this when someone submits an addon, not NVDA.

Sascha @saschacowley@beige.party

@fastfinge @pixelate This only applies to Add-on Store add-ons, and it is done as part of the GitHub Actions workflows that run on nvaccess/addon-datastore. Unless you choose to visit the VirusTotal URL, your machine is entirely unaffected

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@saschacowley @pixelate Then why is my machine reaching out to that virustotal URL that showed up in the log snip and timing out?

Sascha @saschacowley@beige.party

@fastfinge @pixelate Uh, that is weird and bad. HAVE you considered filing a ticket?

Alex Hall @alexhall@mastodon.social

@fastfinge I really hope that's a temporary thing they forgot to take out or something. The intent of keeping users safe is nice, but I feel like uploading things I chose to download to a random virus checker that I didn't approve or ask for is a really bad move.

James Scholes @jscholes@dragonscave.space

@alexhall It was mentioned in the What's New document, so it probably isn't an accident. @fastfinge

James Scholes @jscholes@dragonscave.space

@fastfinge

The only way it would be meaningful is if NVDA code signs all addons, disables remotely downloading files, and only allows approved addons through the store.

Such checks could arguably be meaningful for the Add-On Store even if they were limited in scope to only apply to the Add-On Store.

As in: if downloading from the Add-On Store, you could trust that NV Access had done X, Y, and Z in the interests of security. If you chose instead to bypass the store or install from a different one, that would be entirely on you and come with an appropriate set of warnings.

Frankly it could be an inappropriate set of warnings as long as it was still possible.

@alexhall

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@jscholes @alexhall But that’s never, ever what happens. As soon as any infrastructure exists to stop you running code they can’t control they always use it to stop you doing things they don’t like. In the name of security, of course.

James Scholes @jscholes@dragonscave.space

@fastfinge I can't argue with you on the basis of history. You've been proven right time and time again.

However, I would like to hope that on this occasion, @NVAccess and the community will be willing to discuss and avert these concerns. Rather than assuming the worst and then being angrily smug when it comes to pass.

@alexhall

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@jscholes @NVAccess @alexhall The best way to avert the issue is to make infrastructure decisions that don’t allow it to happen. With a little thought that’d easily be possible. But slapping a virus total scan that can’t be helpful doesn’t demonstrate the kind of thoughtful approach needed.

Jonathan @jonathan859@someplace.social

@fastfinge If that’s the security they’re talking about implementing for corporate environments, that’s definitely going to backfire.

MostlyBlindGamer @MostlyBlindGamer@dragonscave.space

@fastfinge well, I figure I’d end up cloning the repo at some point.

The scan results object (including URL) is part of the add-on view model. Without forcing myself to read more Python, but also having noticed that object is deserialized from JSON, it seems most likely that the store server sends the client the fully auditable virus scan results.

Did you find any communication to VirusTotal in your logs?

MostlyBlindGamer @MostlyBlindGamer@dragonscave.space

@fastfinge clarifying my last question: in your firewall logs, because this is not communication with that domain.

MostlyBlindGamer @MostlyBlindGamer@dragonscave.space

@fastfinge

The add-on data model:

github.com/nvaccess/nvda/blob/master/source/addonStore/models/addon.py

The scan results model where the URL comes up:

github.com/nvaccess/nvda/blob/master/source/addonStore/models/scanResults.py

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@MostlyBlindGamer I did. It could have been unrelated; there's a lot of stuff going on this network. But what would be the point of just including the URL to the results? I can just put anything I want in the manifest and say it passes. For it to mean anything, NVDA has to reach out and check. So this is either a privacy violation, or total security theatre. Not good either way.

MostlyBlindGamer @MostlyBlindGamer@dragonscave.space

@fastfinge who puts it in there?
If it’s the developer, I’d check server-side, before making it available in the store. I don’t have a problem with sharing the URL with the client for transparency and auditing.
If NVDA do the check, that would only be server-side.
There’s no reason to have all the clients hammering VirusTotal servers.
I’d run a packet sniffer and navigate through the store to confirm the possibility that they’re doing something very silly.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@MostlyBlindGamer It looks like it gets put there when it's uploaded to the store. But I develop addons that aren't distributed via the store, because of silly NVDA rules. So I can just put whatever I want there. And I haven't actually tested to see what happens if I upload an addon to the store with the virustotal keys already included.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@MostlyBlindGamer I don't even know why they're doing this. If you don't have a virus scanner enabled on your computer, this will not save you. Any addon can just download remote code whenever it wants to. That is, in fact, how most addon update checkers work. So even if you audit the results, it's completely meaningless unless you also audit all the code. The only thing this does is give users a false sense of security.

MostlyBlindGamer @MostlyBlindGamer@dragonscave.space

@fastfinge it’s about the [puts on sunglasses] optics.
[The Who blares in the background]

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@MostlyBlindGamer Yeah, but I know you've deceived me, Now here's a surprise. I know that you have, 'Cause there's magic in my eyes. I can see for miles and miles, And miles and miles...

MostlyBlindGamer @MostlyBlindGamer@dragonscave.space

@fastfinge not the same CSI, but very appropriate.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

An actual security solution:
* allow user reviews of addons
* allow users to report addons
* remove addons from the store after X number of reports
* have a reputation system for addon developers (How many addons? How many versions? How long have they been around?)
* allow high reputation developers to do code reviews of other addons and submit the results

This would help for addons in the store. For addons not in the store, users are on their own. However, even for non-store addons, NVDA could do things that would reduce risk:
* Check PGP keys. An addon that claims it was by fastfinge but isn't signed by my PGP key won't run. Public keyservers already exist; NVDA doesn't need to build infrastructure, or in any way gatekeep or endorse developers to do this.
* reserve addon names: fastfinge is the known developer of unspoken-ng. So flag a big warning before running a version of unspoken-ng developed by BobTheBad man. And don't run a version of unspoken-ng not signed by fastfinge's known PGP key at all.

There are ways to let users understand if an addon can be trusted, or how much, and who made it, without centralizing on the store, pointlessly scanning with VirusTotal, etc. NVDA addon security based on restricting functionality is never going to work. So instead, we need to create the tools to build trust models of developers, and know exactly who wrote and signed off on the code we're running.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

The blindness community is also quite interconnected. If we wanted to, a "web of trust" would be more possible for us than other communities. We regularly gather at conventions (NFB, ACB, Sight Village, Zero Project, etc.) so with accessible, easy to use and understand tools, keysigning parties could easily happen. We've already moved to the #Fediverse because it's more accessible, and obviously better for our needs. Let's keep thinking differently about centralization and privacy; we might discover there are other methods that will work better for us and our needs, without restricting our rights, privacy, or functionality.

Drew Mochak @prism@infosec.exchange

@fastfinge Wouldn't that encourage brigading? Not to mention an NVDA person would need to review all the reports. Particularly in the AI age, I can easily see someone spamming a bunch of realistic looking reports from accounts. That's not to say they couldn't be discovered, but its extra effort for a job no one wants.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@prism This is why you track reputation. Number of false reports, account age, etc.

NV Access @NVAccess@fosstodon.org

@fastfinge We do allow & encourage user reviews of add-on. Users can report add-ons & we will remove suspicious add-ons from the store. You can verify an author's history by looking at their Github. The community already do informal code reviews in the add-on group.

1/2

NV Access @NVAccess@fosstodon.org

@fastfinge 2/2 You cannot pre-make your own VirusTotal results for an add-on, that's not how it works (& if you find a way to), please RESPONSIBLY disclose this). We introduced the store to have a central, consistent way to update add-ons, not for control, but to discourage add-ons downloading their own updates without any verification.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@NVAccess But you haven’t stopped that from happening. Any addon can still do that. So what threat, exactly is Virus Total defending against? Users who run Windows Defender will already have it intercept known viruses. Unknown viruses aren’t likely to be picked up by defender or Virus Total. Malicious addon authors can just add code into the addon that will download the virus later, and thus never be scanned by Virus Total. I guess maybe you’d catch supply chain attacks? Those of us who distribute outside of the NVDA store can continue doing whatever the heck we want, but now we can also stick fake Virus Total keys into the addon.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@NVAccess And I have to push back here, as well. When I go to the addon manager, and brows for and install addons, where do I see this information? There’s no rating like in the chrome or edge addon store. No number of reviews. No number of installs. No easy way to see other addons by the same developer, so I can judge if they’re an active part of the community, or malicious, or what. No contact info for the author. Maybe this is all on a website, or in a group I’m not subscribed to. But for the purposes of helping a user judge if an addon is trustworthy or not, that’s effectively nowhere.

Joshua @J3317@allovertheplace.ca

@fastfinge @NVAccess it's in the other details section, tab around and you should find it.

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

@J3317 @NVAccess Do you mean the text field with unclickable links?

Joshua @J3317@allovertheplace.ca

@fastfinge @NVAccess Other Details: edit read only multi line Alt+ o Channel: Stable