Note by @fastfinge

@luiscarlosgonzalez @mush42 @Tamasg @ZBennoui Right but Sherpa onnx might replace that. Or the various Microsoft AI providers. We don’t need to care about cross platform for NVDA. And so few people in this community have the expertise to write our own inference framework, I feel strongly that we really need to use something off the shelf. Otherwise, history will repeat itself, and the person working on the framework will get a job or otherwise be forced to drop the project. I can only think of four blind people who even know rust. And I’m not one of them.And the others all have full time jobs.