每秒百万token,Nori LLM是在画饼还是在划红线?
原创
每秒处理超过一百万token——Nori LLM抛出这个数字时,我第一反应不是"好快",而是"凭什么"。在推理延迟已经卷到毫秒级的当下,吞吐量才是大规模部署真正的瓶颈,而它直接把这个数字推到了新的量级。但问题在于,百万token每秒背后是多大的硬件代价?压缩了多少精度?还是只是换了个benchmark口径来比?目前这条新闻只有1分0评论,说明社区还没给出有力回应。我的判断:如果Nori真能在合理成本下兑现这个吞吐,它对边缘部署和实时对话场景会是实质性的推进;如果只是极限条件下的峰值数字,那它更像一张漂亮的幻灯片。
原文:Nori LLM: Achieving Over 1M tok / s · 来源:Hacker News
版权声明
所有资源都来源于爬虫采集,如有侵权请联系我们,我们将立即删除
相关文章
- 0.05ms 往返:localhost 为什么不能直接当网站?
- Fornax's Guide to Fast Ethernet
- Possibly leaked NSA Cyberattack tool behind attacks (2017)
- Steve Wozniak on Writing Apple Integer Basic
- PayByPhone is stealing $50M/year from their users
- Grand Theft Auto 6: A Landmark Achievement That Redefines a Console Generation
- Research Acceleration: The Jagged Frontier
- Google Embraces Geothermal Technology to Vaporize Rock for Clean Power
- Microsoft's Agent Framework PyPI package compromised to steal credentials
- Nvidia's Seven Billionaires Have Lost a Third of Their Wealth
最新文章
- 0.05ms 往返:localhost 为什么不能直接当网站? 11小时前
- Fornax's Guide to Fast Ethernet 11小时前
- Possibly leaked NSA Cyberattack tool behind attacks (2017) 12小时前
- Steve Wozniak on Writing Apple Integer Basic 12小时前
- PayByPhone is stealing $50M/year from their users 12小时前
- Grand Theft Auto 6: A Landmark Achievement That Redefines a Console Generation 12小时前
- Research Acceleration: The Jagged Frontier 13小时前
- Google Embraces Geothermal Technology to Vaporize Rock for Clean Power 14小时前
- Microsoft's Agent Framework PyPI package compromised to steal credentials 15小时前
- Nvidia's Seven Billionaires Have Lost a Third of Their Wealth 15小时前
itfan123





