Brett Winton: 假设您想在桌面上运行一个 chatGPT 大小的语言模型。 不可能。 到目前为止。 （除非你想为每个单词等待一分半钟。） 本文介绍了如何以每秒 1 个令牌在单个 GPU 上执行此操作 github.com/mrcodechef/Fle…

原推：Say you wanted to run a chatGPT-sized language model on your desktop.

Not possible.

Until now. (Unless you wanted to wait a minute and a half for each word.)

This paper figures out how to do it on a single GPU at 1 token per second

github.com/mrcodechef/Fle…

https://twitter.com/wintonARK/status/1627879401328353280