Detailed view for this entity.
“But there was a training run that happened in this crypto project called Bit Tensor Subnet 3. They managed to train a 4 billion parameter llama model totally distributed with a bunch of people contributing excess compute, but they were able to do it statefully and manage a training run, which I thought was like a pretty crazy technical accomplishment.”