From f2fc4be69b67d0dea8a3304d6013ddb8dd97deaf Mon Sep 17 00:00:00 2001 From: Andrej Karpathy Date: Thu, 29 Dec 2022 17:26:42 +0000 Subject: [PATCH] mention 4gpu loss as well in readme --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ce1b44e..2a53afd 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ To my knowledge, running this with the current script with the GPT-2 hyperparame $ python sample.py ``` -Training on 1 GPU overnight currently gets loss ~3.74. Random chance at init is -ln(1/50257) = 10.82. Which brings us to baselines. +Training on 1 A100 40GB GPU overnight currently gets loss ~3.74, training on 4 gets ~3.60. Random chance at init is -ln(1/50257) = 10.82. Which brings us to baselines: ## baselines