osmarks/nanogpt-experiments
1
0
Fork 0
mirror of https://github.com/osmarks/nanogpt-experiments.git synced 2024-12-18 14:10:28 +00:00
Code Issues Releases Wiki Activity

include launch command too. anyone should be able to do this now

Browse Source
This commit is contained in:
Andrej Karpathy 2023-02-03 22:17:05 +00:00
parent f9348f3f18
commit 3969860ff5
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
config/train_gpt2.py
View File

@ -1,4 +1,6 @@
# config for training GPT-2 (124M) down to very nice loss of ~2.85 on 1 node of 8X A100 40GB # config for training GPT-2 (124M) down to very nice loss of ~2.85 on 1 node of 8X A100 40GB
# launch as the following (e.g. in a screen session) and wait ~5 days:
# $ torchrun --standalone --nproc_per_node=8 train.py config/train_gpt2.py
wandb_log = True wandb_log = True
wandb_project = 'owt' wandb_project = 'owt'