osmarks/nanogpt-experiments
1
0
Fork 0
mirror of https://github.com/osmarks/nanogpt-experiments.git synced 2024-11-10 20:09:58 +00:00
Code Issues Releases Wiki Activity

add reference for 6ND to notebook too

Browse Source
This commit is contained in:
Andrej Karpathy 2023-02-04 22:07:32 +00:00
parent eae986c2d2
commit 0bb96d3fff
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
transformer_sizing.ipynb generated
View File

@ -358,7 +358,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"This is not a bad estimate at all. I trained this model and it converged in roughly 4 days." "This is not a bad estimate at all. I trained this model and it converged in roughly 4 days. Btw as a good reference for where 6ND comes from and some intuition around it I recommend [Dzmitry's post](https://medium.com/@dzmitrybahdanau/the-flops-calculus-of-language-model-training-3b19c1f025e4)."
] ]
}, },
{ {