osmarks/nanogpt-experiments
1
0
Fork 0
mirror of https://github.com/osmarks/nanogpt-experiments.git synced 2024-11-14 05:44:51 +00:00
Code Issues Releases Wiki Activity
177 Commits 1 Branch 0 Tags 2.3 MiB
594068e7ae
Commit Graph

12 Commits

Author SHA1 Message Date
Andrej
d9f4735f5e
Merge pull request #10 from LaihoE/master 
batch file write
 2023-04-13 00:39:41 -07:00
Christian Orr
36c7db8c44
bugfix in decode function 
Return was left out of the decoder, so it didn't work.
 2023-03-08 10:16:19 +02:00
DG
2f7fd0ac57 add relative import in shakespeare 2023-01-22 12:18:24 -08:00
DG
bf779456f3 add relative import in shakespeare_char 2023-01-22 11:11:25 -08:00
DG
edb7a7eab0 use relative paths so that running the data prep scripts always create files in local folder, no matter where run from 2023-01-20 10:39:45 -08:00
Andrej Karpathy
2c7806db6e for consistency with previous commit 2023-01-19 23:10:51 +00:00
リョウゼ
be571fff2c
Improve readability of huge numbers 
Before:
  length of dataset in characters:  1115394
  all the unique characters: 
   !$&',-.3:;?ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
  vocab size: 65
  train has 1003854 tokens
  val has 111540 tokens

After:
  length of dataset in characters: 1,115,394
  all the unique characters: 
   !$&',-.3:;?ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
  vocab size: 65
  train has 1,003,854 tokens
  val has 111,540 tokens
 2023-01-16 22:05:32 +01:00
Andrej Karpathy
d17350a31d add support for character-level language models, a new character-level shakespeare dataset, a new config file that shows how to train a character-level baby GPT on it, and adjust the sample function to figure out if it should decode with characters or GPT2 bpe tokens. The current implementation is a bit hacky and basically assumes just these two possibilities. In the future we may want to support more general encoders or decoders. 2023-01-11 05:27:19 +00:00
Laiho
0a2ea95338 batch file write 2023-01-02 17:49:21 +02:00
Andrej Karpathy
2febf4463c candidate changes to apis, have to think through more 2023-01-01 01:29:48 +00:00
Andrej Karpathy
7c6ea8409e simplify the prepare script a lot, write only using one process, seems sufficient for now. ty @LaihoE for suggestion and @proger for flagging 2022-12-30 22:18:20 +00:00
Andrej Karpathy
fe8042867c first very bad commit 2022-12-28 00:58:19 +00:00