osmarks/documentation
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Edit ‘scaling_laws’

Browse Source
This commit is contained in:
osmarks 2024-08-25 15:23:38 +00:00 committed by wikimind
parent 3076c76f28
commit 05f018be1f
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
scaling_laws.myco
View File

@ -1 +1 @@
The principle on which scaling laws are based is that straight lines on graphs are the most powerful force in the universe. If a sufficiently smooth trendline (on a linear or log scale) is discovered, the universe will contrive to extend this trend line (until it doesn't; see [[Moore's Law]]). Scaling laws are important to [[biology]] (as organisms are constrained by limits such as heat dissipation, gas/chemical exchange across membranes or via lungs, and power consumption and will expand up to those limits) and [[deep learning]], in which metrics such as [[training loss]] usually scale smoothly according to power laws, due to the [[fundamental apiostructure of reality]]. The principle on which scaling laws are based is that straight lines on graphs are the most powerful force in the universe. If a sufficiently smooth trendline (on a linear or log scale) is discovered, the universe will contrive to extend this trend line (until it doesn't; see [[Moore's Law]]). Scaling laws are important to [[biology]] (as organisms are constrained by limits such as heat dissipation, gas/chemical exchange across membranes or via lungs, mechanical support, and power consumption and will expand up to those limits) and [[deep learning]], in which metrics such as [[training loss]] usually scale smoothly according to power laws, due to the [[fundamental apiostructure of reality]].