Sunday, September 13, 2026

Training LR update Formula


Let B0 = 124 and LR0 = 0.025 (your tuned baseline).

For a new effective batch size B:

LR_linear = 0.025 * (B / 124)

LR_sqrt   = 0.025 * sqrt(B / 124)

Use linear if B is within 2-4x of 124.

Use sqrt if B is much larger (e.g., >1000) or if linear diverges. 

No comments:

PC Magazine Tips and Solutions

PC World: Latest Technology News

PCWorld.com - Most Popular Downloads of the Week