$\mu\mathrm{P}$:前身、原理与后续发展
$\mu\mathrm{P}$ 的初始化与学习率缩放、可复现的 MLP 实验,以及深度、优化器和 MoE 中的扩展。
$\mu\mathrm{P}$ 的初始化与学习率缩放、可复现的 MLP 实验,以及深度、优化器和 MoE 中的扩展。
I curate some learning materials for CUDA Programming
I curate some learning materials for CUDA Programming
I summarize some self-learning resources for machine learning systems. Please stay tuned!
I summarize some self-learning resources for distributed systems.