smolgym

Questions

Before you write.

The short answers. The loop on the front page is the longer one.

  1. What does Tinker SDK compatible mean?

    The calls match the Tinker SDK. You create a training client, then sample(), forward_backward(), optim_step(), and save_state(). Supervised training skips sample(). The floor is a small model, or weights you upload, on one machine. A run that needs a cluster is outside that.

  2. What do I write, and what do you run?

    You keep the data, the environments, the reward, and the loss. The script can sit on a laptop. SmolGym loads the model, samples it, runs the backward pass, and keeps the checkpoint.

  3. Does supervised training use the same loop?

    Yes. Supervised training skips sample(). The script sends examples. We run forward_backward() and optim_step().

  4. When is the update LoRA, and when is it the whole model?

    LoRA writes the adapter. The base stays loaded, and several runs can share it. A full update writes every parameter, and only when the whole model still fits one machine.

  5. What fits the floor?

    One training machine. A small open model on the rack, or weights you upload, if they fit that machine. A run that needs a cluster does not run here.

  6. How do custom weights work?

    You upload them. The loop is the same loop. Change the model by changing one string, or by uploading the weights. You download the trained weights when you want them.

  7. Who holds the weights during a run?

    Two copies sit on the GPUs. The sampler reads one and does not train. The trainer writes the other when you call optim_step(). Sync copies the written weights onto the sampler, so the next sample() sees the update. Your script sends the loss. It does not hold W.

  8. How does a run start?

    By email to swapnil@rycerz.es. There is no account. Say which model, whether the loop is supervised or reinforcement, and what the reward checks.

  9. Can I download the weights?

    Yes. save_state() keeps optimizer state, and the weights are yours to download.