Skip to content
FlixMLFlixML Studioopen-source generative AI image and video tool

Training a LoRA

Training runs on AI Toolkit on a machine with your own GPU, Windows or Linux. Studio talks to it only through its HTTP API, so the trainer can be the Studio machine or another one on your network.

Setup

Section titled “Setup”

Start the AI Toolkit UI with AI_TOOLKIT_AUTH set, then point Studio at it:

Variable Description
AITK_API_URL AI Toolkit UI URL, e.g. http://<trainer-host>:8675
AITK_API_TOKEN Required. The AI_TOOLKIT_AUTH value the trainer was started with
AITK_GPU_IDS GPU IDs to train on — optional, default 0

Templates

Section titled “Templates”

Studio ships one template per base model in its training/ folder; base_config picks one. Each cites the sources its settings come from.

  • sdxl_character — any SDXL-architecture checkpoint, from a single .safetensors
  • flux2_character — FLUX.2-dev

Workflow

Section titled “Workflow”
  1. Put the dataset on the trainer: images plus one .txt caption per image, in AI Toolkit’s datasets folder (its UI, or POST /api/datasets/upload).
  2. Start training with POST /api/lora-training/start.
  3. Watch it on the LoRA Training page, then download checkpoints when done.

Trigger word

Section titled “Trigger word”

Short, unique, non-dictionary token.

  • Good: ch4rtrig, xy_char01, midnight_tarot
  • Bad: sam, ana, alex, or the character’s real name

Common words fight the base model and dilute identity learning.

Captions

Section titled “Captions”

Format:

<trigger>, a person, <scene description>

Example:

ch4rtrig, a person, sitting at a wooden desk in a navy blazer, soft side lighting, medium shot

Rules:

  • Start with <trigger>, <class word>, where the class word is the generic category of your subject (e.g. a person, a robot, a creature). For FlixML character LoRAs this is usually a person.
  • Describe visible scene details (pose, clothing, setting, lighting, framing).
  • Do not describe identity-specific features (face shape, eye color, hair color).
  • No quality tags (masterpiece, 8k, photorealistic).
  • Keep it to 15–30 words, neutral tone.

The class word lets the base model handle the generic concept while the trigger absorbs identity.

Starting training

Section titled “Starting training”
POST /api/lora-training/start
{
  "job_name": "character_v1",
  "trigger_word": "ch4rtrig",
  "dataset": "character_v1",
  "base_config": "sdxl_character",
  "model_name_or_path": "<path to the checkpoint on the trainer>"
}

That is all a run needs. The recipe (rank, learning rate, resolution, steps, previews) comes from the template. Any recipe field can still be set in the request (steps, lora_rank, learning_rate, sample_prompts, …; see the Agent API); an unset field keeps the template’s value.

sample_prompts are the previews: the trainer renders one image per prompt every sample_every steps, and the LoRA Training page shows them. [trigger] in a prompt becomes the trigger word. Unset, the run previews the template’s prompts; the start response lists the ones it will use.

Monitoring

Section titled “Monitoring”

Every status, step count and message comes from ai-toolkit. When Studio can’t back a job up with the trainer, it says so instead of showing its last saved status.

  • GET /api/lora-training/trainer — the trainer itself, asked live: whether it answers, its GPUs, its queues, every job on it (including ones Studio didn’t start) and its folders.
  • GET /api/lora-training/jobs — every job. An active job carries the trainer’s status, step and info; unreachable means the trainer didn’t respond, failed with an error means the trainer has no record of it.
  • GET /api/lora-training/status?job_name= — step, total steps, speed, ETA, loss history, settings and saved LoRA files for one job.
  • GET /api/lora-training/log?job_name= — the trainer’s own log for the job. Send back the offset from the last response to get only new lines.
  • GET /api/lora-training/samples?job_name= — training previews on disk.
  • GET /api/lora-training/checkpoints?job_name= — downloadable LoRA weights.
  • GET /api/lora-training/datasets/{name}/items — a dataset’s images and the caption each is trained on, read from the trainer.

The LoRA Training page shows every run: progress, time left, speed, loss, settings, the trainer’s GPU, the previews by step (next to the training image when a preview prompt is that image’s caption), and the saved LoRA files. Jobs are started through the API.

A dataset named after a character’s ID shows on that character’s page; any other dataset shows on its run.

References

Section titled “References”