Quoted From Reddit:
Release: https://github.com/bghira/SimpleTuner/releases/tag/v0.9.8
It's here! Runs on 24G cards using Quanto's 8bit quantisation or down to 13G with a 2bit base model for the truly terrifying potato LoRA of your dreams!
If you're after accuracy, a 40G card will do Just Fine, with 80G cards being somewhat of a sweet spot for larger training efforts.
What you get:
- LoRA, full tuning (but probably just don't do that)
- Documentation to get you started fast
- Probably better for just square crop training for now - might artifact for weird resolutions
- Quantised base model unlocks the ability to safely use Adafactor, Prodigy, and other neat optimisers as a consolation prize for losing access to full bf16 training (AdamWBF16 just won't work with Quanto)
not a fine-tune, but, Flux-fast
frequently observed questions
10k images isn't a requirement for training, that's just a healthy amount of regularisation data to have.
Regularisation data with text in it is needed to retain text while tuning Flux. It's sensitive to forgetting.
you can finetune either dev or schnell, and you probably don't even need special training dynamics for schnell. it seems to work just fine, but at lower quality than dev, because the base model is lower quality.
yes, multiple 4090s or 3090s can be used. no, it's probably not a good idea to try splitting the model across them - stick with quantising and LoRAs.
thank you
You all had a really good response to my work; as well as respect for the limitations of the progress at that point, and the optimism on what can happen next.
I'm not sure whether we can really "improve" this state of the art model - probably merely being able to change it without ruining it is good enough for me.
further work, help needed
If any of you would like to take on any of the items in this issue, we can implement them into SimpleTuner next and unlock another level of fine-tuning efficiency: https://github.com/huggingface/peft/issues/1935
The principle improvement for Flux here will be the ability to train quantised LoKr models, where even the weights of the LoRA itself become quantised in addition to the base model.
