日本語版
This article explains LoRA training for a specific theme in conjunction with the Cherry Blossom Festival event. As a note, I will be participating in this event myself, so I will omit specific examples. For detailed steps on training execution on SeaArt, I recommend referring to the previously written article with screenshots.
Types of LoRA
There are various types of LoRA. They are generally called "Style LoRA", "Character LoRA", and "Pose LoRA". Each has a different learning target. More specifically, the number of learning elements increases in the order listed. When learning a style, the entire screen is the learning target, whereas for poses, the background and art style are not learning targets. This difference in learning targets affects the Network rank dim parameter, which will be discussed later. For this event, we will train a LoRA focused on cherry blossoms.
Choosing a LoRA Theme
For this event, we will train a LoRA on cherry blossoms and spring to create an original model. Recommended themes include:
- FILM Cherry Blossom Studio
- Spring Poster
- Cherry Blossom Street Snapshot
- Ukiyo-e
- Cyber Cherry Blossom
- Makoto Shinkai-style
You are not required to choose from these themes. I personally like cyberpunk, so I plan to participate with a Cyber Cherry Blossom theme.
Creating a Dataset
After choosing a theme, create a dataset. The dataset creation steps are as follows:
- Determine Base Model
- Prepare Training Images
- Tagging
- Set Parameters
Determining the Base Model
This literally means deciding the base model for training. This event has no specified base model, so you can set it freely. My personal recommended models are:
- For SD1.5 series: AnyLoRA, CalicoMix
- For SDXL series: Animagine XL, AAM XL
- For Pony series: Use your preferred model
- For Illustrious series: Illustrious v2.0 (Pro version if possible)
- For Flux.1: Flux (base model)
These are just my personal recommendations.
Preparing Training Images
After deciding the base model, prepare training images. This step is actually the most critical in LoRA training and directly impacts the LoRA's performance. Prepare more than 20 images that align with the theme you decided in the previous section. The dataset images need to have consistent characteristics. In this case, the chosen theme represents those characteristics.Regarding the number of images, more is better, but don't stress too much about it.For details on training images, cropping, and tagging, please refer to the previous article.
Cropping and Tagging
Tagging is linguistically describing the elements of the training image. More intuitively, it's the prompt you would use to generate that image. There are five items to set:
- Cropping Method
- Cropping Size
- Tagging Algorithm
- Tagging Threshold
- Trigger Word
Cropping Method has three types: focus crop, center crop, and no crop. As the names suggest, focus crop is recommended as it automatically crops the image nicely. If you've already processed the background and size, cropping is unnecessary.
Cropping Size references the aspect ratio of the training images. However, you don't need to be too serious about this. It's fine to just determine if it's square, portrait, or landscape. Note that cropping size is set in bulk, so you need to look at all your training images when deciding.
Tagging Algorithm should basically be Deepbooru. The exception is if the base model is Flux. This is because other algorithms change the tag delimiter, making it less responsive during generation. Particularly for Illustrious-based models, Danbooru tags are very effective.
Tagging Threshold is recommended to be 0.6-0.65. Lowering the threshold too much can result in unnecessary or incorrect tags.
Finally, Trigger Word. This is literally the "trigger" that makes the LoRA effective. You can set it or not, but it's recommended to set one. You can set any word, but general words like "wallpaper" will get lost in other prompts, so choose a less common word. For this event, something like "Sakura" would work. Be careful to check if the word overlaps with Danbooru tags.
Parameter Settings
There are many parameters for LoRA training, but the four to set for this event are:
- Repeat
- Epoch
- Network rank dim
- Network alpha
We'll divide these into two pairs.
Repeat and Epoch: The product of these values and the number of training images becomes the total training steps. Set each parameter's value based on the total steps. As a guideline:
- SD1.5 base: 2000 steps
- SDXL series: 3000 steps
- Flux: 3000-4000 steps
Regarding dim and alpha, note that alpha has a mathematical significance in relation to dim. Dim is the parameter that determines the LoRA's learning scale. As mentioned in the "LoRA Types" section, dim values need to change depending on the learning target. For LoRAs learning an entire screen, like art style, you need to increase the dim value. For this event's specific theme LoRA, set a medium value.
Regarding the dim and alpha relationship: the closer alpha is to 0, the greater the LoRA's influence; the closer it is to the dim value, the smaller the LoRA's influence. This is simply a mathematically meaningful value, and alpha typically does not exceed the dim value. If you set alpha higher than dim, the LoRA effect may disappear or produce unintended effects.
Practical value combinations for this event:
- SD1.5 base: (dim, alpha) = (128, 64)
- SDXL series, Flux base: (64, 32) recommended
Basically, you don't need to set any other parameters.
Execute Training
Once the dataset is complete, execute the training. It will take time, so have a coffee while waiting. After a while, epoch previews will be displayed. Basically, choose and publish the last epoch. However, if you don't like the 20th preview, you can choose one you like from the later stages. Be flexible about this.
Publish the Model
Simply publishing the model does not meet the event participation requirements.
You must publish it with the #CherryBlossomSeason tag and submit it to the entry form. Models already submitted can be searched using the same tag. It might be good to look at them for inspiration.
