Introduction
Yo, what's up? long time no see, miss me ? I hope not ;P!! Alright, let's get straight into it.
Today, we’re diving deep into my process of training a LoRA model, and by the time you finish reading this, you should have a solid understanding of how to train a better LoRA yourself. No more guessing, no more confusion, I got you covered. Now, I'm breaking this tutorial series down into two parts to keep things clean and easy to follow:
🔥 Part 1: Dataset Section – We’ll go over dataset preparation, what makes a good dataset, and how to get the best results.
🔥 Part 2: LoRA Training – Here, I’ll show you my training setup, the presets I use, and some game-changing tips to level up your results.
This ain't your grandma's tutorial 😉, so expect a more hands-on, straight-to-the-point guide. I'm not just throwing random info at you, I'm actually sharing the presets I use and dropping some tips that have helped me get better results. One important thing to keep in mind, some values might need tweaking depending on your dataset. What works perfectly for me might need a little adjusting on your end. So don't be afraid to experiment and fine-tune things until you get the results you're looking for.
Datasets
When it comes to crafting a high-quality LoRA model, your dataset is the real MVP no joke!!!
Think of your dataset as the blueprint for your LoRA model. Its a collection of images and captions that guides your model to capturing a variety of artistic styles of character and other thing, get that balance right, and you ready to go But here's something you want to consider if your dataset is too small, you'll likely run into overfitting issues that limit your model’s flexibility. On the flip side, a massive dataset might seem ideal, but it can stretch out training times. So, it's all about finding that perfect balance of enough variety to keep your model sharp and versatile.
Key Aspect
- Diversity and Size: A diverse dataset helps the model generalize better by including various styles, subjects, angles, and lighting conditions. While a minimum of 15 images is poorly recommended for style, I recommend using around 45 to 70 images is often ideal for achieving good results without overfitting the Image .
- Quality: Images should be high-resolution (e.g., 1024x1024 pixels or more), clear, and focused on the subject rather than the background.
- Dataset Design: The dataset should be designed to focus on specific styles or subjects if desired, such as landscapes or portraits. Even style including text captions can help the model understand the relationship between text and image features.
Creating a Dataset
You can create dataset from various web images or created using AI image generation tools
- e.g for anime: danbooru, Pixiv, deviant art,fancaps, Pinterest,etc),
- (e.g for realistic dataset, 500px, Pixabay, Shutterstock, Unsplash. Most of the time I will often use stock photo websites)
- Make sure you're dataset is free from errors or poor-quality images or watermark is crucial for effective training (you never want those watermark by any chance)To create good datasets for LoRA training, focus on variety, high quality, and consistency for styles, characters, and concepts here some following explanation and tips
Dataset specific
Style Dataset
This is a dataset to teach AI models about certain art styles without focusing on one character or specific object. For example, if you want to make Ai understand the art style of watercolor, this dataset will be filled with various images from different objects (humans, animals, landscapes, etc), but all of that should use the same watercolor style, also same thing for artist style. Image Count: For training a LoRA model to capture a specific art style, it's recommended to use a dataset of 35 to 100+ images.
Character Dataset
This dataset is specifically for teaching an AI model about a particular character. It includes multiple images of the same character in different poses, expressions, clothing, and backgrounds. The goal is to enable the AI to consistently generate this character in various situations Image Count: Use 15-40 images if the character LoRA doesn't require a lot of detail. Too many images can cause overcooking of LoRA. Aim for 10 to 15 high-quality images for optimal results, but you can use as few as 10 or more than 20
Concept Dataset
- This dataset is designed to teach an AI model to understand and generate specific concepts. These can be objects, scenes, abstract ideas, poses, lighting conditions, or even composition techniques. The goal is to either introduce entirely new elements to the model or enhance its understanding of existing concepts for better generation quality.
- Recommendations: Keep the theme consistent – All images should reinforce the same concept without distractions. Include multiple variations – Use different characters, settings, and angles to help the AI generalize better. Maintain high image quality – Avoid blurry, low-resolution, or cluttered images, as they can confuse the model.
Example creating dataset
Trick 1 (Anime character only)
If you have only 1 IMG of your character, I recommend use this tools 👉 360 Creator
- Upload your full-body image into that tool.
Download the generated video.Go to this site( 📦 gif--png ) to convert your video into image frames
- Upload your video and set the frame rate to the lowest possible before converting.
2. Download the extracted images.
3. Organizing Your Dataset: Now, split your images into the following categories:
Upper body: (5 images) ,Lower body: (3 images),Face close-up: (5 images)
Side view: (3 images),Back view: (3 images),¾ view/profile: (5 images)
4. Image Editing & Upscaling:Open your favorite image editor (Photoshop, Photopea, GIMP, Paint, etc.).Upscale your images to at least 756px, but ideally 1024px or higher for better quality.If there are watermarks, remove them using the Spot Healing Brush in Photoshop (or similar tools).
Trick 2
This method involves using Google Collab.Open this Collab Notebook 👉 Collab notebooks
Follow the guide until Step 3 on that notebook (since we'll use another site for training).In the extra panel, click:
✅ "Reduce dataset file size, and clean only."Go to Google Drive, find your cleaned dataset, and download it.
Final Adjustments in an Image Editor:Open Photoshop (or any preferred editor).Upscale some images to 1024px+ (depending on the aspect ratio).
If you're lazy and have a powerful PC, use this automatic upscaler 👉 Upscale (Upload images, wait, and done!)
Finally, remove any watermarks or logos to prevent unwanted artifacts in your LoRA model.
Trick 3
This trick basically using your fav AI image Generator tools
1.(character)
For character itself it is very straightforward just put your prompt in detail since this is anime dataset then for prompt is e.g (1gir or 1boys, solo, full body,standing, expression,hair color, hair style, eye color, body type, clothing type, boots type, other accessories, random word) , set your setting into Euler A or any your favorites, steps 35 cfg 5.5 make sure seed always set to same number to maintenance the consistency then hit generate .
2.Style
for style this quite tricky, basically you need to add some other people lora to your img generation(2 or 3 lora), try to add different character , background only (scenery), animal,and some abstract stuff, or if you have something cool on your generation result you can generate more with same prompt style but different subject
3.Concept
.for concept (this is long story so you're better finding something else or ask someone else since it will take times to create).basically you generate same concept with different subject (e.g pose concept, your character doing something weird pose then what you need to do is just generate the same pose with different subject and background) 
IF you have times to watch some movie you can also grab some screen capture those move per frame . What you need to do is just capture the essential part of that movie into IMG (character, style, or even concepts)
