Strategies for Prompting Self-Trained SDXL Character LoRA, Insights from Thousands of Images

Introduction

After generating thousands of images, I kinda became familiar with SDXL LoRA. Along the way, I've developed some principles and tips for evaluating your character LoRA (selfie/celebrity)

Self-trained SDXL LoRA can significantly affect the generated image quality. Here are some tips for avoiding this issue and maximizing the effectiveness of your self-trained LoRA.

How to train your character LoRA?

All of my training steps can be found in the link below.

I primarily referred two resources:

Firstly an article

https://aituts.com/sdxl-lora/?ref=simpleaiart.com

Secondly and video:

Prepare Images

To prepare images I utilize #awportrait checkpoint and ControlNet. The goal is to generate a set of 15 images, each taken from a different angle of the head. This diverse collection ensures a comprehensive representation of the face.

Unlike GozukaraFurkan, Ido not currently use regularization imagesin my image preparation process. However, I will explore this approach in the future, especially when fine-tuning the LoRA model.

The results far exceeded expectations

Once the LoRA training was completed, I obtained a high-quality Character LoRA. I now utilize this for#AIphotographyto create portraits in various styles and themes.

These include**cyberpunk, night scenes, Western cowboy, astronaut, Assassin's Creed, Witcher, and Final Fantasy VII Remake cosplay.**It's insanely great!!

Leveraging the power of#SDXL, I am able to capture photographs in an array of themes and styles, such as embodying the beauty of the four seasons.

Spring / Summer / Autumn / Winter Theme/Stylish

Tips and Strategies for Crafting Effective Prompts

While#SDXL and#LoRAmay not be perfect, the results can be truly astounding, particularly with portrait shots. If you follow my training methodology, you'll find that some prompt engineering is essential for enhancing the quality of your photography.

1. The usage of LoRA can significantly affect your image, even if you do not use a trigger word. Image R1C1 vs R1C2, business suit / gray background / medium close up / emotion

2. Adding "business suit, gray background" to negative prompts can reduce the fundamental effects.
3. The SD_XL_Base tends to generatepeculiar skin textures(P1). It's advisable to switch to an SDXL-based checkpoint for improved results. e.g. 'xxmix9realisticsdxl' aligns more accurately with the mainstream Chinese aesthetic standards for skin representation. (P2) (PS: xxmix9realisticsdxl skin are toooooo smoooth, but not that weird in most cases)

4. Creating a pure cartoon version of a portrait is more challenging due to the LoRA effect. However you can also* prompt cartoon-style to create semi-realistic art.*

5. To prompt a wide shot:
   1. Add**'close-up shot'to negative and'wide/long shot'**to positive prompts.
   2. More environmental details.
   3. To capture a full-body shot, describe the color of the shoes. This will make SD aware of the need to include feet. Without these tips, you're more likely to get a close-up/mid-closeup-shot.

6. Utilize ChatGPT to automate the generation of prompts. By providing GPT-4 with a few examples, it can conceptualize themes, topics, and styles on its own. Your role would be to select high-quality images and fine-tune the prompts accordingly.