Photo by Lucas Cao

Exploring '1girl' Prompts Across Checkpoints in Stable Diffusion: A Quick Look

misc2023-08-15109.6 min

00 Foreword

While studying Stable Diffusion, a sudden idea came up: what can be observed by using the same prompts to generate images?

Prompt: 1girl
Negative prompt: nsfw, ng_deepnegative_v1_75t, easynegative,**badhandv4**
Sampler: Euler a
ADetailer: Enable

01 Toon You

02 Anything V5

03 AWPainting

04 AWPortrait

05 twingShadow

06 墨幽人造人

07 cyberrealistic

08 meinamix meina

09 majicmix Realistic

10 leosamsMoonfilm

11 GhostMix

12 dreamshaper

13 deliberate

14 Counterfeit

15 Chilloutmix

16 Bayonetta

0x99 Extended Thoughts

Q01: How do we evaluate the quality of a model?

  1. Accuracy of Prompts -- If the model cannot generate a smiling face when prompted to produce a “smile”, then it’s inaccurate.
  2. Avoid Unnecessary Details
  3. Forcefully linking a rainy day with an umbrella is adding unnecessary details.
  4. Quality of Generated Images - If the generated images are aesthetically pleasing, it suggests that there are no issues with the model itself.

Q02: Does the quality of a model directly correlate with simple prompts?

For example, if the quality of images generated from the ‘1girl’ prompt is high, does this mean the model is of high quality?

Not necessarily. If a model is optimized specifically for the ‘1girl’ prompt, it might produce high-quality images for that prompt. However, when new prompts are introduced, the model might not respond accurately or it might produce random images, indicating the inaccuracy of the prompts.

With so many Checkpoint models, what are the categories? Why are many models on Station C based on 1.5?

Models can be divided into three categories: Base Model, Checkpoint Trained, and Checkpoint Merge.

  1. **Base Model **: This includes versions like v1.4, v2.0, v2.1, and v1.5. Most people fine-tune their models based on v1.5 as it has the most robust ecosystem. It’s similar to Java 8, Python 2.7, or Vue 2 in the Stable Diffusion (SD) world. The latest version is SDXL 1.0.
  2. **Checkpoint Trained **: These are models that have been trained further with images, based on the Base Model. An example would be DreamShaper.
  3. **Checkpoint Merge **: This category involves merging different models to combine their weights. For example, the weight of Model A might be 0.3, and that of Model B might be 0.7. An example of this would be GHOSTMIX.

Q03: Why do some models produce grayish images? How can this be resolved?

For example, when I use the AOM3_orangemixs model, the generated image appears as follows:

This is likely because the model’s author did not incorporate a Variational Autoencoder (VAE) into the model.

Given the color variation that different VAEs bring to this model, I have not baked a specific VAE onto the model.
The VAE used in the sample image is kl-f8, and I appreciate its color saturation.However,
perhaps you prefer orangemix.vae (NAI.vae) or others? Please feel free to try them out.

  1. Download the VAE.
  1. Go to Settings → User Interface → QuickSettings.
  2. Activate the options for sd_vae and CLIP_stop_at_last_layers.
  3. Add the downloaded VAE into the model.
    Q05: Why is there a severe issue of homogenization in models?

Some models are created by merging high-popularity models, including the Orange, Crayon, Anything, and CF models.

A characteristic of the Orange model is that the surface of the characters appears shiny or greasy. This is what’s often referred to as the “AI oily look” problem. The issue of homogenization often arises when multiple models share similar characteristics or are derived from similar base models, leading to a lack of diversity in the results.

0xEE Reference

  1. SD 模型理论科普

Leave a Comment