Using Leonardo.AI To Finetune My Stable Diffusion Model To Create Splinterlands Art

A few days ago I talked a little bit about a new project on the way called Leonardo.AI. There is a feature that I didn't include in the last post because I didn't explore it enough to talk about it and present its capabilities and flaws.

I'm talking about fine-tuning your own models in Stable Diffusion. First of all, I needed a target to work towards and luckily my inspiration was right here on Hive. I saw this Splinterlands Art Contest Week 229 post and that sparked the idea.

I'm going to create a fine-tuned model that can create some kind of Splinterlands art. There are so many amazing-looking cards that I didn't know what to choose from. I went with my all-time favorite element from video game, Water.

At this point, I wasn't really sure what to expect from this fine-tuning process or how exactly it works so I needed to pick a theme from the card available.

So I went with half-fish half-humanoid monsters. Clicked on the Train Model and... well waited, haha.

I was expecting to wait for at least an hour when I saw this message. But to my surprise, I got an email in about 20 minutes.

Well, it was time to try it out. My first prompt was aimed to create a character that looks like a
screaming Medusa with golden hair that is holding a trident. And this was the result:

Queue Fail Trumpet Sound

I mean, it was a first try, but I didn't want to give up on it yet, I started toying around with different styles for prompts and when I tried an anime style prompt I started getting something closer to what I wanted.

This looked... better but not quite. Also, bear in mind that none of these images are upscaled because I can only upscale 10 images per day so I had to pick and choose when it would be the right time to use those. So I figure, the image on the left was getting closer to my target so why not see how it looks upscaled, this is what I got.

Even though a lot of stuff on the image didn't make much sense the style in which the image was drawn was getting pretty close. Now, I have to say that this image was the last one that got anywhere close and after 50 or even more failed images it was getting obvious that I won't be making any more progress with this model.

So I went all the way back and retrained a model.

I added a bunch more monsters from the Water element and went a little bit looser on the theme and just wanted to add more different weapons and poses into the mix. In the help section of the Leonardo.AI website, they say that you shouldn't add too many training images to avoid overtraining the model. But hey, I had to try it and see for myself.

This started to look better from the start and my hope was high at this point. Instead of trying to get that golden hair medusa with a trident (which I didn't get btw) I opted to just let it create all sorts of characters and classes. And at one point I generated this one and just knew I want to use it for this contest.

This one looked cool and if you ask me the style is really close. Anyway, I loved it! But that wasn't good enough so I needed to add some modifications and do a little bit more manual work.

Image by Kevin Phillips from Pixabay

Found this neat-looking background on pixabay and incorporated it into the image.

Had to remove the first background, and added the new one, after that, I returned the glow on the left orb that was removed when I made a mask, and lastly some finishing touches. At the last moment, I noticed that tiny anomaly on her waist so I had to get rid of it.

And voila! This is my entry for the contest.

Medusa Mystic.

I hope you like the end result as much as I do and that you learned something new about AI capabilities along the way.

As always, thank you for reading.

IF you need help starting here at Hive, I highly recommend you to hop on The Terminal discord server, here's an invite: https://discord.gg/Wv5yQwyF