Category: AI

  • How To Get AI To Output Images In the Style of My Own Artworks

    How To Get AI To Output Images In the Style of My Own Artworks

    I have been experimenting with getting generative AI-models to output images that imitate the style of my own artworks. I have mainly two reasons for this:
    1) I want my own art style to shine through to make the images unique and blend well with other artworks in the game that I may have created from scratch. I also do manual edits to the AI-generated images to remain in full control of the final results. These edits also blend very well into the rest of the image when it’s in the same style.
    2) I want to actively avoid imitating other artists styles. Inspired by? Sure! Imitate? No thanks.

    1. Train the AI-Model On Your Own Artworks

    There are many models and services out there. Many of the services that are specialized on image generation have functionality to customize the output of the models. I have tried several different ones. The one I use at the moment is Leonardo AI. I will show how to do it there, but similar workflows can be implemented in many other services.

    1) In Leonardo AI I use the function called Elements. Access it by clicking on More and then Train Your Own Model.

    2. Click Your Elements and Train New Model.

    3. Choose Style. Press Next.

    4. Choose an existing Dataset or create a new one by pressing Create New.

    5. Choose a name for your new Dataset and upload your images. Here you can also see an estimate of the cost to create the dataset and train the model. How many Model Tokens you have will depend on your subscription. Currently, every 15 images in the dataset will increase the Training cost.

    You can select up to 50 images, but in my experience you don’t need that many images to create a great style. I have got great results with as few as five images. It is important that the images you choose are high-quality and have a consistent style while also offering a variety of subjects, settings etc. This is an example of five of my own original artworks I have used to create a style I like:

    6. Go through the remaining steps and confirm that you want to start the training. This will take some time. You will get notified when your new Style is ready.

    2. Create Lots of Images Using the New Elements Style

    1. Go to the Image Generator in Leonardo AI. Press the “image”-button next to the text prompt. Go to Your Elements and select the new style you have created.

    2. The Element is added to your prompt window. You can set the Strength of your Element. Write your text prompt and see if you like the results.

    I usually ask Leonardo to produce 4 or 8 images at a time to get a wide range of alternatives to select from. It is often helpful to get the object/character against a neutral background with a single color. This helps me a bit during the last step in my workflow, where I combine several images and do my own modifications in Photoshop.

    3. Most of the time I create the character and the background separate from each other. This makes it easier to have control over the composition of the image, by combining different parts manually later.

    4. Another trick is to use the edit function (via Nano Banana) in Leonardo AI to change a generated image into a black-and-white line drawing. This allows me full control over the color scheme, by doing the coloring and the lighting myself in Photoshop, while still saving time by generating the base line drawing.

    3. Make Manual Modifications, Combine Images and Finish the Artwork

    1. The AI-models are getting better at details such as hands and feet. Nonetheless it is still very difficult to get the models to get all the details exactly as you want them with text prompting alone. I usually have to manually edit some or many parts of the generated images to have full creative control of the end result. This is also a lot faster than trying to find the perfect prompt, at least if you have some artistic experience and know how to use an image editor.

    2. This is also where I combine several images into a coherent composition. Usually one or several characters and a background. I also adjust or add shadows and lighting as well as adjust levels, saturation etc.

    An added bonus with this last step is that the resulting artwork is legally more clearly a creative work done by you, the human. In most jurisdictions this is often important if you want to claim copyright over the finished work.

    This workflow allows me to create artwork rapidly while staying in creative control. Please let me know what you think!

  • Experimenting with Stable Diffusion AI

    Lately I have seen a lot of example of AI generated images in my Twitter feed. The new Stable Diffusion AI have been mentioned. Recently the code and all necessary stuff to run the code was released to the public. I have not yet tried to install it on my computer. I am not even sure it will run on my laptop with Iris Xe graphics.

    I were still very eager to try some prompts with Stable Diffusion. After some searching I found out that it is actually made available in a very user friendly and easy way at NightCafé. You get to generate a few images for free each day. If you want to do more you can sign up for a subscription or buy credits that can be used to generate more images. I could not resist the temptation, so I got some credits and started generating images like a mad man. Let me share some results after my first two days of AI frenzy.

    Test #1

    Goal: Generate a beautiful visualisation of a dreamlike cityscape. Highly detailed painting. Beautiful lighting. Vivid colors.

    Prompt: City in sunset, Sweden, Balboa trees, 8k, 4k, intricate details, vibrant colors, high focus, unreal 5 render, rhads, Bruce Pennington, Studio Ghibli, tim hildebrandt, digital art, octane render, beautiful composition, trending on artstation, award-winning photograph, masterpiece
    Prompt: City in sunset, gothenburg, Sweden, snow and ice, glimmering, extremely detailed oil painting, intricate details, unreal 5 render, rhads, Bruce Pennington, Studio Ghibli, tim hildebrandt, digital art, octane render, beautiful composition, trending on artstation, award-winning photograph, masterpiece”
    This one I really like. Prompt: City in sunrise, Balboa trees, 8k, 4k, intricate details, vibrant colors, high focus, art noveau, unreal 5 render, rhads, Bruce Pennington, Studio Ghibli, tim hildebrandt, digital art, octane render, beautiful composition, trending on artstation, award-winning photograph, masterpiece
    Prompt: Art noveau city and colorful Balboa trees in sunrise as William Turner, epic in scope and scale, 8k, intricate details, vibrant colors, impressionism, high focus, unreal 5 render, Bruce Pennington, tim hildebrandt, studio Ghibli, digital art, octane render, beautiful composition, trending on artstation, masterpiece

    The results turned out better than I expected. I find the images very beautiful and the composition feels strikingly coherent, compared to earlier iterations of AI image generators. I would love to have some of these images as prints at home. It is interesting to note the difference in style in the last image where I added William Turner and impressionism.

    Test #2

    Goal: Generate colorful and highly detailed portraits inspired by Egypt gods.

    Ibis. Prompt: Egypt godess of the underworld ibis, intricate green and gold design, highly detailed, sharp focus, art by artgerm and greg rutkowski and wlop, detailed painting, 8k resolution concept art, hyperdetailed, polished illustration, impressionism, vivid colors
    One of the generated images above. Upscaled 6x by the AI scaling at NightCafé.
    Osiris. Prompt: Egypt old male god of the underworld osiris, intricate white and gold design, highly detailed, sharp focus, art by artgerm and greg rutkowski and wlop, detailed painting, 8k resolution concept art, hyperdetailed, polished illustration, impressionism, vivid colors, symbolism
    Sekhmet. Prompt: Egypt goddess of the lions sekhmet, intricate red and gold design, highly detailed, sharp focus, art by artgerm and greg rutkowski and wlop, detailed painting, 8k resolution concept art, hyperdetailed, polished illustration, impressionism, vivid colors, mythical
    Some tests turned out a bit weird. Prompt: Egypt god Thoth by Anna Dittmann and Alphonse Mucha. Incredibly detailed, maximalist matte painting. Portrait of a god. Polished illustration. Colorful hue of green, gold, black. Glimmering darkness. 8K resolution, HD, DSLR, realistic oil painting.

    Test #3

    Goal: Generate a profile image for me and for this blog that encapsulates the phrase “urban meeple”.

    This is the one I like the most so far. It took some trial and error to get there. Prompt: A chess piece in a cyberpunk city by Simon Stålenhag, Irina French, Paul Gauguin, Dan Mumford, Cyril Rolando trending on Artstation hyperdetailed Unreal Engine romanticism filmic photorealistic post-apocalyptic sci-fi digital illustration. Vibrant colors. Chess Queen. Epic scale.”
    First try. Beautiful images, but the AI did not pick up on the phrases game pawn or meeple. Prompt: A game pawn in a cyberpunk city by Irina French, Paul Gauguin, Dan Mumford, Cyril Rolando trending on Artstation hyperdetailed Unreal Engine romanticism filmic photorealistic post-apocalyptic sci-fi digital illustration. Vibrant colors. Urban Meeple
    Second try. The AI makes the pawn or chess piece as big as a building and the shape is often very abstract. Prompt: A very big chess piece in a cyberpunk city by Simon Stålenhag, Irina French, Paul Gauguin, Dan Mumford, Cyril Rolando trending on Artstation hyperdetailed Unreal Engine romanticism filmic photorealistic post-apocalyptic sci-fi digital illustration. Vibrant colors. Chess Queen. Epic scale.”

    Check out my profile at NightCafé if you want to see some more examples.

    This is so much fun to play around with! I need to get a proper graphics card and install Stable Diffusion on my own computer. Or else I will use up all my money to buy computing time at NightCafé instead.

    Some of the prompts that are circling around in the community is hilarious. One very common phrase is “unreal 5 engine”, another one is “trending on artstation”. These are added to the prompt almost by routine, with the hope that it will improve the quality of the generated images. And maybe they do. After all, I guess a large part of the dataset that the AI has trained on is composed of images from Artstation, and the ones that are trending there are usually spectacular images And images based on the unreal engine really can be extremely photorealistic. But I have also seen very impressive images without these trendy phrases. It is also fun to see what artist names are used in the prompts in order to simulate their particular style. Artgerm, Greg Rutkowski and WLOP are extremely popular and they are often used in combination on prompts. Studio Ghibli is also frequent. It’s almost like a magic spell used to summon a demon or a creature from the other side. Instead of searching some old dark magic book in a library for a spell, you search the community image archive for prompts.

    It is really exciting to hit the button labeled “Create”, then wait to see what will appear. Almost as exciting as it is to find out what creature was summoned in the magic ritual in the fantasy story. Was it what you were hoping for or was it something completely different, an abomination? Maybe I should have added “unreal 5 engine” or “hyperdetailed” to the spell…eh, I mean…to the prompt.