Jump to content

AI-Generated Game Textures and Concept Art


Josh

8,094 views

 Share

Midjourney is an AI art generator you can interact with on Discord to make content for your game engine. To use it, first join the Discord channel and enter one of the "newbie" rooms. To generate a new image, just type "/imagine" followed by the keywords you want to use. The more descriptive you are, the better. After a few moments four different images will be shown. You can upsample or create new variations of any of the images the algorithm creates.

imagine_command_popup_click.thumb.gif.3185fa8d74dc59007f76813e6319845e.gif

And then the magic begins:

167756032-0059cb74-d437-4747-8778-902c03403be6.gif.bcb3dc6ef99057b813c417a3f1632650.gif

Here are some of the images I "created" in a few minutes using the tool:

JoshKlint_ukraine_post-apocalyptic_urban_environment_wasteland__119c3db6-173c-40b8-8675-f7e7a4f101d6.png.5f147f95c98d568122cbe811582bf1fc.pngJoshKlint_strogg_enemy_210988df-22e0-43f3-8856-bbb3f2868f8c.png.f94ecc9284eb4e0646b8ed754ed1ad06.pngJoshKlint_quake_map_id_software_game_9ea0bca3-83ac-4694-8da0-97cb18ef11bb.png.43a1f351f62589594e158c746f5e1efb.pngJoshKlint_quake_map_id_software_game_3b7b564d-8737-4367-9175-cba17ca4c358.png.8e21166df257e1261eb230aae6bf8f13.pngJoshKlint_doom_2016_game_level_544a65f1-5a2f-43f7-9fd1-4ce96561b5dd.png.059dc8d28f2f85efc669a3a57798865a.pngJoshKlint_cyberpunk_cthulu_204afbaf-acb1-461b-8a44-e0400716fa5f.png.0aac204066b166274d6c2b096b61dca2.pngJoshKlint_cyberpunk_city_photorealistic_4aa102b8-b8bc-401d-ba8f-9f8ffd6a55a8.png.d57eb2c2f7b3230e0f835ca8fec2406b.png8b3d5967-0fc7-4ebe-842d-b2155fdddc56_progress_image_96.jpg.ed18179feffd2b7b171aed8de66ce957.jpg

I'm really surprised by the results. I didn't think it was possible for AI to demonstrate this level of spatial reasoning. You can clearly see that it has some kind of understanding of 3D perspective and lighting. Small errors like the misspelling of "Quake" as "Quke" only make it creepier, because it means the AI has a deep level of understanding and isn't just copying and pasting parts of images.

What do you think about AI-generated artwork? Do you have any of your own images you would like to show off? Let me know in the comments below.

  • Like 1
 Share

24 Comments


Recommended Comments

I've been using Dall-e. Similar concept, type in a description of what you want and it tries to generate that picture.  You can also specify what style you want. Oil painting ,3D graphic or pixel art etc. The misspelling  you are getting is deliberate. Its to stop people creating offensive texts  or ask for a  famous person to hold a sign saying "I am stupid" or worse.

Below is Dall-es' attempt at "UltraEngine" title graphic.

 

1373314715_DALLE2022-09-0806_48.44-_ultraengine_titleinsci-fimetalwithaspacebackground.thumb.png.dac05ba360220cc2c1fee873ec526132.png

1793890919_DALLE2022-09-0806_47.25-ultraenginetitleinmetalwithablackbackground.thumb.png.ba21c3ddbd9d78636b51e248b1976021.png

  • Like 1
Link to comment

The power of these programs is amazing. 

It really will change the way we create art.

Here is a Rabbit walking sprite sheet from Dall-e.

641175759_DALLE2022-09-0818_23.27-cartoonrabbitwalkingspritesheet.thumb.png.850d80ed7119812840f427f46683d53b.png

Normally this would take ages to create in some paint program, now you just type a one line sentence.

  • Like 1
Link to comment

More startlingly good images. If you could just train it on a set of game textures the output would be much better:

download.jpg.63cbd249d6b2b1bb88396556280fdb56.jpg2013742266_download(3).jpg.09312cce81fd5cae10f8ed2d3a9e1869.jpgforest.jpg.3c14db9b11d9140ea3be3853021d94b7.jpgbunker.jpg.092af2acdc1ade51690c3f15431375bb.jpg1670310526_download(2).jpg.217219dfba9d9baa5fe8ce477de5f359.jpg1706082668_download(1).jpg.88feb5ba16a8ca850348781eec81536e.jpg1952065846_download(5).jpg.6a399142c52337f8b0b7645234e61bf0.jpg216097751_download(4).jpg.61c74d061688edcf5c5d04cbb9490529.jpgdownload.jpg.feaf88618eb8a7380d34da71fe55bcba.jpg

  • Like 1
Link to comment

Midjourney seem to make better artistic images then dalle.

I think is just trained with billions of images and has some kind of image recognition and classifications and is just copy pasting, deforming images in a clever way.Maybe im wrong.

Probably is also using input from the people generating the images to learn what good image means.

 

Anyway this is good news for indies by the look of it :)

I plan to use midjourney for some 2d game experiment.

 

old man sitting on a mountain near the sea with a maelstrom swirling in front of him , providence , cthulhu monster , H P Lovecraft world , the color from the sky

 

aiaf_old_man_sitting_on_a_mountain_near_the_sea_with_a_maelstro_21cac0da-f58b-4ed3-9d4a-12a5a7bc9352.png.036ec963ff921725b966be6f9d87c01c.png

  • Like 2
Link to comment

I've been playing with these for a few months now ... these image generators will put a lot of people out of a job. Including me, to some extent.

anyway: Dall-e and midjourney gave me mixed results, and I got threatned to get banned for trying to get some gory textures. Stable Diffusion is the new kid in town, and it can be run locally, on colab, or on their website, and the code is open source, so you can just bypass the NSFW filter and get all the blood and guts you want. also boobs, if you are okay with them having three nipples, or no nipples, occasionally. Texture generation has also been disappointing, so far.

but then there's centipede diffusion, a predecessor of stable diffusion. runs on colab and creates a bunch of small images, quickly, and uses those then as input images for the diffusion, and finally does upscaling. here, you can actually get very good textures, - you'll have to make them tilable afterwards, and the process is very slow. But since it doesn't run on your GPU, but on one of Google's. It's also much easier to control the style - if you really want a specific quality in your image - it helps to know the specific painterly qualities of individual artists, though, you can ask for that. Here's some meat and fat and skin, with the painterly qualities of a bunch of 19th and 16th century painters.

 

[image removed]

  • Like 2
Link to comment

This is all extremely entertaining and all but I've not yet seen a single image actually useful for any production of anything to be hones.

  • Haha 1
Link to comment

I also tried training playform.io with all the Gothic textures from Quake 3 Arena. The results are interesting, and clearly have a similar feel to the source material. However, I would not call these images useful for anything:

2119585122_MyFirstProject_50_0063.thumb.jpeg.a03c2b9b1263563608b7b6be25b767ad.jpeg1381468279_MyFirstProject_50_0047.thumb.jpeg.5bde7d6c3d1433a6ab9345923e20655c.jpeg1502099897_MyFirstProject_50_0002.thumb.jpeg.13300b0a8d0c41ccbe000b6b614bdf6f.jpeg1801959369_MyFirstProject_50_0004.thumb.jpeg.d5915438aa4109bc66975a942f0cc667.jpeg

  • Like 1
Link to comment

I didn't try midjourney yet, but it really interests me. I saw some really cool concept images made with it. However, I've been playing with Dall-e recently for some basic concept images.

Here's some test for some Horror Lovecraftian realistic images (I can't remember the actual keywords I've inputed)

746818442_DALLE2022-08-2418_16.34-AbigspaceshipinthemiddleofthemiddleofBotesvoidonlylituupbyitsoutsidelightsandwindowscalledtheM.O.R.R.I.G.A.N.thumb.png.958c9aa9c96cdfe22f0e39202772f41b.png1508999831_DALLE2022-08-2418_19.05-Lovecraftianhorror.thumb.png.fbd211015fbffbf8795a883a3f778138.png301844331_DALLE2022-08-2418_19.11-Lovecraftianhorror.thumb.png.8d93c57fd9c7f50b27f06392893bbcc9.png2110510435_DALLE2022-08-2418_19.26-Lovecraftianhorror.thumb.png.2720f64ae4e7441fb8fe54fbc5e8912a.png

 

However, just like @chsch because I wanted some horror related images, I got threaten of being banned from using DALL-E, even though I didn't ask for gory stuff.

  • Like 2
Link to comment

Generative Adversarial Networks (or GAN's for short) are the approach taken to get this kind of generative model using deep learning techniques. Supervised learning is the most common method of training GAN's which are usually made up of two sub-models, a discriminator and a counterfeiter. The counterfeiter has the role of generating fake images, where the discriminator then must classify those images as either real or fake. The ultimate goal is to converge training to a solution where the counterfeiter gets so good at generating fake examples the discriminator can then no longer classify the examples as real or fake.

My masters thesis (which I finally finished a few days ago) explored some pretty interesting areas of training supervised learning problems in the computer vision field. Most datasets are typically composed of real-world examples of images, its often an expensive and time-consuming task to create these datasets of images as it not only requires hardware in terms of camera equipment but also heavy investment of valuable time to annotate the images (supervised learning for object detection involves labelling images which is the practice of drawing bounding boxes around the objects within the image in order to provide the model labelled data to learn from). Automating the entire process of data collection is the pursuit of happiness. Applying technologies such as game engines to render the images and also program the logic for automatic annotation of objects in a scene provides an end-to-end approach for data collection. The possibilities to simulate abstract imagery within a game engine opens up the doors for high-quality data that is collected in an efficient way for building large-scale datasets. Synthetic data could improve these kinds of models as we can simulate almost anything we like within a game engine. Feeding the resulting image data that is correctly formatted for model input could really push the results these types of models can achieve.

Check out differentiable rendering, that is crazy stuff and the generalization power is improving over the years 

 

  • Like 4
Link to comment
On 9/9/2022 at 6:06 PM, Josh said:

@chsch Can you train centipede diffusion with your own images?

technically, yes- but realistically, no. There was a person who offered custom trained models though, which resulted in a few very weird image generators . Ukiyo-e diffusion was relatively "useful", but she also released a thing called "textile diffusion" which would create everything as stitched into fabric. She was doing this fo a few hundred bucks if you let her release the model publicly, and a few hundred more if you wanted to keep the model to yourself. - She doesn't do that anymore, though, she got hired by Stable Diffusion.

Stable DIffusion, as I found out by now, can do tiled textures, and it can do image to image. Meaning, you can use it as a filter that creates variations of a given image- or texture. Say, you have one texture of rocks from texturehaven - you can make variations of that, photorealistic or stylized.

Stable diffusion also offers a thing called textual inversion, where you can train it on a handful of images to "learn" a concept, and assign a variable nameto that concept, so you can then ask it to make images of 'variable name' in the style of something else, or to create whatever, in the style of 'variable name'.

It's not quite training a whole model, and I haven't tried out how well it works - but right now, the only thing that makes this not absolutely perfect for the job is that generating images larger than 768x768 takes more VRAM than google colab offers me.

Link to comment
On 9/9/2022 at 9:22 PM, Josh said:

I also tried training playform.io with all the Gothic textures from Quake 3 Arena. The results are interesting, and clearly have a similar feel to the source material. However, I would not call these images useful for anything:

1381468279_MyFirstProject_50_0047.thumb.jpeg.5bde7d6c3d1433a6ab9345923e20655c.jpeg

 

I do have a fine art education, and honestly, as far as capital A Art is concerned, these are great.

  • Haha 1
Link to comment

Not 100% those will tile correctly, but these should be fine for a base texture. I'm sure eventually we'll see plugins for Stable Diffusion that'll make the all the textures for a PBR material and be accurate. 

Link to comment

tiling is a feature of SD, though and has been a mere checkbox away for a while. PBR is likely going to be messy, though, but I was quite impressed by a model trained to create depth maps from pictures of trees.... in the meantime: SD and materialize are now all you need.

Also, there's a blender plugin to create tiling textures from within blender.

Link to comment

I have a Stable Diffusion server on my machine with a RTX 3060. Since I installed Easy Diffusion and not something like Automatic111, I don't seem to have much flexibility, but Easy Diffusion had an automatic installation. 

Link to comment
Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...