Summary
Use one AI to take any instructions I choose and create an ultra high-quality 2D image of an arbitrary 3D model, then use another AI to convert that into an actual 3D model for use in e.g. Unity3D
Usefulness: Practical/Productive
NB: if you want to use 3D models with Scratch (the visual programming language from MIT) and google sent you here, instead check out: https://ninja-script.com/
Techniques
- Start with MidJourney, which still (Spring 2024) produces by far the best high-quality AI generated images with a large amount of fine-control (rejected: Dall-E, it still generates so many awful low-quality images with very little control of the output)
- Then convert to 3D using a specialised Large Multi-view Gaussian Model hosted on Replicate.com: https://replicate.com/camenduru/lgm , which is using the tech in this 2024 paper: https://me.kiui.moe/lgm/
Discoveries
- Total setup time: 10 seconds + (setup a Replicate.com account + setup a Midjourney account)
- Processing time: approx 5 minutes (using Replicate.com’s free-tier / trial package)
Details / Implementation
Input prompt to MidJourney: “/imagine classical statue of male in style of michaelango holding a spear and shield, plain white background“
Replicate hosted model: https://replicate.com/camenduru/lgm
- Used default settings
- Only replaced the input image, gave it a URL to my MidJourney image (copy/paste link – MidJourney images are public URLs by default)
Output directly from model: