Nvidia GANverse3D "inflates flat images into realistic 3D models"

by Mark Tyson on 19 April 2021, 13:11

Tags: NVIDIA (NASDAQ:NVDA)

Quick Link: HEXUS.net/qaeqhw

Add to My Vault: x

Nvidia research scientists have developed a tool called GANverse3D - an Nvidia Omniverse extension which can be fed a standard 2D object image, a single photo, and create a realistic 3D model. These full 3D models can then be visualised and controlled in virtual environments in Omniverse. Nvidia reckons this rapid 3D modelling tech could be a boon to architects, creators, game developers and designers. While the 3D models might look a little basic (reminds me of N64 graphics) they can be polished up later by 3D artists if required, after prototyping ideas.

The above work comes via the Nvidia AI Research Lab in Toronto. There they have been developing GANverse3D to "inflate flat images into realistic 3D models". GAN is an abbreviation of 'generative adversarial network' an increasingly popular AI and ML technology. Combined with real-world data sets the GAN can do a pretty good job of knowing the unknown / unseen aspects of an object. One of the key attractions and advancements of Nvidia's GANverse3D is that previous models for inverse graphics have relied on 3D shapes as well as multi-angle image training data.

Before feeding the single image of the Knight Rider KITT car to the Nvidia GANverse3D, the researchers trained the AI with a library of car images, taken from all angles. No 3D datasets, as mentioned above, but the 3D mesh model result looks pretty good considering the ease of workflow. Nvidia added GANverse3D generated wheels and headlights too. Brought into Omniverse with Physx and real-time ray tracing applied, a developer could get a good feel for driving KITT in a virtual world quite quickly.

Further details about GANverse3D are available via the Nvidia Blog and the full ICLR paper, authored by the Nvidia researchers.



HEXUS Forums :: 10 Comments

Login with Forum Account

Don't have an account? Register today!
Riiiiiiiiiiiiight
The 3D KITT looks like they've turned the brightness so far down there's no detail to miss.

In short; thanks Nvidia, I hate it.
well as a guy that can not draw perspective / 3D it might be interesting.
But yeah look like ray tracing,,,,,, it will probably be better in the 2.0 version next year.
Gentle Viking;4290318
well as a guy that can not draw perspective / 3D it might be interesting.
But yeah look like ray tracing,,,,,, it will probably be better in the 2.0 version next year.

Indeed, it's something that'll be better with more machine learning and refinement. Interesting and could be great for indie games makers.

If you can't tell, the origional comment was written when my meds had worn off and now I'm all drugged up again and happy.
Very suspect… you show it the front of a car and it ‘knows’ what the (out of sight) rear light cluster looks like?

This sets false and really dubious expectations.