Facebook Reality Labs demos high-fidelity hand tracking

by Mark Tyson on 15 December 2020, 13:11

Tags: Facebook

Quick Link: HEXUS.net/qaepwp

Add to My Vault: x

Earlier this month Facebook Reality Labs (FRL) published a paper, with an accompanying video, about the real-time tracking of human hands. Hand tracking for VR is expected to make a major difference to the quality of immersion when it becomes suitably adept. However, at this time users are better of using the specially designed controllers rather than suffering with hand tracking hardware that is limited in its ability which thus lessens the VR experience.

With Facebook's interest in VR though Oculus and so on it isn't surprising to see it working on better VR environment to human interaction. The efforts detailed today focus on the solving of a big problem with previous hand tracing tech – when your hands overlap, touch, or fingers are self-occluded. The FRL team has leve raged recent advances in vision-based tracking and physically based animation during its research and development. Now FRL claims that it has developed "the first algorithm capable of tracking high-fidelity hand deformations through highly self-contacting and self-occluding hand gestures, for both single hands and two hands".

Common hand interactions introduce many self-interactions and massive self occlusions, says the research team. However, it managed to create a robust algorithm by "constraining a vision-based tracking algorithm with a physically based deformable model". The moel has been tested against some of the most complicated interaction of human hands, and you can see the tracking quality and accuracy in the embedded video.

Click to zoom image

So, can we expect an Oculus branded camera setup to deliver this tech to VR-adventurers in the near future? From checking through the research paper it doesn't look like this level of hand tracking tech will be available for consumers very soon. The dedicated hand tracking hardware setup sounds rather beefy to be reduced to the consumer space anytime soon as, for accurate 30fps tracking, it needed; 124 cameras, six threads on a 2.2GHz Intel E5-2698 Xeon processor and an Nvidia Tesla V100 GPU. I note that various camera arrays were tested, from two upwards, and a 43 camera setup also fared rather well.



HEXUS Forums :: 3 Comments

Login with Forum Account

Don't have an account? Register today!
43 cameras to do “I got your nose”. That shouldn't seem awesome, yet somehow it is :D
Hand tracking is where it needs to be at for VR I think, controllers are ok but limited, can see a lot of scope for hand tracking especially with things like touching buttons etc
six threads 2.2GHz Intel E5-2698 Xeon processor and an Nvidia Tesla V100 GPU… all things considered the requirements isn't actually all that high (excluding cameras). You could likely get similar performance (assuming using cuda) with a 3090 and a 6-8 core ryzen.