Eye tracking and control is an area of tech in which I expect to see growth in the coming years. Eye tracking specialist Tobii has been working diligently on the hardware side for several years and it has partnered with both game makers and device makers to promote the tech. More recently Nvidia has been working on implementing variable rate shading (VRS) tied to eye position, and Microsoft has been refining its eye control settings in Windows 10.
It seems like Nvidia wants to get in on the hardware side of things to support its VRS implementations on screens and in HMDs. Researchers working for the green team have recently published a paper with gaze-sensing HMD prototype designs and test results. Importantly, as well as designing a new low-power, accurate, low latency, and light weight solution, the scientists say the prototype hardware is cheap to make using commonly available components.
Nvidia's gaze-sensing LEDs for head mounted displays paper (PDF) was published a few days ago and is very detailed in its proposition - but in essence the researchers modified existing HMDs by exploiting the light emitting and sensing properties of LEDs.
The prototype uses a simpler process than current solutions for eye position tracking, which benefits the design in many ways but at the moment it sounds quite inelegant as it uses various off the shelf bits and bobs including a mobile device, two rings of LEDs, and two microcontrollers.
Above you can see the HMD prototype created by the Nvidia researchers. Rather than using expensive and power hungry gyroscopes, accelerometers, various optical sensors, and so on - it uses LEDs as the core of the design. The bi-directional characteristics of LEDs allows for light capture and illumination, and the ring of LEDs placed in front of the eyes facilitate gaze estimation. The estimation is done speedily by a "supervised adaptive pattern recognition" algorithm which runs on the aforementioned attached mobile device.
While it all might sound complex to non-scientists and non-engineers, the Nvidia researchers assert that their design is "the simplest gaze-detecting design to date". Nvidia's small low cost solution is said to be ideal for HMDs. Nevertheless, there is still work to do on wrinkles in the system. The biggest issues are currently regarding user blinks, and the tuning of tracking accuracy and on its relationship with latency.
