Virtual reality set-up for experiments on fruit flies
VR has become a very powerful and versatile tool in neuroscience, since it allows to observe natural walking behavior of the animals in a fully controlled (simulated) environment while performing advanced imaging and brain activity monitoring techniques that require the subject to stay stationary.
The VR set-up is intended to be used with fruit flies Drosophila Melanogaster and consists of a tracking system (the fly is walking on an omnidirectional ball-treadmill) and VR display system. The specifications set for the VR system (500 Hz ball tracking, minimal latency and highest possible VR framerate) are motivated by insect experiment subject: the typical reaction times of a fly are in 1-2 ms range.
Ball tracking with camera was implemented in C++ using dense optical flow from OpenCV library; VR environment was created using a free open-source game engine Urho3D and displayed with projection screens.
Overall system latency was measured as 30±8 ms, with 500 Hz ball tracking frequency and 120 fps VR display. This seems to be the limit of what one can achieve with stock hardware without going into custom hardware and real-time software development which would be substantially more demanding and expensive.