Portfolio
Brief descriptions of some of my projects. For complete list of publications go here.
Differentiable adaptive optics
Another publication on differentiable adaptive optics.
Paper in Optics Express
Github
Figure: Conceptual scheme of differerntiable adaptive optics.
Inspired by differentiable rendering and physics - here we discover the sample aberration by optimizing a forward model of image formation in the microscope. We match the model to the actual (aberrated) measurements from the actual microscope using gradient-based optimization within a machine learning framework (Tensorflow).
Machine learning for adaptive optics in microscopy
Adaptive optics (AO) – was my main focus at the Max Planck Institute of Neurobiology of Behavior and the topic of my PhD dissertation.
Imaging in fluorescent microscopy relies on focusing light inside the sample with the objective. However when encountering tissue in biological samples, light gets scattered and deflected off its path resulting in broader and dimmer (aberrated) focus, not suitable for imaging. Adaptive optics is a technique of compensating sample aberration such that sharp focus is restored allowing imaging despite the aberration of the sample.
The main problem, however is discovering the correction needed for a particular imaging site. This can be regarded as an inverse problem of finding an aberration from its resulting effect on the images. In my thesis I have explored two diefferent approaches to this problem.
Virtual reality set-up for experiments on fruit flies
VR has become a very powerful and versatile tool in neuroscience, since it allows to observe natural walking behavior of the animals in a fully controlled (simulated) environment while performing advanced imaging and brain activity monitoring techniques that require the subject to stay stationary.
The VR set-up is intended to be used with fruit flies Drosophila Melanogaster and consists of a tracking system (the fly is walking on an omnidirectional ball-treadmill) and VR display system. The specifications set for the VR system (500 Hz ball tracking, minimal latency and highest possible VR framerate) are motivated by insect experiment subject: the typical reaction times of a fly are in 1-2 ms range.
Machine learning approaches for light scattering control
Here we explore machine learning methods (including neural networks) to predict correcting light modulations to focus through strongly scattering media. Among others, generative models (conditional variational autoencoders) were explored to predict focusing patterns to send light through scattering samples.
Figure: example of phase modulations focusing light through aberrating media.
Object and people recognition in RoboCup@home
Participating in RoboCup (2016-2018) I was working on improving the object perception pipeline for the domestic service robot (Care-O-bot 3). Typical task was to program the robot to manipulate a set of household objects (not known prior to the competition).
For this task the pipeline we used was first to detect an object with a 3D camera (points outlying the surface, table, etc were found), and get a cropped RGB image of the detected object(s) only. Then the RGB image was classified with a CNN. To train the classifying CNN, we had to quickly collect a photo dataset of all the objects used in the competition, and train the CNN with dataset augmentation techniques.
Evolving a Neural Network Based Controller for Robot Arm
A study project where I explored NEAT (Neuroevolution of Augmenting Topologies) to evolve a controller for a robotic arm. The project was done with a Python implementation of NEAT algorithm interfaced with a scripted scenario in a robotics simulation environment V-REP.