We propose a neural scene flow prior that utilizes the architecture of neural networks (MLPs) as an implicit regularizer. Unlike learning-based scene flow methods, optimization occurs at runtime, and our approach needs no offline datasets -- making it ideal for deployment in new environments such as autonomous driving. Also, the implicit and continuous scene flow representation allows us to estimate dense long-term correspondences across a sequence of point clouds.

We revisit a recent innovation -- PointNetLK -- and show that the inclusion of an analytical Jacobian can exhibit remarkable generalization properties while reaping the inherent fidelity benefits of a learning framework. Our approach not only outperforms the state-of-the-art in mismatched conditions but also produces results competitive with current learning methods when operating on real-world test data close to the training set.

We propose a novel framework that produces a couple of new approaches that improve upon the state-of-the-art registration in some aspects. The existence of a suite of techniques that are dependent upon: (i) computation complexity, (ii) object specificity, (iii) noise, (iv) choice of loss functions and (v) pose parametrization. The choice of technique largely depends on the user’s requirement. All of these techniques can be unified into a single framework which can serve as a starting point to researchers in this field.

We provide a novel network to align point clouds that utilizes PointNet encoding. Our PCRNet uses data specific information to make accurate registration estimation, which is robust to noise and large initial misalignment between source point cloud and template point cloud. With single pass, PCRNet performs computationally faster than existing methods, while iterative version of PCRNet can refine the registration result to make highly accurate prediction. Our PCRNet can be applied further to 3D tracking, model replacement.

We propose a multi-domain feature learning-based visual place recognition method with 2 modules: feature extraction module, which uses CapsuleNet to extract features under different circumstances; feature separation module, which enforces a separation in environmental condition-invariant and condition-related domains. Combining 2 modules through sequential matching, we can find potential match.

In order to perform sequence matching between short-term testing frame and long-term reference frame in visual place recognition, we develop a multi-resolution sampling-based global search method with coarse-to-fine searching and particle filter, which balances the matching accuracy and searching efficiency. By iteratively updating both potential trajectories and frame sequences through low resolution to high resolution, we can find the best match under the highest resolution level.

Using k-nearest neighbors search and several morphological operations on images, we develop an efficient method to collect data from da Vinci surgical robot with user interface. We implement different deep learning modules to do medical tool segmentation on different augmented data collected in our lab. After comparing several methods, we find a way to do accurate and real-time medical tool segmentation for 15-20 frames per second.

We explore the performance of deep kernels on graphical models, which incorporates both the structural properties of deep learning architectures with non-parametric flexibility of kernels. In our project, we aim to solve the task of reverse-engineering of gene interaction. Inspired by KELLER, Pairwise MRF is used to model interactions between genes. Deep kernels, constructed from neural network and LSTM are used for learning parameters of the Pairwise MRF.

Instead of using the raw image to do Lucas-Kanade tracking, I utilize Difference of Gaussian Pyramid features to do efficient tracking of 2 images of Purcue. I estimate the intrinsic of the camera and perform 3D reconstruction of Purcue. With the combination of Lucas-Kanada algorithm and random affine warp to the point cloud, my Purcue is now playing with polar bear in the video.

We apply Double DQN, Dueling DQN, A2C, A3C and DDPG to the grid-world navigation problem with various reward reshaping techniques and different inputs. Though scent perception can yield good performance for Double DQN and Dueling DQN, adding vision perception helps refine the prediction. A2C, A3C and DDPG is not suitable for this discrete grid-world setting.