Optical Flow Estimation & Action Recognition

Occlusion-aware optical flow estimation and action recognition enhanced by unsupervised optical flow

Optical flow exploits the motion information of videos, and can improve the performance of action recognition.

Occlusion-aware Optical Flow Estimation

Occlusion is always a problem for optical flow estimation, flow in the occluded area is difficult to predict. Our target is to help the network to distinguish the occluded area from non-occluded area, and increase the accuracy of estimation.

The Occlusion is Predicted Correctly.

Our method can run faster than 30 FPS, while the End Point Error (EPE) is very competitive, which is 2.55/3.65 on train/test of Sintel Clean, 3.79/5.01 on train/test on Sintel Final and 4.11/1.4 on train/test of KITTI 2012.

Action Recognition Enhanced by Unsupervised Optical Flow

Learning reliable motion representation has a great promotion to video understaending. While most existng methods have to base on TV-L1 optical flow, whose pre-processing is time-consuming and storage-consuming. My target is to design a unsupervised optical flow module for action recognition.

Unsupervised Optical Flow Estimation Module is Embedded.

Compared with the baseline, I improve the accuracy of action recognition by 2.1% on UCF-101, 4.5% on HMDB-51.