We need 2 things for our app -
1. Round camera preview
2. Round video view
For both of them we'll need a similar solution -
Taking the video/camera preview frame by frame and perform an open GL processing on it (cropping corners to make it a circle) and display the result on a SurfaceView/GLSurfaceView.
Doing that requires both knowing your way around GLES (OpenGL ES) and knowing how to handle camera preview frames and drawing them on a surface - this task requires multithreading.
Please see attached image for example how the video frames are to be displayed on the activity.