ILLUSTRATION OF THE DIFFERENT STEPS OF THE METHOD
AND CONTRIBUTION OF EACH COMPONENT

1. FIRST FRAME ANNOTATION

This video illustrates how the annotation of the first frame is performed, using strokes and clicks. Different colors represent different objects.

2. MULTIPLE OBJECTS SEGMENTATION

2.1 MULTI-OSVOS NETWORK
Illustration of the data augmentation step: red, green and blue labels denote three different objects. The yellow label shows the layer corresponding to the additional border.
Contribution of the smart dilation step
'CLASSICAL DILATION'

2.2 REFINEMENT NETWORK

2.3 KEYFRAME SELECTION
Extract keyframes and masks in these frames for each object.
2.4 MASK PROPAGATION
Backward, forward mask propagation from keyframes to the remaining frames.
FORWARD PROPAGATION-OBJECT 1
If the user is not satisfied with the result, she/he can correct it in one frame and let the algorithm propagate the correction to the other frames.
2.5 MASK LINKING
We test 2 methods:
  1. Pair-wise mask linking (link between masks in two consecutive frames)
  2. 'Global mask linking (GMMCP)'
PAIR-WISE MASK LINKING
2.6. FINAL RESULT

3.OBJECTS REMOVAL

3.1 THE SEGMENTATION MASKS CAN IMPROVE THE VIDEO INPAINTING
3.1.1 BETTER STABILIZATION
WITHOUT OBJECT EXTRACTION
3.1.2 THE SEGMENTATION MASK CAN SEPARATE FOREGROUND/BACKGROUND AND GIVE BETTER INPAINTING RESULTS
Results with 2 differents approaches:
  1. No segmentation mask used
  2. Segmentation masks are used to separate background/foreground.
,'FOUNTAIN_OPERA- METHOD 1'
3.2 OPTICAL FLOW-BASED VIDEO RECONSTRUCTION
3.2.1 FORWARD/BACKWARD PIXEL PROPAGATION
a. Forward propagation b. Backward propagation
3.2.2 POISSON BLENDING
Synthesized video
a. Without Poisson Blending b. With Poisson Blending Real video
a. Without Poisson Blending b. With Poisson Blending (2D only) c. With Adaptive Poisson Blending
3.3 PATCH-BASED VIDEO RECONSTRUCTION