Poisson image editing is an effective approach for seamless image composition. By solving Poisson equations using the user-specifed boundary condition, Poisson image editing seamlessly blends the colors from both images without visible discontinuities around the boundary. Poisson image editing not only has an elegant mathematical formulation, but also appears to be easy to use.
Poisson image editing may generate unnatural blurring artifacts at places where the boundary intersects with salient structures in the target image To obtain seamless composition, the boundary selected should avoid as much as possible salient structures in both the images.
The authors of Drag-and-Drop Pasting, to make Poisson image editing more practical and easy to use, propose a new objective function to compute an optimized boundary condition. A shortest closed-path algorithm is designed to search for the location of the boundary.
In this project an attempt is made to compute the optimized boundary using GVF Snakes. The results are compred to the results obtained by using the method proposed in Drag-and-Drop Pasting.
Active contour model, also called snakes, is a framework for delineating an object outline from a 2D image. This framework attempts to minimize an energy associated to the current contour as a sum of an internal and external energy
The external energy is supposed to be minimal when the snake is at the object boundary position. The most straightforward approach consists in giving low values when the regularized gradient around the contour position reaches its peak value.
The internal energy is supposed to be minimal when the snake has a shape which is supposed to be relevant considering the shape of the sought object. The most straightforward approach grants high energy to elongated contours (elastic force) and to bended/high curvature contours (rigid force), considering the shape should be as regular and smooth as possible.
E(snake)= ∫ Einternal(v(s)) + Eexternal(v(s)) ds
Einternal = α * (dv/ds) + β * (d^2(v)/d(s^2))
Eexternal = wl*Eline + we*Eedge + wt*Eterm
1. Compute the absoulute difference (L2 difference incase of colour images) of source image patch and destination image region. This is to retain the salient features of both the source and destination image patches.
2. Initialize snake at the boundary specified by the user. The weights for the internal and external energy functions are set appropriatley so that the snake can easily deform and align with the edges/boundaries of the salient features.
3. Iteratively deform the snake to minimize its energy. The final snake position should give us the optimal boundary for Possion image editing.
The boundary obtained does not look right!
Initialization is a problem because the capture range of the traditional potential force (here, Edge force) is generally small. If the snake is not initialized close to the object of interest then it does not converge at the object boundary.
Convergence to concave regions is a problem in traditional snakes, because the contour is often left split across boundary concavities
The overall approach is to define a new non-irrotational external force field, which we call the gradient vector flow (GVF) field. GVF field will replace the potential force field in tradition snake external energy function, defining a new snake, which we call the GVF snake. The GVF field points toward the object boundary when very near to the boundary, but varies smoothly over homogeneous image regions, extending to the image border. The main advantages of the GVF field are that it can capture a snake from a long range—from either side of the object boundary—and can force it into concave regions.
The code for GVF snakes used in the project was written by Dejan Tomazevic of Jhons Hopkins Uiversity.
The boundary looks better!
Result obtained when run on GVF snake demo tool
The red boundaries show the source and destination regions
The red, blue and green curves are user defined boundary, optimal boundary computed using Dijkstras and Optimal boundary computed by GVF snakes respectievely.
Drag-and-Drop Pasting result: The region marked in orange shows blurring of the destination image.
GVF snake result: The region marked in orange shows no blurring of the destination image. The region marked in red shows bleeding of the destination image color into the source patch, this is caused as the snake ended up at the inner boundary of the object(moon)
The red boundaries show the source and destination regions
The red, blue and green curves are user defined boundary, optimal boundary computed using Dijkstras and Optimal boundary computed by GVF snakes respectievely.
Drag-and-Drop Pasting result: The region marked in orange shows blurring of the destination image.
GVF snake result: The region marked in orange shows no blurring of the destination image. The region marked in red shows bleeding of the destination image color into the source patch, this is caused as the snake ended up at the inner boundary of the object(moon)
From the results obtained it can be seen that we can obtain better boundary condition by using GVF snakes, but the downside of the approach is that it is practically not easy to come up with the right combination of weights for the GVF snakes to get the optimal boundary.
1. Gradient Vector Flow: A New External Force for Snakes by Chenyang Xu and Jerry L. Prince
2. Drag-and-Drop Pasting by Jiaya Jia, Jian Sun, Chi-Keung Tang and Heung-Yeung Shum
1. Matlab Code
2. Presentation