CN103679201B - Calibration method of point set matching for image matching, recognition and retrieval - Google Patents

Abstract

The invention belongs to the technical field of pattern recognition, image processing and computer vision, and particularly relates to a calibration method of point set matching for image matching, recognition and retrieval. According to the calibration method of the point set matching for the image matching, recognition and retrieval, an adjacent matrix is established according to an initial matching relation of two point sets, and an approximate solving method of a maximum clique problem in a graph theory is proposed to obtain a point set matching relation approximately complying with identical geometric transformation. The similarity degree between the two point sets can be obtained by projecting one point set to the space where the other point set is located through the geometric transformation, and the similarity degree between the two point sets is used as an image similarity degree to achieve the image matching, recognition and retrieval. Experiments show that the calibration method of the point set matching can be combined with any of multiple shape feature extraction methods to obtain good image matching and recognition effects.

Description

A kind of for images match, the bearing calibration of some sets match that identifies, retrieve

Technical field

The invention belongs to pattern recognition, image procossing, technical field of computer vision, be specifically related to a kind of some sets match Bearing calibration, may be used for images match, identify, retrieve.

Background technology

Image includes shape, texture, colouring information, and wherein, shape is image recognition, retrieves the main information relied on, Shape facility describes for image recognition, retrieves extremely important, and similarity and the diversity between shape is often reflected in figure In the characteristic point of picture, therefore a kind of primary solutions of images match is to solve as a sets match. 【S.Belogie,J.Malik,J.Puzicha:“Shape matching and object recognition using shape contexts,IEEE Transactions on Pattern Analysis and Machine Intelligence ", Volume 24, pp.509-52,2002] paper proposes a kind of it is referred to as Shape context (Shape Contexts) shape description method, uses the Bipartite Matching in graph theory on the basis of shape descriptor Similarity Measure A set is mated by method, but its computation complexity is higher, its solve Bipartite Matching algorithm complexity about For O (N⁴)。【David G.Lowe:″Distinctive Image Features from Scale-Invariant Keypoints″,International Journal of Computer Vision,Volume 60,Issue 2,pp.91- 110,2004] Feature Points Extraction of a kind of DoG and a kind of description being referred to as SIFT are proposed in, and based on describing son Between similarity characteristic point is mated, but such set matching result without correction containing error hiding, Can to follow-up images match, identify, retrieve and bring interference.

Summary of the invention

It is an object of the invention to propose a kind of computing cost rationally and can obtain with the collocation of variously-shaped description method Preferable images match, identify, retrieve the bearing calibration of the some sets match of performance.

A kind of bearing calibration for a sets match that the present invention proposes, concrete calculation procedure is as follows:

(1) adjacency matrix is calculated；

(2) maximal clique problem in approximate solution graph theory；

The step that adjacency matrix in step 1 in the bearing calibration of recited above some sets match calculates is as follows:

A () assumes two some set P={P₁,P₂,…,P_mAnd Q={Q₁,Q₂,…,Q_mMatching relationship initial between } is Will abut against matrix initialisation is C={c_ij=0|i,j=1,2,…,m}；B () calculates R ={r_ij=d(P_i,P_j)/d(Q_i,Q_j)|i,j=1,2,…,m;I ≠ j}, here d (P_i,P_j) represent some P_iWith a P_jBetween Europe several In must be apart from, d (Q_i,Q_j) represent some Q_iWith a Q_jBetween Euclidean distance；

C () is by { r_ij=d(P_i,P_j)/d(Q_i,Q_j)|i,j=1,2,…,m;I ≠ j} sorts according to order from small to large, To R₁≤R₂≤…≤R_m(m-1)；By R_kLocation records in R be S [k] ∈ (i, j) | i, j=1,2 ..., m;I ≠ j}, k=1, 2,…,m(m-1)；

D () is by R₁≤R₂≤…≤R_m(m-1)Segmentation, segmentation method is as follows: if there is n-1 section, the subscript that its border is corresponding For I [1]=1, I [2], I [3] ..., I [n-1], I [n]=m (m-1) and meet condition R_I[i]/R_I[i+1]>t∧R_I[i]/R_I[i+1]+1≤ T, here i=1,2 ..., n-1, t be one close to 1 threshold value, then with I [1]=1, I [2], I [3] ..., I [n-1], I [n] are limit Boundary carries out segmentation；

(e) from above-mentioned steps (d) obtain to R₁≤R₂≤…≤R_m(m-1)N-1 segmentation in find the longest one section, Its corresponding order in n-1 segmentation is $i^{*}=\arg \underset{i}{\max }\left\{I\right[i+1]-I[i\left]\right|i=\mathrm{1,2},...,n-1\},$ Extract this section in R Corresponding subscript S [k] | k=I [i^*],I[i^*]+1,…,I[i^*+1]}；

F () makes { c_S[k]=1|k=I[i^*],I[i^*]+1,…,I[i^*+1]}；

Maximal clique problem approximate solution in the graph theory in step 2 in the bearing calibration of recited above some sets match Calculation procedure is as follows:

(a) the lower target set of all nodes of a complete graph is initialized as Θ=1,2 ..., m}；By noise node Corresponding lower target set is initialized as null set Ψ=Φ, the set of the residue node outside noise node is initialized as Ω= Θ；

B () arranges enumeratorHow many nodes are had to be connected with node i for record；

(c) for i=1,2 ..., m: if v_i=0, node i is added the set of noise node, i.e. Ψ ∪ { i} → Ψ；With Time by node from residue node set omega delete, i.e. Ω-{ i} → Ω；

If (d) v_i=v_jArbitrary i ≠ j is set up, here i ∈ Ω and j ∈ Ω, then willΩ institute it is designated as under in The Corresponding matching point comprising element is to output and exits；Otherwise forward step (e) to；

E () finds the node in Ω with Smallest connection limit numberMake Ψ ∪ { i ' } → Ψ and Ω-{i′}→Ω；Find the node being connected with node i ', i.e. Γ=j | c_i′j=1;j=1,2,...,m}；For j ∈ Γ, make v_j- 1→v_j；Forward step (d) to.

Accompanying drawing explanation

Fig. 1 is the composition frame chart of image identification system.

Detailed description of the invention

One image identification system is generally made up of following link, image acquisition, pretreatment, feature extraction, similar Degree calculates, classification, and the target of image identification system is to return the image most like with input picture from image data base, whole The composition of image identification system is shown in Fig. 1.Here, image acquisition can complete physics by camera, scanner etc. are various The sensing equipment of picture completes, and pretreatment uses a kind of bianry image skeletal point extracting method that inventor proposes, and feature extraction is adopted A kind of shape descriptor proposed with inventor.

Embodiment 1:

Step 1: a width input picture is extracted characteristic point, and calculates the shape descriptor of each characteristic point, make P={P₁, P₂,…,P_KAnd { f (F (P_k)) | k=1,2 ..., K} represents the shape descriptor of obtained characteristic point and correspondence thereof, shape respectively The calculation procedure that shape describes son is as follows:

(a) optional characteristic point P_k∈ P is as a reference point, adds up the spatial distribution of further feature point, obtains One corresponding rectangular histogram, is denoted as h (P_k)；Here, histogrammic circular is as follows: with reference point P_kCentered by, will figure As the space at minimum circumscribed circle place is divided into the grid of M × Ν, the number calculating the characteristic point falling into each interval of grid obtains To rectangular histogram, M and N is natural number；As a reference point with each characteristic point respectively, the most corresponding each characteristic point respectively obtains one Individual rectangular histogram, there are K rectangular histogram { h (P_k)|k=1,2,…,K}；

B the rectangular histogram of each Feature point correspondence is sought Fourier transformation by (), if h is (P_k) Fourier transformation be F (P_k), right Matrix F (P_kEach element in) carries out the mathematic(al) manipulation that function f (.) defines, F (P_k) mathematic(al) manipulation f (.) be defined as matrix F (P_kThe W power of the modulus value of each element in), if F_ij(P_k) representing matrix F (P_k) the i-th row, jth row element, then f (F_ij (P_k))=|F_ij(P_k)|^W, W=2；By { f (F (P_k)) | k=1,2 ..., K} is as the shape descriptor of K characteristic point correspondence respectively；；

Step 2: piece image optional in image library is extracted characteristic point, based on the shape descriptor meter described in step 1 Calculation method calculates the shape descriptor of each characteristic point, makes Q={Q₁,Q₂,…,Q_LAnd { f (F (Q_l)) | l=1,2 ..., L} is respectively Characteristic point obtained by expression and the shape descriptor of correspondence thereof；

Step 3: the similarity between calculating input image and each shape descriptor of image library image, is denoted as { d_kl=d(f (F(P_k)),f(F(Q_l)))|k=1,2,…,K;L=1,2 ..., L}, employing inner product is as measuring similarity here, the most respectively by square Battle array f (F (P_k)) and f (F (Q_l)) stretch as vector, then seek two vectorial inner products；

Step 4: according to nearest neighbouring rule to P={P₁,P₂,…,P_KAnd Q={Q₁,Q₂,…,Q_LMate, computational methods As follows: to carry out m=min{K, L} iteration, the point that each iteration obtains a coupling is right, and the concrete calculation procedure of each iteration is such as Under: (a) finds set D={d_kl|k=1,2,…,K;L=1,2 ..., the greatest member d in L}_st；B () willAs one Coupling is to recorded setIn；C () makes d_sl=-∞ and d_kt=-∞: k=1,2 ..., K and l=1,2 ..., L；

Step 5: use the initial some set that step 4 is obtained by the bearing calibration of the some sets match of present invention proposition Join relationIt is corrected, obtains corrected some set matching relationshipn≤m；Order Represent matching double points Coordinate figure；

Step 6: calculate projective transformation, step based on the matching relationship between the corrected some set that step 5 obtains As follows:

A () makes $A=\left[\begin{array}{cccc}{x}_1& y_1& 1& 0\\ y_1& -x_1& 0& 1\\ x_2& y_2& 1& 0\\ y_2& -x_2& 0& 1\\ \·\·\·& \·\·\·& \·\·\·& \·\·\·\end{array}\right];$ Order $\mathrm{\β}=\left[\begin{array}{c}s.\cos \mathrm{\α}\\ s.\sin \mathrm{\α}\\ {\mathrm{\δ}}_x\\ {\mathrm{\δ}}_y\end{array}\right];$ Order $b=\left[\begin{array}{c}{X}_1\\ Y_1\\ X_2\\ Y_2\\ \·\·\·\end{array}\right];$ Here, 4 parameters in β are fixed Justice is: s is scaling ratio, and α is the anglec of rotation, δ_xAnd δ_yIt is the translational movement of X and Y-axis respectively；

B () seeks the least square solution of A β=b, obtain β=(A^TA)^-1A^Tb；

Step 7: between two width images, similarity is defined as P={P₁,P₂,…,P_KAnd Q={Q₁,Q₂,…,Q_LPhase between } Like degree, calculation procedure is as follows:

A () utilizes formula $\left[\begin{array}{c}X\\ Y\end{array}\right]=\left[\begin{array}{ccc}s.\cos \mathrm{\α}& s.\sin \mathrm{\α}& {\mathrm{\δ}}_x\\ -s.\sin \mathrm{\α}& s.\cos \mathrm{\α}& {\mathrm{\δ}}_y\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]$ To { Q₁,Q₂,…,Q_LConvert, if Q_jWarp Cross the coordinate figure that obtains of conversion and be designated as T (Q_j), j=1,2 ..., L；

(b) some set P={P₁,P₂,…,P_KAnd Q={Q₁,Q₂,…,Q_LSimilarity between } is defined as

$S(P,Q)=\min \{\frac{1}{K}\underset{i=1}{\overset{K}{\mathrm{\Σ}}}E\left(\underset{j}{\min }\right\{d(P_i,T\left(Q_j\right))\left\}\right),\frac{1}{L}\underset{j=1}{\overset{L}{\mathrm{\Σ}}}E\left(\underset{i}{\min }\right\{d(P_i,T\left(Q_j\right))\left\}\right)\}$

In above formula: function $E\left(d\right)=\left\{\begin{array}{cc}1& d\≤t^{\′}\\ 0& \mathrm{else}\end{array}\right.,$ T ' is threshold value, here t '=30 set in advance；d(P_i,T(Q_j)) Represent P_iWith T (Q_j) Euclidean distance between 2, i=1,2 ..., K, j=1,2 ..., L；

Step 8: set the image having S width to prestore in image library, repeated execution of steps 2 to step 7, calculate each image respectively With the similarity of input picture, according to arest neighbors principle of classification, arrange according in similarity Sequential output image library from big to small Image in front T position is as identifying or the result of retrieval, T≤S.

Step 1 and step 2 in embodiment 1 have employed a kind of characteristics of image point extracting method realization figure that inventor proposes As pretreatment, extracting the skeletal point of bianry image here as characteristic point, circular is as follows:

Step 1: use the gaussian kernel wave filter of 7 × 7 that image is smoothed, Gauss kernel parameter selection σ=2 here；

Step 2: the binarization method using inventor to propose turns to 1 and 0,1 and 0 point to by each pixel two-value of image Biao Shi foreground point and background dot；

Step 3: bianry image carries out endpoint detections, and marginal point is defined as the pixel value not phase of two consecutive points here Deng point；

Step 4: for each marginal point, finding the pixel value comprising this marginal point is 1 vertical and horizontal straight line continuously Section, the midpoint making shorter in vertical and horizontal straightway one is skeletal point；

Step 5: other skeletal point comprised in deleting the certain radius of each skeletal point, radius value takes 1 here, if one Individual skeletal point has been deleted, and will no longer scan it in subsequent step.

The concrete calculation procedure of the binarization method described in the step 2 of characteristics of image point extracting method recited above is such as Under:

Step 21: assume that image has n pixel, is ranked up obtaining c by the gray value of all for image pixels₁≤c₂ ≤,…,≤c_n；

Step 22: make x_i=i and y_i=c_i, i=1 here, 2 ..., n；Make t=c₁；

Step 23: for i=1,2 ..., n: calculate point (x_i,y_i) to distance d of certain straight line_i, straight line referred herein by Point (x₁,y₁) and point (x_I,y_I) determine；

Step 24: make j=1；Make I=min{i | c_i>0}；Make t_j=c_I；Make r_j=(n-I)/n；

Step 25: for j=2,3: ordert_j=c_I, r_j=(n-I)/n；

Step 26: for j=3,2,1: if r_jMore than certain threshold value r set in advance, take r=2% here, then make t=t_j；

Step 27: for i=1,2 ..., n: if c_i>=t, makes b_i=1；Otherwise, b is made_i=0；Here b₁,b₂,…,b_nRepresent The pixel value of each point after image binaryzation.

The method described based on embodiment 1 devises Symbol recognition program, and to GREC2003 image library (http:// www.iapr-tc10.org) tested, test 6900 width images altogether, it addition, with other two kinds of feature extracting method generations Having carried out contrast experiment for the feature extracting method implementing 1 employing, respectively [poplar is long-standing: Yi Zhongtong for both feature extracting methods The character description method for Symbol recognition, patent of invention, authorize the time: on February 6th, 2008, authorize country origin: China, Grant number: 200410016733.0] and [S.Belogie, J.Malik, J.Puzicha: " Shape matching and object recognition using shape contexts,IEEE Transactions on Pattern Analysis and Machine Intelligence”,Volume 24,pp.509-52,2002】.In experimental result, PLC and SC generation respectively Table above two feature extracting method, SSC represents the feature extracting method that embodiment 1 uses, and experimental result is as follows:

Table 1: discrimination (%) (50 kinds of models of ideal image;Rank 1:5 kind symbol, 5 width images;Rank 2:20 kind accords with Number, 20 width images;Rank 3:50 kind symbol, 50 kinds of images)

	PLC	SSC	SC
				Rank 1	100	100	100
Rank 2	100	100	100
				Rank 3	100	100	100

Table 2: rotate and discrimination (%) (50 kinds of models of the image that stretches;Rank 1:5 kind symbol, 25 width images;Rank 2: 20 kinds of symbols, 100 width images;Rank 3:50 kind symbol, 250 width images)

Table 3: discrimination (%) (50 kinds of models of deformation pattern;Rank 1:5 kind symbol, 25 width images;Rank 2:15 kind accords with Number, 75 width images)

Table 4: noise jamming hypograph discrimination (%) (rank 1:5 kind model, 5 kinds of symbols, 25 width images;Rank 2:20 kind Model, 20 kinds of symbols, 100 width images;Rank 3:50 kind model, 50 kinds of symbols, 250 width images)

Table 5: the image recognition rate (%) (15 kinds of models, 15 kinds of symbols, 75 width images) when noise and deformation occur simultaneously

Claims (1)

1. one kind for images match, the bearing calibration of some sets match that identifies, retrieve, it is characterised in that comprise adjacency matrix Calculate and two parts of maximal clique problem approximate solution in graph theory；Wherein:

The step that described adjacency matrix calculates is as follows:

A () assumes two some set P={P₁,P₂,…,P_mAnd Q={Q₁,Q₂,…,Q_mMatching relationship initial between } is Will abut against matrix initialisation is C={c_ij=0 | i, j=1,2 ..., m}；

B () calculates F={r_ij=d (P_i,P_j)/d(Q_i,Q_j) | i, j=1,2 ..., m；i≠j}；Here d (P_i,P_j) represent some P_iWith Point P_jBetween Euclidean distance, d (Q_i,Q_j) represent some Q_iWith a Q_jBetween Euclidean distance；

C () is by { r_ij=d (P_i,P_j)/d(Q_i,Q_j) | i, j=1,2 ..., m；I ≠ j} sorts according to order from small to large and obtains R₁ ≤R₂≤…≤R_m(m-1)；By R_kLocation records in F be S [k] ∈ (i, j) | i, j=1,2 ..., m；I ≠ j}, k=1, 2,…,m(m-1)；Here, R_k=r_ijTime, k is R_kSubscript, (i j) is r_ijSubscript, it is right that S [k] have recorded between subscript Should be related to, so available { the R of record₁,R₂,…,R_m(m-1)And { r_ij| i, j=1,2 ..., m；Between the member of i ≠ j} one One corresponding relation；

D () is by R₁≤R₂≤…≤R_m(m-1)Segmentation, segmentation method is as follows: if there is n-1 section, is designated as I under its border is corresponding [1]=1, I [2], I [3] ..., I [n-1], I [n]=m (m-1) and meet condition R_I[i]/R_I[i+1]>t∧R_I[i]/R_I[i+1]+1≤ T, here i=1,2 ..., n-1, t be one close to 1 threshold value, then with I [1]=1, I [2], I [3] ..., I [n-1], I [n] are Border carries out segmentation；

(e) from above-mentioned steps obtain to R₁≤R₂≤…≤R_m(m-1)N-1 segmentation in find the longest one section, it is at n-1 Corresponding order in individual segmentation isExtract this section corresponding subscript in F S [k] | k=I [i^*],I[i^*]+1,…,I[i^*+1]}；

F () makes the value of adjacency matrix in step (a) be { c_S[k]=1 | k=I [i^*],I[i^*]+1,…,I[i^*+1]}；Here, S Step (c) is shown in the definition of [k]；

In described graph theory, the calculation procedure of maximal clique problem approximate solution is as follows:

(A) the lower target set of all nodes of a complete graph is initialized as Θ=1,2 ..., m}；By noise node pair The lower target set answered is initialized as null set Ψ=Φ, by outside noise node residue node set be initialized as Ω= Θ；

(B) enumerator is setHow many nodes are had to be connected with node i for record；

(C) for i=1,2 ..., m: if v_i=0, node i is added the set of noise node, i.e. Ψ ∪ { i} → Ψ；Simultaneously Node is deleted from the set omega of residue node, i.e. Ω-{ i} → Ω；

(D) if v_i=v_jArbitrary i ≠ j is set up, here i ∈ Ω and j ∈ Ω, then willIt is designated as Ω under in be comprised The Corresponding matching point of element is to output and exits；Otherwise forward step (E) to；

(E) node with Smallest connection limit number is found in ΩMake Ψ ∪ { i ' } → Ψ and Ω- {i′}→Ω；Find the node being connected with node i ', i.e. Γ=j | c_i′j=1；J=1,2 ..., m}；For j ∈ Γ, make v_j- 1→v_j；Forward step (D) to.