CN102184186A - Multi-feature adaptive fusion-based image retrieval method - Google Patents

Abstract

The invention discloses a multi-feature adaptive fusion-based image retrieval method. The method comprises the following steps of: selecting image features, namely extracting more than two image features of each image in an image library; establishing a semantic tree of each image feature of each image in the image library; performing normalization processing on all image features of the images which is input by a user and is to be retrieved; performing fusion on the scores of the retrieval results of all image features; in screened images, respectively weighting and adding the scores of the retrieval results of all image features of each image to obtain final scores of the retrieval results; and outputting the images according to an order of the final scores of the retrieval results from high to low, wherein the score weights of the retrieval results of each image feature are not fixed. Due to the adoption of the multi-feature fusion method, the retrieval accuracy is greatly improved; and due to the adoption of a dynamic weight, a traditional fixed weight is changed, and the effect of image retrieval is further improved.

Description

Image search method based on the fusion of many features self-adaptation

Technical field

The present invention relates to a kind of image search method, specifically is a kind of image search method that merges based on many features self-adaptation.

Background technology

In the existing image search method that adopts, the CBIR technology is by extracting user's interest feature in the image, comprise some visual signatures such as color, shape, texture, image with wide input is retrieved in big image set, realized the retrieval of real image vision content characteristic.This retrieval mode is the important breakthrough to " looking for figure with key word ".Along with the development of CBIR technology, a lot of image indexing systems have been arranged.Though the field or the function of image indexing system are varied, basic search method all comprises following step: extract characteristics of image and write corresponding image library; The image of user's input is extracted feature and compares the calculating similarity with the feature in the image library; The similarity backward is returned to the user; Wherein, Feature Extraction mainly comprises color characteristic, shape facility and textural characteristics.Algorithms most in use during color characteristic extracts has color histogram, color moment method; Algorithms most in use in the shape facility comprises classic algorithm such as chain code representation, border square, Fourier descriptors; The textural characteristics algorithm mainly contains Tamura textural characteristics (roughness, direction degree, contrast), gray level co-occurrence matrixes, 0abor filtering algorithm etc.

For the image retrieval of big quantity, picture material widely different, a kind of feature is difficult to that all images type is all had good description.Recent years some outstanding based on point sparse features (SURF) and put forward in succession based on the dense feature (HOG) in zone, the test findings proof has effect preferably for some images, but these features also are a certain class picture material to be had characterize effect preferably, sparse features (SURF) may only be extracted seldom unique point to the fairly simple image of some textures, ensuing retrieving portion can not have good effect, and dense feature (HOG) generally is based on the zone, powerless again to the details of image coupling, and problem such as image rotation can not well solve.

In recent years, the someone has proposed for big data quantity, multi-class classification, and it is necessary adopting various features and different Feature Fusion methods.Also do a lot of work aspect Feature Fusion, but these work are based on the linear weighted function of feature mostly, for all retrieving images, the weights of each feature are fixing, can not analyze the retrieval effectiveness of each feature automatically, thereby carry out dynamic fusion.

Summary of the invention

The purpose of this invention is to provide the high image search method of a kind of retrieval precision based on the fusion of many features self-adaptation.

For achieving the above object, the technical solution adopted in the present invention is:

Image search method based on many features self-adaptation merges includes following steps:

Choosing of A, characteristics of image: the plural characteristics of image that extracts each image in the image library;

B, the picture in the image library is extracted selected feature, utilize layering kmeans algorithm to construct the semantic tree of each feature correspondence then;

C, the retrieving images that the user is imported extract feature, in each semantic tree, retrieve then, and returning result for retrieval, result for retrieval is that the image in the database is high more to the similar more score of retrieving images according to each image in the database of being arranged on earth by height with the similarity of retrieving images;

D, according to the score distribution rule of each semantic tree result for retrieval, the quality of the corresponding semantic tree result for retrieval of this feature of automatic Evaluation, thus determine the weights of this semantic tree;

E, each semantic tree return the score of image and carry out normalized, make that the mark after the normalization can linear, additive;

F, the mark of the result for retrieval of all characteristics of image is merged: the final PTS of each image be multiply by the weights of this semantic tree in the database by the score of this image in each semantic tree, addition obtains then, then image is exported with the descending order of the mark of final result for retrieval; Wherein the weights of each semantic tree correspondence are that step e is dynamically determined.

In the described steps A, the characteristics of image that is extracted has two, be based on respectively a little sparse features (SURF) and based on the dense feature (HOG) in zone.

In the described step e, the normalized computing formula of described sparse features (SURF) is:

SURFNormScore _iBe the characteristics of image mark after the normalization, SURFScoreMax, SURFScoreMin are respectively the maximal value and the minimum value of N picture mid-score returning of the corresponding semantic tree of sparse features, SURFScore _iIt is the score of i picture returning of the corresponding semantic tree of sparse features (SURF); Dense feature (HOG) is identical with the normalized computing formula of sparse features (SURF).

The computing method of described unfixed weights may further comprise the steps:

(1) the adjacent mean distance of calculating result for retrieval:

$\mathrm{SURFAveDist}=\frac{{\mathrm{\Σ}}_{i=1}^{N_1-1}(\mathrm{SURFScore}\left(i\right)-\mathrm{SURFScore}(i+1))}{N_1-1};$

N1 is preceding 5% a picture number of result for retrieval, and SURFAveDist, HOGAveDist are respectively SURF Vocabulary tree and the adjacent mean distance of the corresponding result for retrieval of HOG Vocabulary tree; SURFScore _iIt is the score of i picture returning of the corresponding semantic tree of sparse features (SURF); HOGScore _iIt is the score of i picture returning of the corresponding semantic tree of dense feature (HOG);

(3) weights of the semantic tree of each feature construction calculate:

$\mathrm{SURFWeight}=\frac{\mathrm{SURFAveDist}}{\mathrm{SURFAveDist}+\mathrm{HOGAveDist}};$

SURFWeight is the weights of sparse features (SURF), and HOGWeight is the weights of dense feature (HOG).

Beneficial effect of the present invention:, improved the accuracy of retrieval greatly owing to adopt the method for above-mentioned many Feature Fusion; Owing to adopt dynamic weights, traditional fixedly weights have been changed, the further effect that improves graphic retrieve.

Description of drawings

Below in conjunction with the drawings and specific embodiments the present invention is described in further detail:

Fig. 1 is the process flow diagram of this off-line training semantic tree;

Fig. 2 is the process flow diagram of search method of the present invention;

Fig. 3 is the system schematic that SURF semantic tree of the present invention and HOG semantic tree are formed.

Embodiment

As shown in Figure 1, 2, the image search method based on many features self-adaptation merges includes following steps:

Choosing of A, characteristics of image: as shown in Figure 1, off-line training step is extracted based on the SURF feature of point with based on the HOG feature in zone each image in the image library; Selected feature will have certain complementarity to the describing method of picture material.

B, database images is extracted SURF and HOG feature, utilize layering kmeans algorithm to carry out cluster, make up SURF semantic tree and HOG semantic tree (Vocabulary tree), as shown in Figure 3, the semantic tree that square representative is made of the SURF feature, the semantic tree that the triangle representative is made of the HOG feature.

C, as shown in Figure 2 in the online retrieving stage, extracts SURF feature and HOG feature for the image of the needs retrieval of user's input.According to the search method of semantic tree, utilize SURF feature and HOG feature respectively at the enterprising line retrieval of corresponding semantic tree, and return result for retrieval separately.The database picture of result for retrieval for arranging from high to low by mark.

D, by the observation of experimental result being found when the result for retrieval of certain feature is more satisfactory the mark of result for retrieval has a more stable and distribution clocklike.When the retrieval effectiveness of certain feature is better, return in the result for retrieval, the score of adjacent image presents the downtrending of stair shape ladder, on the contrary, then presents mild downtrending.According to this certain characteristic key of rule automatic Evaluation result's quality, and give corresponding weights, its weight calculation method is as follows:

(1) mean distance calculates

$\mathrm{SURFAveDist}=\frac{{\mathrm{\Σ}}_{i=1}^{N_1-1}(\mathrm{SURFScore}\left(i\right)-\mathrm{SURFScore}(i+1))}{N_1-1}$

$\mathrm{HOGAveDist}=\frac{{\mathrm{\Σ}}_{i=1}^{N_1-1}(\mathrm{HOGScore}\left(i\right)-\mathrm{HOGScore}(i+1))}{N_1-1}$

N1 is 5% of a database picture sum.SURFAveDist, HOGAveDist are respectively SURF Vocabulary tree and the adjacent mean distance of the corresponding result for retrieval of HOG Vocabulary tree.

(2) dynamically weights calculate

$\mathrm{SURFWeight}=\frac{\mathrm{SUFRAveDist}}{\mathrm{SURFveDist}+\mathrm{HOGAveDist}}$

$\mathrm{HOGWeight}=\frac{\mathrm{HOGAveDist}}{\mathrm{SURFAveDist}+\mathrm{HOGAveDist}}$

E, mark normalization:

Because finally will merge the result for retrieval of various features, and the meaning of the absolute size of the reciprocal fraction of different characteristic representative and inequality, so will carry out normalization to mark.Each image has a mark respectively in the database in SURF vocabulary tree and HOG vocabulary tree result for retrieval, and wherein i picture returning of SURF Vocabulary tree must be divided into SURFScore _i, i the picture that HOG Vocabulary tree returns must be divided into HOGScore _i

${\mathrm{SURFNormScore}}_i=\frac{{\mathrm{SURFScore}}_i-\mathrm{SURFScoreMax}}{\mathrm{SURFScoreMax}-\mathrm{SURFScoreMin}}$

${\mathrm{HOGNormScore}}_i=\frac{{\mathrm{HOGScore}}_i-\mathrm{HOGScoreMax}}{\mathrm{HOGScoreMax}-\mathrm{HOGScoreMin}}$

Wherein, SURFNormScore _i, HOGNormScore _iBe respectively the SURFScore after the normalization _iWith HOGScore _iSURFScoreMax, SURFScoreMin are respectively the maximal value and the minimum value of N picture mid-score returning of the corresponding vocabulary tree of SURF.HOGScoreMax, HOGScoreMin are respectively the maximal value and the minimum value of N picture mid-score returning of the corresponding vocabulary tree of HOG.

Step F: the fusion of SURF vocabulary tree result for retrieval and HOG vocabulary tree result for retrieval:

Each image has a score at the result for retrieval of SURF Vocabulary tree in the database, equally a score is arranged also in the result for retrieval of HOG Vocabulary tree, and according to mark series arrangement from high to low.According to the score of each picture in different Vocabulary tree, and the weights of this Vocabulary tree, after the linear, additive,, and return the highest picture of mark to the total points rearrangement, computing formula is as follows:

TotalScore _i＝HOGWeight*HOGScore _i+SURFWeight*SURFScore _i

Wherein, TotalScore _iBe the final total points of database picture i,, return result for retrieval according to total points series arrangement from high to low.

The above is a preferred implementation of the present invention, certainly can not limit the present invention's interest field with this, should be understood that, for those skilled in the art, can make amendment or be equal to replacement technical scheme of the present invention, and not break away from the essence and the scope of technical solution of the present invention.

Claims (5)

1. image search method that merges based on many features self-adaptation is characterized in that: include following steps:

Choosing of A, characteristics of image: the plural characteristics of image that extracts each image in the image library;

B, the picture in the image library is extracted selected feature, utilize layering kmeans algorithm to construct the semantic tree of each feature correspondence then;

C, the retrieving images that the user is imported extract feature, in each semantic tree, retrieve then, and returning result for retrieval, result for retrieval is that the image in the database is high more to the similar more score of retrieving images according to each image in the database of being arranged on earth by height with the similarity of retrieving images;

D, according to the score distribution rule of each semantic tree result for retrieval, the quality of the corresponding semantic tree result for retrieval of this feature of automatic Evaluation, thus determine the weights of this semantic tree;

E, each semantic tree return the score of image and carry out normalized, make that the mark after the normalization can linear, additive;

F, the mark of the result for retrieval of all characteristics of image is merged: the final PTS of each image be multiply by the weights of this semantic tree in the database by the score of this image in each semantic tree, addition obtains then, then image is exported with the descending order of the mark of final result for retrieval; Wherein the weights of each semantic tree correspondence are that step e is dynamically determined.

2. the image search method that merges based on many features self-adaptation according to claim 1, it is characterized in that: in the described steps A, the characteristics of image that is extracted has two, be based on respectively a little sparse features (SURF) and based on the dense feature (HOG) in zone.

3. the image search method that merges based on many features self-adaptation according to claim 2 is characterized in that: step e, and the normalized computing formula of described sparse features (SURF) is:

SURFNormScore _iBe the characteristics of image mark after the normalization, SURFScoreMax, SURFScoreMin are respectively the maximal value and the minimum value of N picture mid-score returning of the corresponding semantic tree of sparse features, SURFScore _iIt is the score of i picture returning of the corresponding semantic tree of sparse features (SURF); Dense feature (HOG) is identical with the normalized computing formula of sparse features (SURF).

4. the image search method that merges based on many features self-adaptation according to claim 2, it is characterized in that: the computing method of described dynamic weights may further comprise the steps:

(1) the adjacent mean distance of calculating result for retrieval:

$\mathrm{SURFAveDist}=\frac{{\mathrm{\Σ}}_{i=1}^{N_1-1}(\mathrm{SURFScore}\left(i\right)-\mathrm{SURFScore}(i+1))}{N_1-1};$

N1 is preceding 5% a picture number of result for retrieval, and SURFAveDist, HOGAveDist are respectively SURF Vocabulary tree and the adjacent mean distance of the corresponding result for retrieval of HOG Vocabulary tree; SURFScore _iIt is the score of i picture returning of the corresponding semantic tree of sparse features (SURF); HOGScore _iIt is the score of i picture returning of the corresponding semantic tree of dense feature (HOG);

(2) weights of characteristics of image calculate:

$\mathrm{SURFWeight}=\frac{\mathrm{SURFAveDist}}{\mathrm{SURFAveDist}+\mathrm{HOGAveDist}};$

SURFWeight is the weights of sparse features (SURF), and HOGWeight is the weights of dense feature (HOG).

5. according to the image search method based on the fusion of many features self-adaptation described in the claim 2, it is characterized in that: the computing formula of described fusion method is:

TotalScore _i＝HOGWeight*HOGScore _i+SURFWeight*SURFScore _i

Wherein, TotalScore _iBe the final total points of database picture i,, return result for retrieval according to total points series arrangement from high to low.

Images