CN103793884A - Knowledge-constrained bridge target image pneumatic optical effect correction method - Google Patents

Abstract

The invention discloses a knowledge-constrained bridge target image pneumatic optical effect correction method. According to the method, a space constraint relationship knowledge base between a bridge target and a background area is established on a ground; ground preparation is carried out before a flight, and the space constraint relationship knowledge base is used to establish a multi-scale template; images taken during a high speed flight are preliminarily corrected by a pneumatic optical effect, then an interested bridge area is extracted, and then a target area of the interested bridge area is precisely corrected; and finally the corrected interested bridge target area and the background area are merged to obtain a processed real-time remote sensing flight image. In the invention, the space constraint relationship knowledge base is established; for the real-time bridge images captured in the hypersonic situation, the constraint knowledge is used to extract the interested bridge area to carry out the pneumatic optical effect precise correction; the correction accuracy of the bridge target is ensured; and the real-time performance of the correction is greatly improved.

Description

A kind of Bridge object image aero-optical effect bearing calibration of knowledge constraints

Technical field

The invention belongs to the crossing domain of Models For Space Science And Technology and digital image processing techniques, be specifically related to a kind of Airport Images intelligent correcting method based on knowledge constraints, be applied to navigation, remote sensing and the detection of hypersonic aircraft.

Background technology

Hypersonic vehicle remote sensing, detection, navigation and guidance are key areas of 21st century Aero-Space career development, have important scientific meaning and using value in following high-tech and the national economic development.Remote sensing, detection, navigation and guidance take hypersonic vehicle as platform faces the challenge of aero-optical effect.

Pneumatic optical is the subject that the impact on high-speed aircraft imaging detection is streamed at a high speed in research.While flight in endoatmosphere with the high-speed aircraft of optical imagery detection system, between optical dome and incoming flow, form complicated flow field, cause optical wavefront transmission distortion or transmission except heat radiation to disturb to optical imagery detection system, cause skew, the shake, fuzzy of object being observed image, this effect is just called pneumatic optic transmission effect.This effect has reduced the usefulness of imaging detection system, causes the forfeiture of remote sensing, detection, navigation and guidance function.Therefore need to invent new digital processing technology, improve and revert to the quality of picture, Here it is, and image recovers and proofreaies and correct.

The high frame frequency characteristic of the hypersonic flight of aircraft and imaging system, has higher requirement for operation efficiency and the performance of correction and recognizer.Mostly blind deconvolution algorithm is to process for full figure, has not only wasted a lot of time for the correction of nontarget area, affects the real-time of algorithm, and non-target interval characteristic (as flatness etc.) can finally have influence on target area and proofreaies and correct.And the method for correcting image of knowledge constraints carrys out guide image correction by the knowledge constraints of extracting in image, for target area (bridge) fine correction, and for slightly proofreading and correct nontarget area, the processing of recursion from coarse to fine, reach suspicious object region (region of interest) in real time, accurately proofread and correct, can significantly improve correcting algorithm efficiency and precision.

Summary of the invention

The invention provides a kind of Bridge object aero-optical effect bearing calibration of knowledge constraints, can be under hypersonic flight condition, both guaranteed the precision that Bridge object is proofreaied and correct, greatly improve again the real-time of proofreading and correct.

The Bridge object image aero-optical effect bearing calibration of a kind of knowledge constraints provided by the invention, is characterized in that, the method comprises the steps:

A sets up space constraint relation:

Image | bridge; Waters; Land; Bridge is across waters, and the body of a bridge is longitudinally waters, extends transverse to land }, using this as constraint knowledge;

The process of establishing of the multiple dimensioned template of B is:

(B1) obtain the visible images to low latitude by high-altitude of corresponding bridge region from google, and record respectively position in the drawings, Bridge object district under each yardstick;

(B2) will be divided into three intervals to low latitude by high-altitude, each interval obtains a width visible images, and the line number of small scale, mesoscale, large scale template is identical with line number and the columns of the visible images of respective bins with columns;

The flight of C Real-time Remote Sensing is processed:

(C1) aero-optical effect preliminary correction

Adopt the maximal possibility estimation algorithm under the constraint of Hu square, the realtime graphic of each yardstick of taking is from high to low carried out to preliminary correction;

(C2) extract bridge interested district

At Real-time Remote Sensing in-flight, for the image of the different scale after preliminary correction, with the template matches of corresponding yardstick, extract Bridge object interested district respectively;

(C3) Bridge object interested district fine correction

Adopt the maximal possibility estimation algorithm under the constraint of Hu square to carry out aero-optical effect fine correction to Bridge object interested district;

(C4) the Bridge object interested district after proofreading and correct and background area are merged, obtain the Real-time Remote Sensing image after treatment that flies.

The present invention is directed in high-speed flight process, relatively low to the correcting rate of obtained image, lack real-time, high efficiency, a kind of Bridge object aero-optical effect bearing calibration of knowledge constraints has been proposed, technique effect of the present invention is embodied in:

1. the present invention sets up the space constraint relational knowledge base of Bridge object and background area on ground, and if bridge is across waters, the body of a bridge is longitudinally waters, extends transverse to land, and bridge, land and waters gray scale have notable difference, and waters uniform gray level.

2. the present invention carries out ground preparation before flight, utilizes space constraint relational knowledge base, sets up multiple dimensioned template, for multiple dimensioned realtime graphic, the template matches that adopts corresponding scale, makes in high-speed flight process, can extract in real time, efficiently Bridge object interested district.

3. the bearing calibration based on knowledge constraints that the present invention proposes, only interested Bridge object district is carried out to aero-optical effect fine correction for the image of taking in high-speed flight process, the correction time of reduction greatly, well improve the real-time that under hypersonic flight condition, Bridge object is proofreaied and correct.

The present invention sets up space constraint relational knowledge base, to the bridge realtime graphic of hypersonic lower shooting, utilize constraint knowledge, extract bridge interested district and carry out aero-optical effect fine correction, both guarantee the correction accuracy of Bridge object, greatly improved again the real-time of proofreading and correct.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of the inventive method;

Fig. 2 is the visible images of the bridge region that obtains from google;

Fig. 3 (a), Fig. 3 (b), Fig. 3 (c) are the visible images to low latitude by high-altitude of the corresponding bridge region that obtains;

Fig. 4 (a), Fig. 4 (b), Fig. 4 (c) be by high-altitude to the bridge interested district recording in the visible images in low latitude;

Fig. 5 (a), Fig. 5 (b), Fig. 5 (c) are small scale, mesoscale, the large scale templates of setting up respectively;

Fig. 6 (a) is mesoscale realtime graphic, and Fig. 6 (b) is large scale realtime graphic;

Fig. 7 is the large scale image Pre_recover after aero-optical effect preliminary correction;

Fig. 8 has shown the position of coupling rear pattern plate in image Pre_recover;

Fig. 9 has shown the position of Bridge object interested district in image Pre_recover;

Figure 10 is Bridge object interested district;

Figure 11 is the Bridge object interested district after aero-optical effect fine correction;

Figure 12 is the image after Bridge object interested district and the background area merging after proofreading and correct.

Embodiment

Below in conjunction with accompanying drawing, the specific embodiment of the present invention is described further.It should be noted that at this, be used for helping to understand the present invention for the explanation of these embodiments, but do not form limitation of the invention.In addition,, in each embodiment of described the present invention, involved technical characterictic just can combine mutually as long as do not form each other conflict.

As shown in Figure 1, the inventive method mainly comprises that prepare on the space constraint relation of setting up, ground and Real-time Remote Sensing flight is processed;

A sets up space constraint relation

Obtain the visible images of corresponding bridge region from google, as Fig. 2.Can analyze and draw down depending on the typical scene of bridge image and formed by bridge, waters, land.

Bridge is expressed as:

Bridge | bridge length breadth ratio P, P _min<P<P _max(P _minfor possible minimum length breadth ratio, P _maxfor possible maximum length breadth ratio); Parallel neighborhood is waters; Vertical neighborhood is land-based area; Bridge is conventionally brighter };

Waters is expressed as:

Waters | and average gray value G<H, H is the higher limit (waters is conventionally darker) of the G of estimation; Waters is blocked by bridge; Area sw, sw _min<sw<sw _max(sw _minfor possible minimum area, sw _maxfor possible maximum area) };

Land is expressed as:

Land | land is cut off by waters, is connected by bridge; Land gray scale is inhomogeneous };

Overall scenario image expression is:

Image | bridge; Waters; Land; Bridge is across waters, and the body of a bridge is longitudinally waters, extends transverse to land }, using this as constraint knowledge.

B ground set-up procedure, specifically comprises the steps:

(1) obtain the visible images to low latitude by high-altitude of corresponding bridge region from google, successively as shown in Fig. 3 (a), Fig. 3 (b), Fig. 3 (c), and record respectively position in the drawings, Bridge object district under each yardstick, for explaining conveniently, the starting point unification in Bridge object district is designated as s (x _s, y _s), terminal unification is designated as E (x _e, y _e), successively as shown in Fig. 4 (a), Fig. 4 (b), Fig. 4 (c).

(2) set up multiple dimensioned template

To be three intervals by Dao Fen low latitude, high-altitude, as more than pressing 5km, 2-5km, below 2km, the visible images that each interval obtains, the line number of small scale, mesoscale, large scale template and columns are with corresponding identical to line number and the columns of the visible images in low latitude by high-altitude.

According to constraint knowledge, the template of foundation is made up of bridge, land, these three regions, waters, and the shape in each region is identical with the shape in three regions in corresponding visible images, records in each yardstick template the starting point in Bridge object district and terminal.

In each yardstick template, the starting point in Bridge object district and terminal also unify to be designated as s (x accordingly _s, y _s) and E (x _e, y _e).

According to the difference of gray scale, the gray-scale value in template Bridge, land and waters is set.By analyzing the gamma characteristic in bridge, land, waters, as this three regions height and homogeneity etc. of gray scale separately, conventionally the gray scale of each template bridge target is made as to 255, the gray scale of land background is made as 128, and the gray scale of waters background is made as 0.

Fig. 5 (a), Fig. 5 (b), Fig. 5 (c) are respectively small scale, mesoscale, the large scale templates of setting up.

The flight of C Real-time Remote Sensing is processed, and specifically comprises the steps:

(1) aero-optical effect preliminary correction

Realtime graphic to low yardstick, mesoscale and the large scale of taking from high to low carries out preliminary correction.

Fig. 6 (a) and Fig. 6 (b) are the mesoscale of shooting and the realtime graphic of large scale.

Be explained as an example of realtime graphic example below:

To realtime graphic Blur, adopt the maximal possibility estimation algorithm under the constraint of Hu square to carry out preliminary correction, obtain image Pre_recover.

If f (x) be target image in coordinate x place intensity, h (x) be point spread function in coordinate x place intensity, g (x) be degraded image in coordinate x place intensity, n is iterations,

represent convolution.The iterative estimate formula of target image f is

$f^{(n+1)}\left(x\right)=(\frac{g\left(x\right)}{f^{\left(n\right)}\left(x\right)\&CircleTimes;h^{\left(n\right)}\left(x\right)}\&CircleTimes;h^{\left(n\right)}(-x))f^{\left(n\right)}\left(x\right)---\left(1\right)$

The iterative estimate formula of point spread function h is

$h^{(n+1)}\left(x\right)=(\frac{g\left(x\right)}{f^{\left(n\right)}\left(x\right)\&CircleTimes;h^{\left(n\right)}\left(x\right)}\&CircleTimes;f^{\left(n\right)}(-x))h^{\left(n\right)}\left(x\right)---\left(2\right)$

If f is (x ₁, y ₁) be that image f is at x ₁row y ₁the gray-scale value at row place, m and n are line number and the columns of target image f, the p+q rank moment of the orign of target image f is defined as

$M_{\mathrm{pq}}=\underset{x_1=1}{\overset{m}{\mathrm{\&Sigma;}}}\underset{{\mathrm{<figure-callout\; id="1"\; label="y"\; filenames="HDA0000450836750000011.png"\; state="\{\{state\}\}">y</figure-callout>}}_1=1}{\overset{n}{\mathrm{\&Sigma;}}}{x_1}^p{y_1}^{q}f(x_1,y_1)$

Image f p+qJie center square is defined as

$u_{\mathrm{pq}}=\underset{x_1=1}{\overset{m}{\mathrm{\&Sigma;}}}\underset{{\mathrm{<figure-callout\; id="1"\; label="y"\; filenames="HDA0000450836750000011.png"\; state="\{\{state\}\}">y</figure-callout>}}_1=1}{\overset{n}{\mathrm{\&Sigma;}}}{(x_1-x_0)}^p{(y_1-y_0)}^{q}f(x_1,y_1)$

Wherein x ₀, y ₀for image center of gravity

$\left\{\begin{array}{c}{x}_0=M_{10}/M_{00}\\ y_0=M_{01}/M_{00}\end{array}\right.$

Standardization center square is defined as

$I_{\mathrm{pq}}=u_{\mathrm{pq}}/u_{00}^{1+(p+q)/2}$

One of Hu not bending moment be

C ₁=I ₂₀+I ₀₂

Carry out concrete alternative manner for each realtime graphic as follows:

1. set the parameter of iteration

Using realtime graphic Blur as degraded image, the initial value of target image is Blur, and initial point spread function is α * α matrix, and the value of each element of matrix is 1/ (α * α), sets maximum iteration time n _max, in this example, α=31, n _max=5;

2. according to formula (2), point spread function h is carried out to iteration;

3. according to formula (1), target image f is carried out to iteration.If the C of target image after iteration ₁square is greater than the C of the target image of a front iteration ₁square, abandons this iteration, and proceeds to step 2..Otherwise proceed to step 4..

4. the C of target image after iteration ₁square is less than the C of the front target image of iteration ₁the setting multiple (as 0.9 times) of square, proceed to step 2., otherwise iteration completes.Output iteration result, i.e. target image after last iteration, as the image Pre_recover after aero-optical effect preliminary correction.Image after large scale is proofreaied and correct is as Fig. 7.

(2) extract bridge interested district

At Real-time Remote Sensing in-flight, for the image of the different scale after preliminary correction, should be respectively and the template matches of corresponding yardstick.Detailed process is as follows:

By image Pre_recover and corresponding yardstick template matches.If the size of image Pre_recover is m _p× n _p, the size of corresponding yardstick template is m ₀× n ₀.

Asking all sizes in Pre_recover is m ₀× n ₀the correlation coefficient r of subnumber group A and corresponding yardstick template array B:

$r=\frac{\underset{m_0}{\mathrm{\&Sigma;}}\underset{n_0}{\mathrm{\&Sigma;}}(A_{m_0{n}_0}-\stackrel{\&OverBar;}{A})(B_{m_0{n}_0}-\stackrel{\&OverBar;}{B})}{\sqrt{\left[\underset{m_0}{\mathrm{\&Sigma;}}\underset{n_0}{\mathrm{\&Sigma;}}{(A_{m_0{n}_0}-\stackrel{\&OverBar;}{A})}^2\right][\underset{m_0}{\mathrm{\&Sigma;}}\underset{n_0}{\mathrm{\&Sigma;}}}{(B_{m_0{n}_0}-\stackrel{\&OverBar;}{B})}^2]}$

Wherein

for the average of array A, for the average of array B.

for array A is at m ₀row n ₀the element value at row place,

for array B is at m ₀row n ₀the element value at row place, the r obtaining the successively size of conduct is (m _p-m ₀+ 1) × (n _p-n ₀+ 1) value of the element of matrix R.

In matrix R, the coordinate (x when finding out r and getting maximal value _maxy _max).In figure Pre_recover, the position of template is (x _max, y _max) to (x _max+ m ₀-1, y _max+ n ₀-1) rectangular area between, large scale as Fig. 8.Interested Bridge object district Bridge is (x _max+ x _s-1, y _max+ y _s-1) to (x _max+ x _e-1, y _max+ y _e-1) rectangular area between, large scale as Fig. 9, the district as a setting, region beyond Bridge object interested district.Demonstrate separately Bridge object interested district, large scale as Figure 10.

(3) Bridge object interested district fine correction

Adopt the maximal possibility estimation algorithm under the constraint of Hu square to carry out aero-optical effect fine correction to Bridge object interested district.In iterative process, degraded image is Bridge, and target image initial value is Bridge, and initial point spread function is α * α matrix, and the value of each element of matrix is 1/ (α * α), predetermined maximum iteration time n _max(if value is 40).

Bridge object interested district after correction is Bridge_recover, large scale as Figure 11.

(4) the Bridge object interested district after proofreading and correct and background area are merged

In figure Pre_recover, the gray-scale value of the each pixel of background area is constant, (x _max+ x _s-1, y _max+ y _s-1) to (x _max+ x _e-1, y _max+ y _e-1) rectangular area between is filled by the Bridge object interested district Bridge_recover after proofreading and correct, and the image after filling is the Real-time Remote Sensing image after treatment that flies, as shown in figure 12.

The above is preferred embodiment of the present invention, but the present invention should not be confined to the disclosed content of this embodiment and accompanying drawing.Do not depart from the equivalence completing under spirit disclosed in this invention so every or revise, all falling into the scope of protection of the invention.

Claims (5)

1. a Bridge object image aero-optical effect bearing calibration for knowledge constraints, is characterized in that, the method comprises the steps:

A sets up space constraint relation:

Image | bridge; Waters; Land; Bridge is across waters, and the body of a bridge is longitudinally waters, extends transverse to land }, using this as constraint knowledge;

The process of establishing of the multiple dimensioned template of B is:

(B1) obtain the visible images to low latitude by high-altitude of corresponding bridge region from google, and record respectively position in the drawings, Bridge object district under each yardstick;

(B2) will be divided into three intervals to low latitude by high-altitude, each interval obtains a width visible images, and the line number of small scale, mesoscale, large scale template is identical with line number and the columns of the visible images of respective bins with columns;

The flight of C Real-time Remote Sensing is processed:

(C1) aero-optical effect preliminary correction

Adopt the maximal possibility estimation algorithm under the constraint of Hu square, the realtime graphic of each yardstick of taking is from high to low carried out to preliminary correction;

(C2) extract bridge interested district

At Real-time Remote Sensing in-flight, for the image of the different scale after preliminary correction, with the template matches of corresponding yardstick, extract Bridge object interested district respectively;

(C3) Bridge object interested district fine correction

Adopt the maximal possibility estimation algorithm under the constraint of Hu square to carry out aero-optical effect fine correction to Bridge object interested district;

(C4) the Bridge object interested district after proofreading and correct and background area are merged, obtain the Real-time Remote Sensing image after treatment that flies.

2. the Bridge object image aero-optical effect bearing calibration of knowledge constraints according to claim 1, is characterized in that, the specific implementation process of step (C1) is:

1. set the parameter of iteration

Degraded image is realtime graphic, is designated as Blur, and the initial value of target image is Blur, sets initial point spread function, and the value of each element of matrix, maximum iteration time n _max;

2. according to following formula II, point spread function h is carried out to iteration:

$h^{(n+1)}\left(x\right)=(\frac{g\left(x\right)}{f^{\left(n\right)}\left(x\right)\&CircleTimes;h^{\left(n\right)}\left(x\right)}\&CircleTimes;f^{\left(n\right)}(-x))h^{\left(n\right)}\left(x\right);$ Formula II

Wherein, establish f (x) for target image in coordinate x place intensity, h (x) be point spread function in coordinate x place intensity, g (x) be degraded image in coordinate x place intensity, n is iterations,

represent convolution;

3. according to formula I, target image f is carried out to iteration, if the C of target image after iteration ₁square is greater than the C of the target image of a front iteration ₁square, abandons this iteration, and proceeds to step 2., otherwise proceeds to step 4.;

$f^{(n+1)}\left(x\right)=(\frac{g\left(x\right)}{f^{\left(n\right)}\left(x\right)\&CircleTimes;h^{\left(n\right)}\left(x\right)}\&CircleTimes;h^{\left(n\right)}(-x))f^{\left(n\right)}\left(x\right)$ Formula I

Wherein, the C of target image ₁the calculating formula of square is:

C ₁=I ₂₀+I ₀₂

The p+q rank moment of the orign of target image f is defined as

$M_{\mathrm{pq}}=\underset{x_1=1}{\overset{m}{\mathrm{\&Sigma;}}}\underset{y_1=1}{\overset{n}{\mathrm{\&Sigma;}}}{x_1}^p{y_1}^{q}f(x_1,y_1)$

Image f p+qJie center square is defined as

$u_{\mathrm{pq}}=\underset{x_1=1}{\overset{m}{\mathrm{\&Sigma;}}}\underset{y_1=1}{\overset{n}{\mathrm{\&Sigma;}}}{(x_1-x_0)}^p{(y_1-y_0)}^{q}f(x_1,y_1)$

Wherein x ₀, y ₀for image center of gravity

$\left\{\begin{array}{c}{x}_0=M_{10}/M_{00}\\ y_0=M_{01}/M_{00}\end{array}\right.$

Standardization center square is defined as

$I_{\mathrm{pq}}=u_{\mathrm{pq}}/u_{00}^{1+(p+q)/2}$

4. the C of target image after iteration ₁square is less than the C of the target image of a front iteration ₁2. the setting multiple of square, proceed to step, otherwise enter step 5.; ;

5. export iteration result, i.e. target image after last iteration, as the each scalogram picture after aero-optical effect preliminary correction.

3. the Bridge object image aero-optical effect bearing calibration of knowledge constraints according to claim 1, is characterized in that, the specific implementation process of step (C2) is:

By image Pre_recover and corresponding yardstick template matches, the size of establishing image Pre_recover is m _p× n _p, the size of corresponding yardstick template is m ₀× n ₀;

Asking all sizes in Pre_recover is m ₀× n ₀the correlation coefficient r of subnumber group A and corresponding yardstick template array B:

$r=\frac{\underset{m_0}{\mathrm{\&Sigma;}}\underset{n_0}{\mathrm{\&Sigma;}}(A_{m_0{n}_0}-\stackrel{\&OverBar;}{A})(B_{m_0{n}_0}-\stackrel{\&OverBar;}{B})}{\sqrt{\left[\underset{m_0}{\mathrm{\&Sigma;}}\underset{n_0}{\mathrm{\&Sigma;}}{(A_{m_0{n}_0}-\stackrel{\&OverBar;}{A})}^2\right][\underset{m_0}{\mathrm{\&Sigma;}}\underset{n_0}{\mathrm{\&Sigma;}}}{(B_{m_0{n}_0}-\stackrel{\&OverBar;}{B})}^2]}$

Wherein

for the average of array A,

for the average of array B.

for array A is at m ₀row n ₀the element value at row place, for array B is at m ₀row n ₀the element value at row place, the r obtaining the successively size of conduct is (m _p-m ₀+ 1) × (n _p-n ₀+ 1) value of the element of matrix R;

In matrix R, the coordinate (x when finding out r and getting maximal value _max, y _max);

Interested Bridge object district Bridge is (x _max+ x _s-1, y _max+ y _s-1) to (x _max+ x _e-1, y _max+ y _e-1) rectangular area between, the district as a setting, region beyond Bridge object interested district.

4. the Bridge object image aero-optical effect bearing calibration of knowledge constraints according to claim 1, is characterized in that, the specific implementation process of step (C3) is:

In iterative process, degraded image is Bridge, and target image initial value is Bridge, sets the value of each element of initial point spread function and matrix, and maximum iteration time n _max.

5. the Bridge object image aero-optical effect bearing calibration of knowledge constraints according to claim 4, initial point spread function is 31*31 matrix, the value of each element of matrix is 1/ (31*31).

Images