WO2016071896A1

WO2016071896A1 - Methods and systems for accurate localization and virtual object overlay in geospatial augmented reality applications

Info

Publication number: WO2016071896A1
Application number: PCT/IL2014/051098
Authority: WO
Inventors: Garry Haim ZALMANSON; Yosef SHVARTZMAN; Nir Yehezkel SOLOMON
Original assignee: L.M.Y. Research & Development Ltd.
Priority date: 2014-11-09
Filing date: 2014-12-16
Publication date: 2016-05-12

Abstract

A method and system for accurate localization and virtual object overlay in geospatial augmented reality applications are provided herein. The method may include: obtaining a plurality of source images having a common region of interest; interlinking said source images by at least one of: finding tie points between each source image to at least one other source image; and associating pixels in said source images with a common geolocation data; determining a geolocation of objects based on geo locating techniques, wherein the objects are accurately geo-referenced in a frame of reference of the interlinked plurality of source images; and interlinking a newly obtained image to said source images so that said objects are accurately geo-referenced onto the newly obtained image.

Description

METHODS AND SYSTEMS FOR ACCURATE LOCALIZATION AND VIRTUAL OBJECT OVERLAY IN GEOSPATIAL AUGMENTED REALITY

APPLICATIONS

FIELD OF THE INVENTION

The present invention relates generally to the field of augmented reality, more particularly it relates to accurately determining georeferenced line-of-site for each pixel on an electronic device, allowing accurate positioning of digital objects onto real- world view.

BACKGROUND OF THE INVENTION

Augmented Reality is the concept of adding synthetic objects onto the vision of a spectator, in order to enhance their perception of the world. A pilot helmet HUD is an example of such system (hardware and software), projecting flight information, such as airplane bearing and inclination, onto a transparent screen so that the pilot won't have to look away from the sky in order to look at flight information. The system thus enables the pilot to perform his/hers task more efficiently.

An extremely useful augmented reality application feature is to accurately project information placed in context of its real-world location ("geo-tagged information"), onto the user's display. A non-limiting example can be a navigation application, where the application can highlight precise geo-tagged destinations, such as the desired destination's entrance door.

Accurately positioning geo-tagged information onto mobile devices' display has not been yet been demonstrated without the aid of physical markers on the targets, and this can be attributed to the properties of the physical devices. Since the late 2000's, compact mobile devices such as smartphones became readily available, and are a fully-fledged platform for augmented reality applications, sporting a camera, display, orientation and location (6DOF) sensing, communication and processing capabilities. Future platforms, such digital glasses, are to sport similar capabilities.

The main problem with such devices, although relatively cheap, is that the accuracy of the 6DOF sensing is relatively low, namely the GPS (tens of meters), the digital compass (several degrees), and elevation and roll (several degrees). These accuracies may be sufficient for many applications, but the accurate positioning of real-world coordinates onto the screen is not possible without severe errors. For example, 5 degree pointing error causes a typical phone to miss by 250 pixels. In addition, random noise (pointing error changes rapidly over time) causes the image to jitter. On top of that, adding self-location errors, and pointing errors which are typically much worse than 5 degree, yields display placement errors of hundreds of pixels, which renders the application useless for practical use, as the information is placed in the wrong place.

Photogrammetry is the art and science for determining geometrical properties of an object from one or more images.

Geo referenced spatial imagery has been known to be used with photogrammetric tools in order to improve the accuracy of airborne and space-borne imagery for mapping application. It has not been used in the context of improving ground based imagery for augmented reality applications. It would be advantageous to combine the tools of photogrammetry in order to address some of the challenges imposed by accurate positioning of geo-tagged information in augmented reality application.

SUMMARY OF THE INVENTION

According to some embodiments of the present invention, a method for accurately placing geo-tagged information onto a non-accurate device's display, primarily for augmented reality applications.

In some embodiments, a method and a system for accurate localization and virtual object overlay in geospatial augmented reality applications are provided herein. The method may include: obtaining source images containing at least one object associated with geolocation information; interlinking said source images by finding tie points between each source image to at least one other source image; indexing the objects of the interlinked source images, to yield a geo- referenced database of clustered objects; determining a virtual geospatial location of the objects based on geo locating techniques, wherein the objects are clustered; storing the determined virtual geospatial location of the objects on a clustered database; and utilizing the virtual location of the cluster to geo-reference at least one object contained in a newly taken image, wherein said method is applicable for line of sight (LOS) and/or augmented reality (AR) applications.

These additional, and/or other aspects and/or advantages of the present invention are set forth in the detailed description which follows. BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention and in order to show how it may be implemented, references are made, purely by way of example, to the accompanying drawings in which like numerals designate corresponding elements or sections. In the accompanying drawings: Figure 1 illustrates the core structure of the proposed system, in an exemplar embodiment. It illustrates the data flow, from raw, inaccurately positioned imagery, to new imagery with virtual objects precisely overlayed on top of it;

Figure 2 illustrates the core process of the proposed method, in an exemplar embodiment. It is roughly suited to be used in the system showed in Figure 1 , although it can be used;

Figure 3 illustrates the meaning of the terms "Virtually Geo Referenced" and "Virtually Geo Located", a core concept in understanding the novelty of the present invention; and

Figure 4 illustrates a process of an exemplar application in the field of tourism, based on the core process illustrated in figure 2.

The drawings together with the following detailed description make the embodiments of the invention apparent to those skilled in the art.

DETAILED DESCRIPTION OF THE INVENTION

Prior to setting a detailed description of embodiments of the present invention, it may be helpful to set forth definitions of certain terms that will be used hereinafter.

The term "coordinate frames" as used herein, is defined as framed used to represent and measure properties of objects, such as their position and orientation. Ground frame is a three-dimensional Cartesian coordinate system (X, Y, Z) adapted for locating objects in the physical space. Ground frames may be global (e.g. WGS84) or local - determined ad hoc to support relative observations.

The term "image frame" as used herein, is defined as a two-dimensional coordinate system (x, y) related to the image plane of the camera. The origin of the image coordinate system (x, y) is located at the intersection of the camera optical axis with the image plane. A point in the image plane (x, y) may be identified by pixel index (c, r) since digital cameras use two-dimensional array sensors in the image plane to capture the incoming electromagnetic signal. The term "georeferencing" (aka camera calibration, in computer vision terminology) as used herein, refers to the process of determining the external and the internal parameters of an image. The external parameters of an image (also referred to as external orientation) refer to a position and orientation of an image in space, i.e. the position and a line of sight of the camera which had acquired the image. The external parameters are usually referred to as "6DOF" (six degrees of freedom) since these parameters comprise three rotation angles (Euler angles) that describe the rotations about three principal axes needed to rotate from the ground system into the image system (augmented with a z axis pointing along the camera optical axis) and three coordinates of the camera in the ground system. The term "internal parameters" as used herein, refer to intrinsic properties of the camera. Internal parameters may be comprised of the camera's focal length, distortions and a geometric transformation aligning the detector array within the camera system.

The term "camera model" as used herein, refers to a mathematical formula which models a transformation from an object domain to an image domain using the internal parameters and external parameters.

The term "tie point" as used herein, refers to a scene point in the physical space, if this scene point can be identified in a reference georeferenced image and in an image undergoing the georeferencing process. The term "control point" as used herein, refers to a scene point in the physical space, if this scene point has known coordinates in the ground system and can be identified in an image undergoing the georeferencing process.

The term "camera central Line of Sight" or simply "camera central LOS" as used herein, refers to a direction of a vector corresponding to an optical axis of a camera acquiring an image. It can be understood as a vector originating at the optical center of the camera and passing through an object in the physical world appearing at a center of the image. In our case we will have this vector determining an orientation (or the normal vector) of the surface to be imaged by a remote camera. Given the rotation matrix R of section "camera model" above, the camera central LOS is specified by a vector corresponding to the third row of the R matrix multiplied by (-1).

The term "georeferencing using control points" as used herein, refers to a process that given a sufficient number of control points (X, Y, Z) <→ (R,C) the image internal and external orientation parameters (K,R and C) can be determined using many well-known linear and iterative optimization strategies. The exact number of the required control points depends on a prior knowledge of the imaging geometry and potentially additional constraints applied to the 11 parameter calibration model.

The term "georeferencing using tie points" as used herein, refers to a process that given at least two georeferenced images, Ref_A and Refe, (with camera parameters deemed sufficient for the sought application) and an image Img_c undergoing georeferencing, the parameters of Imgc can be computed using the "gold- standard" iterative optimization method called Bundle-Adjustment (BA). BA is defined as the problem of simultaneously refining the 3D coordinates describing the scene geometry as well as the camera parameters of the images, according to an optimality criterion involving the corresponding image projections of all points. In embodiments of the present invention case, since Ref_A and Ref_B images are already georeferenced (hence uniquely defining the ground referential) the Tie Points between Imgc and the pair of Ref_A and Refe images allow the computation of Imgc parameters in the ground reference system. The tie points may be obtained manually or by some image processing means. With specific reference now to the drawings in detail, it is stressed that the particulars shown are for the purpose of example and solely for discussing the preferred embodiments of the present invention, and are presented in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention. The description taken with the drawings makes apparent to those skilled in the art how the several forms of the invention may be embodied in practice. Before explaining the embodiments of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of the components set forth in the following descriptions or illustrated in the drawings. The invention is applicable to other embodiments and may be practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting.

Referring to Figure 1 , the technical goal of the system is to enable the accurate overlay of virtual objects onto images originating from source n+1 (210), thus producing the product (280). Accurate, in this sense, means overlay (by means of projection) of geo-tagged information under an acceptable threshold in the image plane, typically under tens of pixels. Source n+1 , and all other n image sources (110) however, sport sensors which enable the projection in an unacceptable accuracy, typically hundreds of pixels, as described in the background of the invention. The proposed system is as follows. It refers to two subsystems - a preprocess system (100) and a real-time, continuous operation system (200).

The preprocess system (100) operates as follows. Image Sources (110) are generally available geo-referencing imagery sources, said geo-referencing can be relatively inaccurate. Notable non-limiting examples are geo-tagging-enabled handheld calibrated cameras, available panoramic imagery databases, and crowdsourced geo-tagged imagery from smartphones. Imagery Interlinker (120), in its broadest sense, uses some sort of mathematical manipulation to index, register and improve the relative geo-referencing of images from said image sources. It typically consists of automatic or automated image registration techniques used for the creation of tie and control points between the images and/or external data, and using said points in a process minimizing the relative errors between the geo-referencing of the image. The result is typically an improved external orientation model (accurate central Line-of- Sight, LOS). If the sources have precise camera models, necessarily there is an accurate LOS for any pixel in the image. If not, some of said points can be used to calibrate the internal parameters of the camera models of the sources.

Virtually Geo Referenced Imagery Cluster Database (VGIC) (130) is an indexed image database, which contains the geo-referenced imagery and any metadata needed for the relevant retrieval of the images. This can, but not limited to, a photo-sphere described in Israeli patent application 217666 by some of the present inventors.

Geolocation Techniques (140) are (usually) generic techniques used to approximate 3D location of objects from well geo-positioned images using methods such as ray intersection, single ray intersection with a modeled surface, among others, and should be known to any person skilled in the art. This can also refer to more specialized methods of geolocation, a combination of methods, or others. These techniques can be used to geolocate points, or regions, and attaching relevant metadata to them (such as the place's name). These objects can be projected from the virtual 3D space onto any image plane.

Virtually Geo Located Objects Database (VGODB) (150) is a database of geospatial entities, derived using said techniques from images from the VGIC (130), along with metadata and generally available retrieval methods.

All the previous steps are done as a preprocess subsystem (100), before the actual processing of images from source n+1 begins. The following paragraphs describe the processing done by subsystem which serves a user holding image source n+1 (200). It should be noted that the system can by distributed among a client and a server (such as a smartphone app communicating with a server over the internet), and can be replicated as many times as needed (for example, multiple clients). Source n+1 records an image m (220), along with inaccurate georeferencing (such as location and pointing direction).

Find Match (230) is a key module in the process. It searched the VGIC (130), using, among others, key features in the image, and the rough georeferencing metadata of the image. It produces a candidate group k (240), which consists of images {ko, k_n}, all existing in the VGIC, and are of the highest probability of matching success with image m. Matching in this case is an automatic or automated process of creating tie points, between image m and image(s) from group k. It can be a simple LOS proximity mechanism, or a specialized mechanism. "Match" can also be defined as maximal score in Generalized Proximity Criterion.

Imagery Inter linker (250) is the module in charge of improving image m's external (and sometimes internal) parameters, based on the matching tie points found between m and any subset of group k (including one and all images). This module can be identical to Imagery Interlinker (120), or a specialized version tailored to the case of matching one image to a group. One can even envision an embodiment in which Imagery Interlinker (250) and Imagery Interlinker (120) are actually the same module. This possible embodiment may yield the same results in fewer modules, but may impose a centralized architecture and or higher communication bandwidth.

The result of the process is an interlinked image m (260), interlinked in the sense is that its georeferencing is accurate, in relation to selected images in group k. Since the selected images in group k (240) are, in turn, have accurate georeferencing in relation to any image in the VGIC (130). Therefore, image m in interlinked with the imagery in the VGIC, an important property that can enable two actions.

The first, and most important, is that any object in the VGODB (150), can be accurately projected onto the image coordinate frames, even if its geolocation is not absolutely accurate. This property of the objects in the VGODB, will be further explained in Figure 3. Thus, relevant object(s) of choice can be projected using the sensor model (readily understandable by any person skilled in the art) using a dedicated module (270), thus producing the required result - geo-located objects accurately displayed onto image m (280).

The second action, which is not critical to the main process but is of value, is that the new interlinked image m and its metadata (260) can be stored back in the VGIC (130), thus producing an update to the VGIC. This is extremely useful when used to update future frames from source n+1 , or from different sourced in the same location.

Referring to Figure 2, a method is described, for the precise overlay geo-located object onto a new image. The method complements the system in Figure 1, describing similar steps, which can be implemented in the relevant modules. Explanations are relevant to both, but are not repeated in order not to redundantly repeat information.

The method is divided between two sub-processes: an offline, one time process (300) and an online, continuous one (400). The steps of the pre-process are as follows:

Take images with (unnecessarily accurate) geolocation information. Interlink said images by finding tie points between each image to at least one other image, perform robust weighted free-net adjustment to the telemetry of each image(310).

Perform indexing and store the solution in a database (320), called Virtually Geo Referenced Imagery Cluster, or VGIC. This can, but not limited to, a photo-sphere feature in which proximity metric is calculated based on an image to every member of a selection of geo-referenced images, for each pair. The calculating include the following steps: calculating the difference in angular orientation between said images, calculating the weighted resolution ratio between said images, calculating the spatial orientation accuracy, providing weighting for each of said calculations, calculating a single metric using the weighting and said calculations, using said proximity metric to select from the group, wherein the selected images are the closest matches to said image for utilizing image matching algorithms. The aforementioned proximity metric is described in further details in Israeli patent application No. 217666 which is incorporated herein by reference in its entirety. Use common geolocation techniques such as ray intersection, single ray intersection with surface etc. or more advanced techniques to create geospatial objects, in order to determine the virtual point, area or volume location of object(s) of interest (330). Store the object location, along with relevant context, in a database (340), called Virtually Geo Located Objects Database, or VGODB.

The continuous process can be described in the following steps:

Take new image, including rough localization.

Use the localization metadata to find the most candidate matches in the VGIC (recall that some sort of indexing took place in the creation of said database, using to query the database). Matching can be a simple measure (as a minimal displacement between two central LOS vectors) or can be further realized by saliency / distinctiveness / prominence, planarity measure, aspect ratio proximity (two axes) or other metrics. "Match" can also be defined as maximal score in Generalized Proximity Criterion, as defined in patent application 211374. The output of the process is group K={k0,kl..kn} of images which can be used to be matched to the image (410)

Interlink the image to images in K:

Apply some sort of image matching technique between subset of K, each image in its entirety and/or fragments from which. Matching can be using legacy image matching techniques, with or without extending and sequencing matching algorithms such as Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), Maximally Stable Extremal Regions (MSER) and others, with or without utilizing outlier elimination methodologies such as RANSAC, together with least-squares based outlier rejection algorithms from Robust Estimation family. Improve geolocation via Free-Net Adjustment (420).

Using general methods of "ground to image" transformation (such as narrowing the error margins between line of sight to the target using Newton-Raphson method), projecting said objects onto the said image (440), said objects being accurately projected onto the new image. Optional intermediate step can be used before, after, or in parallel to the previous step. The step 430, includes storing the new, interlinked image into the VGIC, for future use, by same source (as in video) or by a different source from similar location. Referring to Figure 3, further explaining the novelty of the invention. Demonstrated are two coordinate frames: Real World (400) and Virtual (410). The Virtual coordinate system has an unknown 6 degree displacement (420) to the Real World. The displacement can have a constant bias locally changing over space and time. Georeferencing a cluster can assure that if using an image which was georeferenced to the virtual coordinate system, each object which was conceived in said virtual coordinate systems, will be accurately projected onto said image frame. This is a known property to a person skilled in the art.

By utilizing geolocation techniques on georeferenced images and sometimes georeferenced 3D surfaces, one yields georeferenced entities that can be accurately projected on any image in said cluster regardless the local displacement.

The novelty of the invention is therefore the fact that for accurate augmented reality applications, in which precise placement of geolocated objects onto the image frame is needed, can be done by utilizing georeferencing techniques on background imagery, georeferencing new image to the background imagery, and by projecting objects that originated from the background imagery.

Referring Figure 4, which to an exemplar application, an augmented reality (AR)tourism application. It follows the process of Figure 2, but has other features, needed for a complete experience. It demonstrates how applications may use the framework described in figure 1 and 2 in order to achieve a precise AR experience. It also follows the framework of a one-time preprocess (500) and real time (600).

Let a tourist AR application give the user tourist information, precisely attached on their surroundings, including historic data, overlay, navigation route and so forth. In order to achieve that, the following steps can be done: In the proposed tour, one can gather imagery with initial geolocation. This for example can be done by capturing dense imagery in a high quality camera and a geo tagger (510).

The imagery will be interlinked (as described in figure 1 and 2), indexed, and stored in an accessible server (520).

A tour editor will identify places of interest, geolocate their location from the collected imagery itself (530), and attach relevant data (for example, and audio explanation) (540). All data regarding places of interest will be stored in an accessible server. One or plurality of users start the tour, and load a mobile application (app) on a handheld or wearable device. They record imagery and initial localization metadata (location and orientation) (610).

The app sends relevant data to the server via internet. This can include the image, portion of the image, descriptors for image matching, and localization metadata, or any other relevant information (620). This can also be done locally.

The server finds candidates for matching from the imagery found in step 510 (630), and performs interlinking (640). This can also be done locally.

The server or the app accurately places relevant objects, which were created in step 540, to the image (650). The user naturally moves their hand or head, supporting the device. Standard video tracking tools can be utilized in order to keep track of the user's movement, relative to the frame sent in step 620, as long as the user does not change their location (660).

When the user changes their location above a predefined displacement (670), a new image reference will be sent and the process repeats from step 610. If not, video tracking is sufficient.

The user can interact with the information - ask for more information, browse between layers, and so forth (670). The user can also create objects and attach additional information, get and share information over social networks, etc. Geolocation can be achieved, since the image is interlinked and other images can be utilized to perform ray intersection. This may prove to be a difficult process. The whole process can be replicated for an arbitrary number of sites.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or an apparatus. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit," "module" or "system."

The aforementioned flowchart and block diagrams illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In the above description, an embodiment is an example or implementation of the inventions. The various appearances of "one embodiment," "an embodiment" or "some embodiments" do not necessarily all refer to the same embodiments. Although various features of the invention may be described in the context of a single embodiment, the features may also be provided separately or in any suitable combination. Conversely, although the invention may be described herein in the context of separate embodiments for clarity, the invention may also be implemented in a single embodiment.

Reference in the specification to "some embodiments", "an embodiment", "one embodiment" or "other embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least some embodiments, but not necessarily all embodiments, of the inventions. It will further be recognized that the aspects of the invention described hereinabove may be combined or otherwise coexist in embodiments of the invention.

It is to be understood that the phraseology and terminology employed herein is not to be construed as limiting and are for descriptive purpose only.

The principles and uses of the teachings of the present invention may be better understood with reference to the accompanying description, figures and examples.

It is to be understood that the details set forth herein do not construe a limitation to an application of the invention. Furthermore, it is to be understood that the invention can be carried out or practiced in various ways and that the invention can be implemented in embodiments other than the ones outlined in the description above.

It is to be understood that the terms "including", "comprising", "consisting" and grammatical variants thereof do not preclude the addition of one or more components, features, steps, or integers or groups thereof and that the terms are to be construed as specifying components, features, steps or integers.

If the specification or claims refer to "an additional" element, that does not preclude there being more than one of the additional element.

It is to be understood that where the claims or specification refer to "a" or "an" element, such reference is not be construed that there is only one of that element.

It is to be understood that where the specification states that a component, feature, structure, or characteristic "may", "might", "can" or "could" be included, that particular component, feature, structure, or characteristic is not required to be included. Where applicable, although state diagrams, flow diagrams or both may be used to describe embodiments, the invention is not limited to those diagrams or to the corresponding descriptions. For example, flow need not move through each illustrated box or state, or in exactly the same order as illustrated and described.

Methods of the present invention may be implemented by performing or completing manually, automatically, or a combination thereof, selected steps or tasks.

The term "method" may refer to manners, means, techniques and procedures for accomplishing a given task including, but not limited to, those manners, means, techniques and procedures either known to, or readily developed from known manners, means, techniques and procedures by practitioners of the art to which the invention belongs.

The descriptions, examples, methods and materials presented in the claims and the specification are not to be construed as limiting but rather as illustrative only.

Meanings of technical and scientific terms used herein are to be commonly understood as by one of ordinary skill in the art to which the invention belongs, unless otherwise defined.

The present invention may be implemented in the testing or practice with methods and materials equivalent or similar to those described herein.

While the invention has been described with respect to a limited number of embodiments, these should not be construed as limitations on the scope of the invention, but rather as exemplifications of some of the preferred embodiments. Other possible variations, modifications, and applications are also within the scope of the invention.

Claims

1. A method comprising:

obtaining a plurality of source images having a common region of interest;

interlinking said source images by at least one of: finding tie points between each source image to at least one other source image; and associating pixels in said source images with a common geolocation data; determining a geolocation of objects based on geo locating techniques, wherein the objects are accurately geo-referenced in a frame of reference of the interlinked plurality of source images;

interlinking a newly obtained image to said source images so that said objects are accurately geo-referenced onto the newly obtained image; and

applying said method to highly oblique and/or low angle line of sight (LOS) applications.

2. The method according to claim 1, wherein the highly oblique and/or low angle line of sight (LOS) applications comprise augmented reality (AR) applications.

3. The method according to claim 1 , wherein the interlinking is carried out using at least one of: Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and Maximally Stable Extremal Regions (MSER).

4. The method according to claim 1, further comprising projecting said objects onto the said newly obtained image, said objects being accurately projected onto said newly obtained image.

5. The method according to claim 1 , further comprising storing the new, interlinked image into a database, for future use, by same source or by a different source from similar location.

6. The method according to claim 1, wherein the geo-referencing in a frame of reference of the interlinked plurality of source images; assures that if using an image which was georeferenced to the virtual coordinate system, each object which was conceived in said virtual coordinate systems, will be accurately projected onto said image frame.

7. The method according to claim 1, wherein 3D location data and surfaces are georeferenced so that they can be accurately projected on any image in said plurality of source images, regardless of a local displacement.

8. The method according to claim 1, wherein the plurality of source images are updated with new data via crowdsourcing.

9. The method according to claim 1 , wherein the database is a photo-sphere.

10. The method according to claim 1 , wherein the newly obtained image is usable for updating the plurality of source images, wherein the update relates to at least one of: geographical data, temporal data.

11. The method according to claim 1 , further comprising utilizing the georeferenced at least one object contained in a newly obtained image for navigation purposes.

12. A system comprising:

a computer processor configured to:

obtain a plurality of source images having a common region of interest;

interlink said source images by at least one of: finding tie points between each source image to at least one other source image; and associating pixels in said source images with a common geolocation data;

determine a geolocation of objects based on geo locating techniques, wherein the objects are accurately geo-referenced in a frame of reference of the interlinked plurality of source images;

; and

interlink a newly obtained image to said source images so that said objects are accurately geo-referenced onto the newly obtained image.

13. The system according to claim 12, wherein the system is usable for line of sight (LOS) and/or augmented reality (AR) applications.

14. The system according to claim 12, wherein the interlinking is carried out by at least one of: Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and Maximally Stable Extremal Regions (MSER).

15. The system according to claim 12, wherein the processor is further configured to project said objects onto the said image, said objects being accurately projected onto the newly taken image.

16. The system according to claim 12, wherein the processor is further configured to store the new interlinked image on a database, for future use, by same source or by a different source from similar location.

17. The system according to claim 12, wherein the georeferencing of a cluster of objects assures that if using an image which was georeferenced to the virtual coordinate system, each object which was conceived in said virtual coordinate systems, will be accurately projected onto said image frame.

18. The system according to claim 12, wherein 3D location data and surfaces are georeferenced so that they can be accurately projected on any image in said cluster regardless of a local displacement.

19. The system according to claim 12, wherein the source images are updated with new data via crowdsourcing.

20. The system according to claim 12, wherein the database is a photo-sphere.

21. The system according to claim 12, wherein the newly taken image is usable for updating the database, wherein the update relates to at least one of: geographical data, temporal data.

22. The system according to claim 12, further comprising utilizing the georeferenced at least one object contained in a newly taken image for navigation purposes.

23. The system according to claim 12, wherein the clustered database further comprises meta data relating to the georeferenced at least one object.

24. The system according to claim 12, wherein the geo-referenced at least one object further includes data relating to at least one of: line of sight, depth. The system according to claim 12, wherein the geo-referenced at least object is usable to accurately project 3D objects.