US20050062845A1 - Video user interface system and method - Google Patents
Video user interface system and method Download PDFInfo
- Publication number
- US20050062845A1 US20050062845A1 US10/802,451 US80245104A US2005062845A1 US 20050062845 A1 US20050062845 A1 US 20050062845A1 US 80245104 A US80245104 A US 80245104A US 2005062845 A1 US2005062845 A1 US 2005062845A1
- Authority
- US
- United States
- Prior art keywords
- view
- data
- panoramic
- virtual
- virtual view
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19617—Surveillance camera constructional details
- G08B13/19626—Surveillance camera constructional details optical details, e.g. lenses, mirrors or multiple lenses
- G08B13/19628—Surveillance camera constructional details optical details, e.g. lenses, mirrors or multiple lenses of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19678—User interface
- G08B13/19682—Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19678—User interface
- G08B13/19689—Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19678—User interface
- G08B13/19691—Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/62—Control of parameters via user interfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
- H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
- H04N23/635—Region indicators; Field of view indicators
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
Definitions
- This invention generally relates to imaging systems, and more particularly, to a system and method for generating a graphical user interface from image data, such as video images.
- Conventional video surveillance systems allow users to view portions of monitored areas by either physically aiming a video camera at a desired portion or by generating a perspective corrected view of the desired portion from a wide angle (e.g., fisheye) image of the monitored area.
- a video surveillance system may display a close-up image of a person or an object (the displayed portion) in a room or a parking lot (the monitored area). Without an image of the entire room or lot and an indication of the area represented by the close-up image, it may be difficult or impossible to quickly determine the position of the person or object in the monitored room or lot. This is especially true for a crowded room or lot, where it may be difficult to distinguish one isolated portion from another.
- video surveillance systems typically do not allow an operator to see anything but the portions of monitored areas being displayed.
- the undisplayed portions may comprise a substantial portion of a monitored area, in which significant activity of interest may occur. For example, if a displayed image represents only 10% of a monitored parking lot, the other 90% may be unmonitored at any given time.
- the invention overcomes one or more of the problems in conventional video surveillance systems as set forth above, it should be apparent that the invention may have applicability to a wide array of imaging devices and methods, particularly those employing images of moving objects, regardless of the particular format of the image data.
- the invention avoids the drawbacks of the prior art by providing a graphical user interface that may, according to one aspect of the invention, include one or more panoramic views of a monitored area, one or more virtual views preferably corresponding to portions of the monitored area, and/or reference data such as a reference window overlaid in a panoramic view to identify a portion of the panoramic view corresponding to a virtual view.
- Panoramic views of the invention which may be derived from wide-angle lens image data, exhibit a determined (e.g., upright) orientation and substantially reduced (or no) wide-angle image distortion, thus providing a readily discernible view.
- Virtual views of the invention which may be based upon either transformed wide-angle lens image data or image data from a camera system without a wide-angle lens, also exhibit substantially reduced (or no) wide-angle image distortion, thus providing a readily discernible view.
- the virtual views and reference windows may respond to operator commands.
- a system for creating signals indicative of a graphical user interface from wide-angle image data, such as video images, corresponding to a monitored area.
- the system includes a buffer configured to receive wide-angle image data such as video images, corresponding to the monitored area.
- a processor operably coupled to the buffer is configured to transform wide angle image data received by the buffer into panoramic view data corresponding to at least one panoramic view of the monitored area, and into virtual view data corresponding to at least one virtual view of a portion of the at least one panoramic view.
- a method of creating a graphical user interface includes the steps of buffering wide-angle data corresponding to a wide-angle image, such as a video image, of a monitored area; transforming the buffered data into panoramic data for at least one panoramic view using a panoramic transformation; and transforming portions of the buffered data into virtual view data for at least one virtual view using a virtual view transformation.
- the graphical user interface may include a reference window overlaid in a panoramic view to identify a portion of the panoramic view represented by a virtual view.
- the system may include a dome camera system configured to provide image data for a virtual view according to commands. The dome camera image data may exhibit higher resolution than the wide-angle data.
- a system for creating signals indicative of a graphical user interface from wide-angle image data corresponding to a monitored area includes a means for storing wide-angle image data corresponding to a monitored area.
- the system also includes a means for processing and transforming data received from said storing means into panoramic view data corresponding to at least one panoramic view of the monitored area and into virtual view data corresponding to at least one virtual view of a portion of the at least one panoramic view.
- FIG. 1 is a block diagram that conceptually illustrates a graphical user interface constructed in accordance with the principles of the invention
- FIG. 2 is a block diagram that conceptually shows an imaging system for producing a graphical user interface in accordance with an exemplary embodiment of the invention
- FIG. 3 conceptually shows 0 (i.e., the pan rotation angle around the Z-axis) and (p (the tilt rotation angle around the X-axis) for a transformation in accordance with an exemplary implementation of the invention
- FIG. 4 conceptually shows image center ICxy based on image radii IRx and IRy in accordance with an exemplary implementation of the invention
- FIG. 5 is a high level flowchart conceptually illustrating steps of a methodology for creating a graphical user interface in accordance with an exemplary implementation of the invention.
- FIG. 6 is a high level flowchart conceptually illustrating steps of a methodology for creating a graphical user interface including using dome camera system positioning in accordance with an exemplary implementation of the invention.
- An exemplary system in accordance with the invention includes a processor configured to transform digital image data corresponding to the wide-angle images, such as video images, of a monitored area into data corresponding to one or more panoramic views of the monitored area.
- a user interface in accordance with an exemplary implementation of the invention is conceptually shown.
- Strips 110 and 160 which may be rectangular in shape as illustrated, correspond to transformed panoramic views.
- the transformed panoramic views 110 and 160 exhibit upright orientation and substantially reduced (or no) wide-angle image distortion, thus providing a readily discernible view of the monitored area. Examples of a suitable transformation carried out by the processor are discussed subsequently.
- the processor is also configured to provide data corresponding to one or more virtual views 130 and 140 , each of which is an undistorted (or substantially undistorted) view of a user-determined portion of the panoramic views 110 and 160 .
- Virtual view data may be obtained either by transformation of portions of the wide-angle video image data, or by positioning and focusing a camera having a normal lens (i.e., not a wide angle lens) or a telephoto lens to produce image data for the determined portion of the monitored area. Again, a suitable transformation to obtain the virtual view data is discussed subsequently.
- the processor may be further configured to provide reference data (e.g., windows—i.e., graphically generated boxes or other shapes overlaid in the panoramic views—boxes 120 and 150 as shown in FIG. 1 ) corresponding to the undistorted view of the determined portions of the panoramic views represented by the virtual views 130 and 140 .
- reference data e.g., windows—i.e., graphically generated boxes or other shapes overlaid in the panoramic views—boxes 120 and 150 as shown in FIG. 1
- the reference data 120 and 150 reveals what portions of the panoramic views are represented by the virtual views 130 and 140 .
- the corresponding reference window 120 or 150 overlaid on top of a panoramic view 110 and 160 moves in the panoramic view 110 or 160 , thus indicating the part of the panoramic view 110 and 160 represented by the virtual view 130 or 140 .
- the system is preferably configured to allow a user to select which virtual view 130 or 140 to currently control.
- the active (i.e., currently controlled) virtual view 130 or 140 may be distinguished from the inactive virtual view by an indicia. For example, a white frame may surround the active virtual window and a black frame may surround the inactive virtual view.
- the corresponding reference window 120 or 150 may also be highlighted in either black or white based on its associated virtual view's active/inactive status.
- the exemplary display screen may have a resolution of 486 by 720 pixels.
- the exemplary panoramic views, as shown in FIG. 1 are each 720 pixels wide and 120 pixels high.
- the exemplary virtual views, as shown in FIG. 1 may each be 360 pixels wide and 246 pixels high.
- Those skilled in the art will appreciate that the invention is not limited to the exemplary resolution or view sizes. Many other resolutions and sizes are feasible and come within the scope of the invention.
- the reference data may be controllable via a user input device, such as a joystick, cursors or other pointing device.
- a user may move and resize reference windows 120 and 150 to cover any portions of the panoramic views 110 and 160 .
- the portions of the panoramic views covered by the reference windows 120 and 150 correspond to or define the virtual views 130 and 140 .
- the system may also automatically determine the portions of the panoramic views 110 and 160 to cover, such as by one or more determined algorithms (e.g., move reference windows as a function of time) or in response to one or more signals (e.g., a motion detector, gate sensor, or alarm signal).
- Other manifestations of reference data such as numerical coordinates or cross hairs, may be used in lieu of or in addition to reference windows 120 and 150 .
- Panoramic views 110 and 160 and virtual views 130 and 140 may be displayed concurrently on a display monitor, thus providing a readily discernible view of the entire monitored area and an undistorted (or substantially undistorted) view of a portion of the monitored area.
- the reference data 120 and 150 may also be displayed with the panoramic views 110 and 160 , thus enabling a user to readily identify the portions of the panoramic views 110 and 160 that are represented by the virtual views 130 and 140 .
- panoramic views 110 and 160 may be provided without departing from the scope of the invention.
- the user interface components may be arranged as shown in FIG. 1 and discussed above, or differently, on one or more display monitors, or in one or more windowed portions of one or more display monitors, without departing from the scope of the invention.
- the interface may be configured to allow a user to control the display arrangement.
- a user may hide or move a panoramic view 110 or 160 , and/or a virtual view 130 or 140 and/or a reference window 120 or 150 ; or hide all virtual views 130 and 140 reference windows 120 and 150 ; or opt to display only a selected panoramic view 110 or 160 or virtual view 130 or 140 .
- image sources may supply image data for use by a system in accordance with the invention.
- one or more imagers such as digital or analog video cameras may be used to supply video data, which may be in any known format.
- video broadly refers to images, such as images of stationary and/or moving objects, which may or may not be produced using signals in standard formats, such as NTSC signals.
- the system of the invention may operate in real time, processing video data as it is produced using the video imagers and without appreciably delaying output to a display monitor or recorder.
- saved analog or digital video signals or data may serve as the source of images.
- the video imaging system 200 includes a video camera 210 equipped with a wide-angle lens 205 , such as a fisheye lens.
- a wide-angle lens 205 such as a fisheye lens.
- Various wide-angle lenses are available, many of which, for example, have an angular field of view of approximately 50 degrees or more.
- the wide-angle lens may preferably be suitable for producing an image of an entire area to be monitored (e.g., portions of a room excluding the ceiling) when the wide-angle lens is positioned at a determined point relative to the monitored area (e.g., on a ceiling of the room, at or near the center of the ceiling, pointing downwardly).
- a fisheye lens which is a type of wide-angle lens, and which may have a field of view of 180 degrees
- the camera 210 may be a digital camera such as a camera that employs a charge coupled device (CCD) array, a Time Delay Integration (TDI) CCD array, complementary metal oxide semiconductor (CMOS) image sensors or other sensors or devices for electronic image production.
- the camera 210 may be an analog camera that produces analog (e.g. NTSC or PAL) output signals.
- the wide-angle lens 205 is aimed at the area to be monitored.
- the lens 205 may be a fisheye lens or another type of wide-angle lens that provides a 180° field of view, or a wider or narrower field of view.
- the position and orientation may generate a fisheye view of the entire monitored area.
- a video decoder 215 operably coupled to the camera translates analog signals from the camera into digital video data.
- the decoder 215 may include a clock circuit (e.g., a 27 MHz video clock circuit) suitable for synchronization and processing of video image data streams.
- Clock signals 217 from the clock circuit may be transmitted to processor 225 .
- a suitable decoder 215 may include commercially available decoders, such as an SAA7113H 9-bit video input processor available from Phillips Electronics N.V.
- the SAA7113H 9-bit video input processor includes an anti-aliasing filter, an automatic clamp and gain control, a clock generation circuit, a digital multi-standard decoder, and a brightness, contrast and saturation control circuit. It is able to decode Phase Alternation Line (PAL), French Sequential Couleur Electro Memoire (SECAM) and American National Television Systems Committee (NTSC) standard signals into CCIR-601 (International Radio Consultative Committee, now the International Telecommunication Union Radiocommunication Sector, standard for encoding analog video signals in digital form) compatible color component data values.
- the SAA7113H accepts as analog inputs CVBS (i.e., composite Chroma Video Blanking & Sync inputs) or S-video (Y/C).
- Digital video data from decoder 215 may be stored in an image buffer 220 .
- the image buffer 220 provides a means for storing wide-angle image data corresponding to a monitored area.
- the image buffer 220 may be comprised of volatile or non-volatile memory or other devices configured to store video image data.
- the data may be stored temporarily, for example, until no longer needed for transformation by the image data processor 225 , as described below, until new video image data is required to be stored in the occupied portion of the buffer, or until some other event occurs or a period of time elapses.
- the buffer 220 may be sized to store image data corresponding to an entire field or frame of video, or more or less data.
- One or more user input modules 290 may also be provided to generate user commands.
- the module may include an input device 275 such as a keypad, digitizer, joystick, microphone and voice recognition module, or some other device configured to enable a user to enter commands.
- the input device 275 may be operably coupled to an input processor 280 and input memory 285 configured to produce user command data 292 corresponding to user input commands.
- the input processor 280 may be implemented as a special purpose computer; a programmed general purpose computer; a programmed microprocessor, microcontroller or programmable read-only memory and peripheral integrated circuit elements; an application specific integrated circuit (ASIC) or other integrated circuits; a digital signal processor; a hardwired electronic or logic circuit such as a discrete element circuit; a programmable logic device such as a field programmable gate array (FPGA), or the like.
- ASIC application specific integrated circuit
- FPGA field programmable gate array
- the input memory 285 may include volatile or non-volatile memory such as, for example, static random access memory (SRAM), dynamic random access memory (DRAM), fast page mode dynamic random access memory (FPM DRAM), extended data-out dynamic random access memory (EDO DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic RAM (DDR SDRAM), electronically erasable programmable read only memory (EEPROM) such as flash memory with or without a controller, hard disk enabled virtual memory, and/or other data storage devices that may be operably coupled to the input processor 280 .
- SRAM static random access memory
- DRAM dynamic random access memory
- FPM DRAM fast page mode dynamic random access memory
- EEO DRAM extended data-out dynamic random access memory
- SDRAM synchronous dynamic random access memory
- DDR SDRAM double data rate synchronous dynamic RAM
- EEPROM electronically erasable programmable read only memory
- flash memory with or without a controller, hard disk enabled virtual memory, and/or other data storage devices that may be operably coupled to the
- An image data processor 225 may receive buffered image data 222 , clock signals 217 and user command data 292 to produce panoramic view data, virtual view data and reference data.
- the image data processor 225 thus provides a means for processing and transforming data received from the storing means (i.e., the image buffer 220 ) into panoramic view data corresponding to at least one panoramic view of the monitored area and into virtual view data corresponding to at least one virtual view of a portion of the at least one panoramic view.
- the processor 225 may be implemented as a special purpose computer; a programmed general purpose computer; a programmed microprocessor, microcontroller or programmable read-only memory and peripheral integrated circuit elements; an application specific integrated circuit (ASIC) or other integrated circuits; a digital signal processor; a hardwired electronic or logic circuit such as a discrete element circuit; a programmable logic device such as a field programmable gate array (FPGA), or the like.
- a preferred implementation uses an FPGA or an ASIC as the processor 225 . When a system design is considered stable, an FPGA could be replaced with an ASIC to reduce system cost.
- the image data processor 225 may also be operably coupled to memory 230 , which may, for example, be non-volatile memory in the form of electronically erasable programmable read only memory (EEPROM), such as flash memory, with or without a controller; a solid-state floppy-disk card (SSFDC) such as SmartMedia® memory commercially available from Kabushiki Kaisha Toshiba, Toshiba Corporation, and Toshiba America Electronic Components, Inc.; CompactFlash® memory by SanDisk Corporation; hard disk enabled virtual memory; and/or other non-volatile storage devices.
- memory 230 may be used to facilitate processor computations by storing one or more tables of values for frequently calculated functions, especially functions that might otherwise consume significant processor 225 resources.
- memory 230 may store a table of values, such as cosine, sine, 1/cosine, 1/sine, or other values based on trigonometric or other types of functions used in an applicable transformation algorithm, as discussed more fully below.
- the indices for the table may be angular values in degrees, radians, other types of measurements, or some other possible value for a variable.
- the table entries may be pre-calculated values based on the corresponding indices. Trigonometric functions and complex floating point calculations can otherwise be a time and resource consuming part of processor 225 calculations. The use of pre-determined stored values conserves processor resources and may lead to an appreciable increase in overall computation speed.
- the image data processor 225 mathematically transforms wide angle video image data in the image buffer 220 into panoramic views using a panoramic view transformation algorithm.
- the image data processor 225 may also transform wide angle video image data in the image buffer 220 into virtual views using a virtual view transformation algorithm, as discussed more fully below.
- the processor 225 Upon transformation, the processor 225 sends transformed video data to an encoder 235 .
- the encoder converts the digital video data into output signals compatible with an output device such as a display monitor 240 and/or video recorder 245 .
- the encoder may be any suitable commercially available decoder, such as an SAA7121H digital video encoder by Phillips Electronics, N.V.
- the SAA7121H digital video encoder circuit accepts CCIR compatible YUV data with 720 active pixels per line and encodes the digital YUV video data to NTSC, PAL, CVBS or S-video signals.
- Output from the encoder 235 is sent to an output device, such as a standard display monitor 240 and/or video recorder 245 .
- the display monitor may be any device configured to visually display images based on electronic video signals output from the encoder 235 .
- the recorder may be any device configured to record video signals from the encoder 235 on removable or non-removable storage media such as, for example, a magnetic tape, diskette, CD-ROM, DVD, hard disk, memory (e.g., nonvolatile EEPROM) or the like. If both a display monitor 240 and recorder 245 are used, the devices may be configured in parallel or in series (e.g., without output from the video recorder being sent to a display monitor).
- the image data processor 225 may generate virtual view data from video image data produced by a one or more additional camera systems 250 .
- An additional camera system may include an analog or digital camera 260 with a normal lens 255 (i.e., not a wide angle lens) aimed at a determined portion of the monitored area.
- the lens 255 may have a narrower Field of View (FOV), e.g., 45°, which would provide higher resolution than wide-angle lens 205 , thus providing a more detailed video image for a virtual view.
- FOV Field of View
- the additional camera enables switching from a low resolution virtual view derived from wide-angle lens image data to a high resolution virtual view, with high zoom factor capability, produced using the additional camera system. This gives the user the ability to isolate small details of a monitored area.
- the additional camera system 250 may also include a video decoder 265 that is operably coupled to the camera and configured to translate analog signals from the camera into digital video data.
- the decoder 265 may also include a clock circuit (e.g., a 27 MHz video clock circuit) suitable for synchronization and processing of video image data streams.
- Clock signals 267 may be transmitted to processor 225 .
- the additional camera system 250 may further include an image buffer 270 .
- the image buffer 270 may be comprised of volatile or non-volatile memory or other devices configured to temporarily store video image data, such as, for example, static random access memory (SRAM), dynamic random access memory (DRAM), fast page mode dynamic random access memory (FPM DRAM), extended data-out dynamic random access memory (EDO DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic RAM (DDR SDRAM), Rambus dynamic random access memory (RDRAM), multiport dynamic random access memory (MPDRAM), synchronous graphics RAM (SGRAM), electronically erasable programmable read only memory (EEPROM), hard disk enabled virtual memory and/or other data storage device.
- SRAM static random access memory
- DRAM dynamic random access memory
- FPM DRAM fast page mode dynamic random access memory
- EEO DRAM extended data-out dynamic random access memory
- SDRAM synchronous dynamic random access memory
- DDR SDRAM double data rate synchronous dynamic RAM
- RDRAM Rambus dynamic random access memory
- the buffer 270 may be sized to store image data corresponding to an entire field or frame of video, or more or less data. Buffered video data 272 may be transmitted to processor 225 . That data may then be processed and encoded 235 for display in either virtual view 130 or 140 .
- the additional camera system 250 may be configured as a programmable dome camera system. Dome cameras can typically rotate (i.e., pan) 360 degrees, and tilt and zoom according to control signals.
- the dome camera 260 may be coupled to a processor programmed to produce control signals causing the camera 260 to aim at particular portions of a monitored area in particular patterns.
- processor 225 may produce control data 294 , based upon user command data 292 from user input module 290 , to control positioning and movement of the dome camera.
- a control interface (not shown) may also be operably coupled to dome camera 260 and processor 225 , as necessary to convert control data output from the processor 225 into signals for controlling dome camera 260 position, movement and zooming.
- panoramic views 110 and 160 and virtual views 130 and 140 may be produced by transforming wide-angle image data (including fisheye image data) into 2-dimensional space, without distortion (or with substantially reduced distortion).
- Many such transformation processes also referred to as dewarping, mapping, remapping, planar projection, conversion and perspective correction
- panoramic views 110 and 160 and virtual views 130 and 140 with linear perspective geometry from wide-angle image data (including fisheye image data) are known in the art and can be adapted, in accordance with the teachings herein, to produce panoramic views 110 and 160 and virtual views 130 and 140 with linear perspective geometry from wide-angle image data (including fisheye image data). While the invention is not limited to any specific transformation processes, an exemplary panoramic view transformation and an exemplary virtual view transformation are described more fully below for use with a system of the invention employing fisheye lens image data.
- the panoramic view transformation process involves mathematically transforming wide angle video image data into 2-dimensional space for producing panoramic views.
- the exemplary transformation entails determining, for each output window pixel coordinate, an address within the image buffer 220 for pixel data to be encoded and output to a display 240 and/or recorder 245 .
- this address calculation is timed such that the calculated address for each output pixel is ready just in time to feed the output encoder. This implementation advantageously eliminates the need for an output image buffer.
- two panoramic views 110 and 160 are produced.
- One panoramic view 110 corresponds to half of the monitored area (e.g., a 0-180 degree panoramic view strip).
- the other panoramic view corresponds to the other half of the monitored area (e.g., a 181-360° panoramic view strip).
- the second panoramic view is offset from the first panoramic view by half of the field of view.
- the points dividing one half from the other half may be pre-programmed, user configurable or user controllable.
- the two panoramic views combined provide a full (e.g., a 360°) view of the monitored area.
- the exemplary panoramic views are 720 pixels wide and 120 pixels high.
- Output window pixel coordinates range from ⁇ 360 to +360 for Wx and from ⁇ 60 to +60 for Wy.
- the Wx and Wy coordinates are derived from the output view column and line pixel counters, and together with the desired Zoom factor, are transformed into ⁇ (i.e., the pan rotation angle around the Z-axis) and ⁇ (the tilt rotation angle around the X-axis), as conceptually illustrated in FIG. 3 .
- This information is then converted to an address in the input capture buffer.
- the data from this address is output to the encoder for display.
- the variables Pan, Tilt and Zoom may be programmed or commanded via user input module 290 .
- the horizontal component of the panoramic view (starting with output window upper left at ⁇ 360,+120) is converted from pixel units to degrees(SFx). Each output pixel equates to 0.25 degrees (180°/720 pixels). The programmable Pan value is added to each pixel value.
- the second panoramic view 160 is offset from the first 110 by +180 degrees.
- Ix (Radius ⁇ cos( ⁇ ) ⁇ IRx )+ ICxj Equation 8
- Iy (Radius ⁇ sin( ⁇ ) ⁇ IRy )+ ICy Equation 9
- the image buffer 220 is preferably addressed as a 1024 byte by 1024 byte array.
- PixelData may then be sent to the encoder 235 for output to the output device such as a display monitor 240 and or a recorder 245 .
- Pixel Data for each pixel comprising the panoramic views 110 and 160 may be determined in a similar manner.
- Each virtual view window 130 and 140 provides a perspective corrected view (e.g., a view with no, or substantially reduced, fisheye distortion) of any portion of a panoramic view 110 or 160 .
- the virtual view windows are preferably 360 pixels wide and 240 pixels high. The center of the window is defined as 0,0.
- Wx values range from ⁇ 180 to +180 and Wy values range from +120 down to ⁇ 120.
- the Wx and Wy coordinates are taken from the output window line and pixel counters and, together with the desired Pan, Tilt and Zoom Scale, are transformed into ⁇ , ⁇ , and Radius using the virtual view transformation algorithm.
- the processor determines which algorithm (i.e., the virtual view transformation algorithm or the panoramic view transformation algorithm) to use based on where the counters indicate it is in the output video frame.
- ⁇ is defined as the Pan angle around the Z-Axis, where the positive Y direction, as shown in FIG. 3 , is 0 degrees.
- ⁇ is the Tilt angle around the X-Axis where the positive Z direction, pointing straight down as shown in FIG. 3 , is 0 degrees.
- Z is the perpendicular distance from the X-Y plane down to the surface of a unity radius hemispheric fisheye model.
- the virtual view transformation algorithm entails converting the X and Y output window coordinates to an equivalent proportion of a unity radius hemisphere model using a current Zoom factor and a measured radius-X and radius-Y from an input fisheye circular image. These adjusted values are used to calculate a corresponding Z value.
- the X, Y and Z coordinates are rotated around the unity radius hemisphere model using the commanded Pan and Tilt from the user input module 290 .
- the rotated X and Y values are then rescaled back to pixels and lines, based on measured radius-X and radius-Y from the input fisheye circular image.
- the rescaled X and Y are then converted to an address in the input capture buffer. Data from this address is then output to the encoder for display.
- the variables Pan, Tilt, and Zoom are the angle of interest commanded by the user input module 290 .
- the vertical and horizontal components of the virtual view window (center at origin 0,0) are converted from pixel units to unit values from 0 to less than one by dividing pixel number by the respective image radius.
- One is the maximum length from the center of the virtual output image, to the edge of the image.
- the desired radius squared is calculated from the vertical and horizontal components using the Pythagorean theorem.
- Wr 2 Wx 2 +Wy 2 Equation 15
- a rotation of the desired window to a desired spot on the image can be performed using the Pan and Tilt values to simulate a movable dome.
- the image buffer 220 may be addressed as a 1024 byte by 1024 byte array.
- PixelData may then be sent to the encoder 235 for output to the display window.
- the panoramic views 110 and 160 provided by the invention provide context for the virtual views 130 and 140 .
- a user never loses sight as to what is happening in the monitored area outside a virtual view, even as the user zooms in on a specific event or object.
- the user does not have to keep changing video sources to view areas outside the virtual views 130 and 140 .
- the reference windows 120 and 150 provide a visual indication of the portion of a panoramic view represented by a virtual view.
- the ability to switch to the additional camera to produce a high resolution, high zoom factor, virtual view gives the user the ability to resolve small details of a scene.
- FIG. 5 a high level flow chart for a process of creating a video user interface in accordance with an exemplary implementation of the invention is shown. The steps may be implemented using a system with a programmed processor as described above. FIG. 5 may equally represent a high level block diagram of an exemplary system constructed in accordance with the invention, implementing the steps thereof.
- wide-angle data (e.g., fisheye lens image data) is buffered.
- the data may be supplied from a video camera system with a decoder or from a source of pre-recorded data.
- pan, tilt and zoom angles are determined to specify regions of interest in the panoramic views for display in virtual views.
- Pan, tilt and zoom angles may be supplied by a programmed algorithm, a table or a user input module.
- values other than pan, tilt and zoom e.g., Cartesian coordinate values
- reference data for reference windows are determined.
- the reference data preferably corresponds to rectangular windows overlaying the virtual view regions of interest within the panoramic views.
- the rectangular windows move according to a programmed algorithm, a table or user commands from a user input module. As the rectangular windows move, the virtual views correspondingly change.
- step 540 buffered wide-angle data are transformed using a panoramic transformation algorithm for panoramic view data, and using a virtual view transformation algorithm for virtual view data.
- the transformed data and reference data are encoded for output.
- the transformed data defines any other virtual views and the panoramic views.
- the reference data defines the reference windows.
- the encoded output may be sent to a display monitor and/or to a recorder.
- reference data may be determined before, after or during transformation of the buffered data.
- a pre-programmed pan, tilt and zoom may be determined before wide-angle data is initially buffered.
- FIG. 6 a high level flow chart for another process of creating video user interface in accordance with an exemplary implementation of the invention is shown. As with the process illustrated by the flowchart of FIG. 5 , the steps may be implemented using a system with a programmed processor as described above. FIG. 6 may equally represent a high level block diagram of an exemplary system constructed in accordance with the invention, implementing the steps thereof.
- wide-angle data (e.g., fisheye lens image data) is buffered.
- the data may be supplied from a video camera system with a decoder or from a source of pre-recorded data.
- pan, tilt and zoom angles are determined to specify regions of interest in the panoramic views for display in virtual views.
- Pan, tilt and zoom angles may be supplied by a programmed algorithm, a table or a user input module.
- values other than pan, tilt and zoom e.g., Cartesian coordinate values
- reference data for reference windows are determined.
- the reference data preferably corresponds to rectangular windows overlaying the virtual view regions of interest within the panoramic views.
- the rectangular windows move according to a programmed algorithm, a table or user commands from a user input module. As the rectangular windows move, the virtual views correspondingly change.
- step 640 position data is communicated to a dome camera system.
- the communicated position data will cause the dome camera to aim at and zoom to the region of the monitored area corresponding to the determined pan, tilt and zoom values.
- dome camera data is buffered.
- the buffered data may be encoded, without transformation, for output to a display monitor and/or to a recorder as a virtual view.
- step 660 buffered wide-angle data are transformed using a panoramic transformation algorithm for panoramic view data, and using a virtual view transformation algorithm for virtual view data.
- step 670 buffered dome camera data are obtained for encoding.
- the dome camera data may be obtained form a dome camera buffer by a processor and communicated with transformed wide-angle data for encoding.
- the transformed data, dome camera data and reference data are encoded for output.
- the dome camera data defines one virtual view.
- the transformed data defines any other virtual views and the panoramic views.
- the reference data defines the reference windows.
- the encoded output may be sent to a display monitor and/or to a recorder.
- the reference data may be determined before, after or during transformation of the buffered data.
- a pre-programmed pan, tilt and zoom may be determined before the wide-angle data is initially buffered.
- dome position data may be sent before reference data is determined.
- dome data may be buffered after transformation of wide-angle data.
- dome camera data may be obtained by the processor before completion of transformation.
Abstract
Description
- The present application claims the benefit of the filing date of U.S. Provisional Application Ser. No. 60/502,728, filed Sep. 12, 2003, entitled “Video User Interface System and Method,” the entire teachings of which are incorporated herein by reference. This application is related to co-pending U.S. Application Serial No. 10/______, filed Mar. 17, 2004, entitled “Imaging System and Method for Displaying and/or Recording Undistorted Wide-Angle Image Data.”
- 1. Field of the Invention
- This invention generally relates to imaging systems, and more particularly, to a system and method for generating a graphical user interface from image data, such as video images.
- 2. Background Description
- Conventional video surveillance systems allow users to view portions of monitored areas by either physically aiming a video camera at a desired portion or by generating a perspective corrected view of the desired portion from a wide angle (e.g., fisheye) image of the monitored area. One shortcoming of such conventional systems is that they do not conveniently enable a user to discern the part of the monitored area represented by the displayed portion. For example, a video surveillance system may display a close-up image of a person or an object (the displayed portion) in a room or a parking lot (the monitored area). Without an image of the entire room or lot and an indication of the area represented by the close-up image, it may be difficult or impossible to quickly determine the position of the person or object in the monitored room or lot. This is especially true for a crowded room or lot, where it may be difficult to distinguish one isolated portion from another.
- Another shortcoming is that video surveillance systems typically do not allow an operator to see anything but the portions of monitored areas being displayed. The undisplayed portions may comprise a substantial portion of a monitored area, in which significant activity of interest may occur. For example, if a displayed image represents only 10% of a monitored parking lot, the other 90% may be unmonitored at any given time.
- Yet another shortcoming is that high zoom factor images mathematically derived from wide-angle lens image data tend to exhibit low resolution. The greater the zoom factor, the lower the resolution. This impairs an operator's ability to distinguish fine details (e.g., facial features or automobile license plate numbers) in a monitored area.
- While the invention overcomes one or more of the problems in conventional video surveillance systems as set forth above, it should be apparent that the invention may have applicability to a wide array of imaging devices and methods, particularly those employing images of moving objects, regardless of the particular format of the image data.
- The invention avoids the drawbacks of the prior art by providing a graphical user interface that may, according to one aspect of the invention, include one or more panoramic views of a monitored area, one or more virtual views preferably corresponding to portions of the monitored area, and/or reference data such as a reference window overlaid in a panoramic view to identify a portion of the panoramic view corresponding to a virtual view. Panoramic views of the invention, which may be derived from wide-angle lens image data, exhibit a determined (e.g., upright) orientation and substantially reduced (or no) wide-angle image distortion, thus providing a readily discernible view. Virtual views of the invention, which may be based upon either transformed wide-angle lens image data or image data from a camera system without a wide-angle lens, also exhibit substantially reduced (or no) wide-angle image distortion, thus providing a readily discernible view. The virtual views and reference windows may respond to operator commands.
- In particular, according to a first aspect of the invention, a system is provided for creating signals indicative of a graphical user interface from wide-angle image data, such as video images, corresponding to a monitored area. The system includes a buffer configured to receive wide-angle image data such as video images, corresponding to the monitored area. A processor operably coupled to the buffer is configured to transform wide angle image data received by the buffer into panoramic view data corresponding to at least one panoramic view of the monitored area, and into virtual view data corresponding to at least one virtual view of a portion of the at least one panoramic view.
- In another aspect of the invention, a method of creating a graphical user interface is provided. The method includes the steps of buffering wide-angle data corresponding to a wide-angle image, such as a video image, of a monitored area; transforming the buffered data into panoramic data for at least one panoramic view using a panoramic transformation; and transforming portions of the buffered data into virtual view data for at least one virtual view using a virtual view transformation.
- Additionally, the graphical user interface may include a reference window overlaid in a panoramic view to identify a portion of the panoramic view represented by a virtual view. As another option, the system may include a dome camera system configured to provide image data for a virtual view according to commands. The dome camera image data may exhibit higher resolution than the wide-angle data.
- In yet another aspect of the invention, a system for creating signals indicative of a graphical user interface from wide-angle image data corresponding to a monitored area is provided. The system includes a means for storing wide-angle image data corresponding to a monitored area. The system also includes a means for processing and transforming data received from said storing means into panoramic view data corresponding to at least one panoramic view of the monitored area and into virtual view data corresponding to at least one virtual view of a portion of the at least one panoramic view.
- Additional features, advantages, and embodiments of the invention may be set forth or apparent from consideration of the following detailed description, drawings, and claims. Moreover, it is to be understood that both the foregoing summary of the invention and the following detailed description are exemplary and intended to provide further explanation without limiting the scope of the invention as claimed.
- The accompanying drawings, which are included to provide a further understanding of the invention, are incorporated in and constitute a part of this specification, illustrate preferred embodiments of the invention and together with the detail description serve to explain the principles of the invention. In the drawings:
-
FIG. 1 is a block diagram that conceptually illustrates a graphical user interface constructed in accordance with the principles of the invention; -
FIG. 2 is a block diagram that conceptually shows an imaging system for producing a graphical user interface in accordance with an exemplary embodiment of the invention; -
FIG. 3 conceptually shows 0 (i.e., the pan rotation angle around the Z-axis) and (p (the tilt rotation angle around the X-axis) for a transformation in accordance with an exemplary implementation of the invention; -
FIG. 4 conceptually shows image center ICxy based on image radii IRx and IRy in accordance with an exemplary implementation of the invention; -
FIG. 5 is a high level flowchart conceptually illustrating steps of a methodology for creating a graphical user interface in accordance with an exemplary implementation of the invention; and -
FIG. 6 is a high level flowchart conceptually illustrating steps of a methodology for creating a graphical user interface including using dome camera system positioning in accordance with an exemplary implementation of the invention. - An exemplary system in accordance with the invention includes a processor configured to transform digital image data corresponding to the wide-angle images, such as video images, of a monitored area into data corresponding to one or more panoramic views of the monitored area. Referring to
FIG. 1 , a user interface in accordance with an exemplary implementation of the invention is conceptually shown.Strips panoramic views - The processor is also configured to provide data corresponding to one or more
virtual views panoramic views - The processor may be further configured to provide reference data (e.g., windows—i.e., graphically generated boxes or other shapes overlaid in the panoramic views—
boxes FIG. 1 ) corresponding to the undistorted view of the determined portions of the panoramic views represented by thevirtual views reference data virtual views - As an operator pans, tilts or zooms either of the
virtual views panoramic views corresponding reference window panoramic view panoramic view panoramic view virtual view - The system is preferably configured to allow a user to select which
virtual view virtual view corresponding reference window - The exemplary display screen may have a resolution of 486 by 720 pixels. The exemplary panoramic views, as shown in
FIG. 1 , are each 720 pixels wide and 120 pixels high. The exemplary virtual views, as shown inFIG. 1 , may each be 360 pixels wide and 246 pixels high. Those skilled in the art will appreciate that the invention is not limited to the exemplary resolution or view sizes. Many other resolutions and sizes are feasible and come within the scope of the invention. - The reference data (e.g.,
reference windows 120 and 150) may be controllable via a user input device, such as a joystick, cursors or other pointing device. Thus, for example, a user may move and resizereference windows panoramic views reference windows virtual views panoramic views reference windows -
Panoramic views virtual views reference data panoramic views panoramic views virtual views - Those skilled in the art will appreciate that the invention is not limited to the exemplary number, size, shape or arrangement of panoramic views and/or reference windows shown in
FIG. 1 and discussed above. For example, one or morepanoramic views virtual views reference windows FIG. 1 and discussed above, or differently, on one or more display monitors, or in one or more windowed portions of one or more display monitors, without departing from the scope of the invention. Furthermore, the interface may be configured to allow a user to control the display arrangement. For example, a user may hide or move apanoramic view virtual view reference window virtual views reference windows panoramic view virtual view - Various image sources may supply image data for use by a system in accordance with the invention. By way of example and not limitation, one or more imagers such as digital or analog video cameras may be used to supply video data, which may be in any known format. Thus, unless otherwise apparent from the context, as used herein “video” broadly refers to images, such as images of stationary and/or moving objects, which may or may not be produced using signals in standard formats, such as NTSC signals. The system of the invention may operate in real time, processing video data as it is produced using the video imagers and without appreciably delaying output to a display monitor or recorder. Alternatively, saved analog or digital video signals or data may serve as the source of images.
- Referring now to
FIG. 2 , a high-level block diagram of avideo imaging system 200 for use with a system in accordance with an exemplary implementation of the invention is shown. Thevideo imaging system 200 includes avideo camera 210 equipped with a wide-angle lens 205, such as a fisheye lens. Various wide-angle lenses are available, many of which, for example, have an angular field of view of approximately 50 degrees or more. The wide-angle lens may preferably be suitable for producing an image of an entire area to be monitored (e.g., portions of a room excluding the ceiling) when the wide-angle lens is positioned at a determined point relative to the monitored area (e.g., on a ceiling of the room, at or near the center of the ceiling, pointing downwardly). By way of example and not limitation, a fisheye lens, which is a type of wide-angle lens, and which may have a field of view of 180 degrees, may be utilized. Thecamera 210 may be a digital camera such as a camera that employs a charge coupled device (CCD) array, a Time Delay Integration (TDI) CCD array, complementary metal oxide semiconductor (CMOS) image sensors or other sensors or devices for electronic image production. Alternatively, thecamera 210 may be an analog camera that produces analog (e.g. NTSC or PAL) output signals. - The wide-
angle lens 205 is aimed at the area to be monitored. Thelens 205 may be a fisheye lens or another type of wide-angle lens that provides a 180° field of view, or a wider or narrower field of view. The position and orientation may generate a fisheye view of the entire monitored area. - A
video decoder 215 operably coupled to the camera translates analog signals from the camera into digital video data. Thedecoder 215 may include a clock circuit (e.g., a 27 MHz video clock circuit) suitable for synchronization and processing of video image data streams. Clock signals 217 from the clock circuit may be transmitted toprocessor 225. - By way of example and not limitation, a
suitable decoder 215 may include commercially available decoders, such as an SAA7113H 9-bit video input processor available from Phillips Electronics N.V. The SAA7113H 9-bit video input processor includes an anti-aliasing filter, an automatic clamp and gain control, a clock generation circuit, a digital multi-standard decoder, and a brightness, contrast and saturation control circuit. It is able to decode Phase Alternation Line (PAL), French Sequential Couleur avec Memoire (SECAM) and American National Television Systems Committee (NTSC) standard signals into CCIR-601 (International Radio Consultative Committee, now the International Telecommunication Union Radiocommunication Sector, standard for encoding analog video signals in digital form) compatible color component data values. The SAA7113H accepts as analog inputs CVBS (i.e., composite Chroma Video Blanking & Sync inputs) or S-video (Y/C). - Digital video data from
decoder 215 may be stored in animage buffer 220. Theimage buffer 220 provides a means for storing wide-angle image data corresponding to a monitored area. Theimage buffer 220 may be comprised of volatile or non-volatile memory or other devices configured to store video image data. The data may be stored temporarily, for example, until no longer needed for transformation by theimage data processor 225, as described below, until new video image data is required to be stored in the occupied portion of the buffer, or until some other event occurs or a period of time elapses. Thebuffer 220 may be sized to store image data corresponding to an entire field or frame of video, or more or less data. - One or more
user input modules 290 may also be provided to generate user commands. By way of example, the module may include aninput device 275 such as a keypad, digitizer, joystick, microphone and voice recognition module, or some other device configured to enable a user to enter commands. Theinput device 275 may be operably coupled to aninput processor 280 andinput memory 285 configured to produceuser command data 292 corresponding to user input commands. Theinput processor 280 may be implemented as a special purpose computer; a programmed general purpose computer; a programmed microprocessor, microcontroller or programmable read-only memory and peripheral integrated circuit elements; an application specific integrated circuit (ASIC) or other integrated circuits; a digital signal processor; a hardwired electronic or logic circuit such as a discrete element circuit; a programmable logic device such as a field programmable gate array (FPGA), or the like. Theinput memory 285 may include volatile or non-volatile memory such as, for example, static random access memory (SRAM), dynamic random access memory (DRAM), fast page mode dynamic random access memory (FPM DRAM), extended data-out dynamic random access memory (EDO DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic RAM (DDR SDRAM), electronically erasable programmable read only memory (EEPROM) such as flash memory with or without a controller, hard disk enabled virtual memory, and/or other data storage devices that may be operably coupled to theinput processor 280. Using auser input device 275, a user may identify a portion of a panoramic view to be displayed as a virtual view. A user may also control zoom factors for virtual views, and other display parameters via theuser input device 275. - An
image data processor 225 may receive bufferedimage data 222, clock signals 217 anduser command data 292 to produce panoramic view data, virtual view data and reference data. Theimage data processor 225 thus provides a means for processing and transforming data received from the storing means (i.e., the image buffer 220) into panoramic view data corresponding to at least one panoramic view of the monitored area and into virtual view data corresponding to at least one virtual view of a portion of the at least one panoramic view. Theprocessor 225 may be implemented as a special purpose computer; a programmed general purpose computer; a programmed microprocessor, microcontroller or programmable read-only memory and peripheral integrated circuit elements; an application specific integrated circuit (ASIC) or other integrated circuits; a digital signal processor; a hardwired electronic or logic circuit such as a discrete element circuit; a programmable logic device such as a field programmable gate array (FPGA), or the like. A preferred implementation uses an FPGA or an ASIC as theprocessor 225. When a system design is considered stable, an FPGA could be replaced with an ASIC to reduce system cost. - The
image data processor 225 may also be operably coupled tomemory 230, which may, for example, be non-volatile memory in the form of electronically erasable programmable read only memory (EEPROM), such as flash memory, with or without a controller; a solid-state floppy-disk card (SSFDC) such as SmartMedia® memory commercially available from Kabushiki Kaisha Toshiba, Toshiba Corporation, and Toshiba America Electronic Components, Inc.; CompactFlash® memory by SanDisk Corporation; hard disk enabled virtual memory; and/or other non-volatile storage devices. Illustratively,memory 230 may be used to facilitate processor computations by storing one or more tables of values for frequently calculated functions, especially functions that might otherwise consumesignificant processor 225 resources. By way of example and not limitation,memory 230 may store a table of values, such as cosine, sine, 1/cosine, 1/sine, or other values based on trigonometric or other types of functions used in an applicable transformation algorithm, as discussed more fully below. The indices for the table may be angular values in degrees, radians, other types of measurements, or some other possible value for a variable. The table entries may be pre-calculated values based on the corresponding indices. Trigonometric functions and complex floating point calculations can otherwise be a time and resource consuming part ofprocessor 225 calculations. The use of pre-determined stored values conserves processor resources and may lead to an appreciable increase in overall computation speed. - As discussed more fully below, the
image data processor 225 mathematically transforms wide angle video image data in theimage buffer 220 into panoramic views using a panoramic view transformation algorithm. Depending upon the implementation, theimage data processor 225 may also transform wide angle video image data in theimage buffer 220 into virtual views using a virtual view transformation algorithm, as discussed more fully below. - Upon transformation, the
processor 225 sends transformed video data to anencoder 235. The encoder converts the digital video data into output signals compatible with an output device such as adisplay monitor 240 and/orvideo recorder 245. - By way of example and not limitation, the encoder may be any suitable commercially available decoder, such as an SAA7121H digital video encoder by Phillips Electronics, N.V. The SAA7121H digital video encoder circuit accepts CCIR compatible YUV data with 720 active pixels per line and encodes the digital YUV video data to NTSC, PAL, CVBS or S-video signals.
- Output from the
encoder 235 is sent to an output device, such as astandard display monitor 240 and/orvideo recorder 245. The display monitor may be any device configured to visually display images based on electronic video signals output from theencoder 235. The recorder may be any device configured to record video signals from theencoder 235 on removable or non-removable storage media such as, for example, a magnetic tape, diskette, CD-ROM, DVD, hard disk, memory (e.g., nonvolatile EEPROM) or the like. If both adisplay monitor 240 andrecorder 245 are used, the devices may be configured in parallel or in series (e.g., without output from the video recorder being sent to a display monitor). - In an alternative embodiment, instead of transforming wide angle video image data in the
image buffer 220 into a virtual view using a virtual view transformation algorithm, theimage data processor 225 may generate virtual view data from video image data produced by a one or moreadditional camera systems 250. An additional camera system may include an analog ordigital camera 260 with a normal lens 255 (i.e., not a wide angle lens) aimed at a determined portion of the monitored area. Thelens 255 may have a narrower Field of View (FOV), e.g., 45°, which would provide higher resolution than wide-angle lens 205, thus providing a more detailed video image for a virtual view. Thus, the additional camera enables switching from a low resolution virtual view derived from wide-angle lens image data to a high resolution virtual view, with high zoom factor capability, produced using the additional camera system. This gives the user the ability to isolate small details of a monitored area. - The
additional camera system 250 may also include avideo decoder 265 that is operably coupled to the camera and configured to translate analog signals from the camera into digital video data. Thedecoder 265 may also include a clock circuit (e.g., a 27 MHz video clock circuit) suitable for synchronization and processing of video image data streams. Clock signals 267 may be transmitted toprocessor 225. - The
additional camera system 250 may further include animage buffer 270. Theimage buffer 270 may be comprised of volatile or non-volatile memory or other devices configured to temporarily store video image data, such as, for example, static random access memory (SRAM), dynamic random access memory (DRAM), fast page mode dynamic random access memory (FPM DRAM), extended data-out dynamic random access memory (EDO DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic RAM (DDR SDRAM), Rambus dynamic random access memory (RDRAM), multiport dynamic random access memory (MPDRAM), synchronous graphics RAM (SGRAM), electronically erasable programmable read only memory (EEPROM), hard disk enabled virtual memory and/or other data storage device. Thebuffer 270 may be sized to store image data corresponding to an entire field or frame of video, or more or less data.Buffered video data 272 may be transmitted toprocessor 225. That data may then be processed and encoded 235 for display in eithervirtual view - Moreover, the
additional camera system 250 may be configured as a programmable dome camera system. Dome cameras can typically rotate (i.e., pan) 360 degrees, and tilt and zoom according to control signals. Thedome camera 260 may be coupled to a processor programmed to produce control signals causing thecamera 260 to aim at particular portions of a monitored area in particular patterns. Alternatively,processor 225 may producecontrol data 294, based uponuser command data 292 fromuser input module 290, to control positioning and movement of the dome camera. A control interface (not shown) may also be operably coupled todome camera 260 andprocessor 225, as necessary to convert control data output from theprocessor 225 into signals for controllingdome camera 260 position, movement and zooming. - Referring again to
FIG. 1 ,panoramic views virtual views panoramic views virtual views - The panoramic view transformation process involves mathematically transforming wide angle video image data into 2-dimensional space for producing panoramic views. The exemplary transformation entails determining, for each output window pixel coordinate, an address within the
image buffer 220 for pixel data to be encoded and output to adisplay 240 and/orrecorder 245. In a preferred implementation, this address calculation is timed such that the calculated address for each output pixel is ready just in time to feed the output encoder. This implementation advantageously eliminates the need for an output image buffer. - In an exemplary implementation, two
panoramic views panoramic view 110 corresponds to half of the monitored area (e.g., a 0-180 degree panoramic view strip). The other panoramic view corresponds to the other half of the monitored area (e.g., a 181-360° panoramic view strip). Thus, in this implementation, the second panoramic view is offset from the first panoramic view by half of the field of view. The points dividing one half from the other half may be pre-programmed, user configurable or user controllable. The two panoramic views combined provide a full (e.g., a 360°) view of the monitored area. - The exemplary panoramic views are 720 pixels wide and 120 pixels high. Output window pixel coordinates range from −360 to +360 for Wx and from −60 to +60 for Wy. The Wx and Wy coordinates are derived from the output view column and line pixel counters, and together with the desired Zoom factor, are transformed into θ (i.e., the pan rotation angle around the Z-axis) and φ (the tilt rotation angle around the X-axis), as conceptually illustrated in
FIG. 3 . This information is then converted to an address in the input capture buffer. The data from this address is output to the encoder for display. The variables Pan, Tilt and Zoom may be programmed or commanded viauser input module 290. - Given:
-
-
- Zoom—1X_Tilt_FOV=45°
- ScaleFactorX(SFx)=180°/720 pixels=0.25°/pixel
- Zoom Scale(ZS)=1 (Requested zoom factor ie: 1, 2, 3 etc.)
- ScaleFactorY(SFy)=(Zoom—1X_FOV)×(1/ZS/120 pixels)
- ScaleFactorY(SFy)=45°/120 pixels=0.375°/pixel
- Pan=Requested Pan angle in degrees
- Tilt=Requested Tilt angle in degrees
- Wx=Output Window pixel x coordinate (−360 to +360)
- Wy=Output Window pixel y coordinate (+60 down to −60
- IRx=Image buffer fisheye image X axis radius in pixels
- IRy=Image buffer fisheye image Y axis radius in pixels
- ICx=Image buffer fisheye image center X coordinate in pixels, as conceptually shown in
FIG. 4 - ICy=Image buffer fisheye image center Y coordinate in pixels, as conceptually shown in
FIG. 4 .
- The horizontal component of the panoramic view (starting with output window upper left at −360,+120) is converted from pixel units to degrees(SFx). Each output pixel equates to 0.25 degrees (180°/720 pixels). The programmable Pan value is added to each pixel value. The second
panoramic view 160 is offset from the first 110 by +180 degrees. - Therefore:
θ=[(Wx+360)×SFx]+Pan=[(Wx+360)×0.25]+Pan Equation 1
θ=0.25Wx+90+Pan Equation 2 - The vertical component of the panoramic view (starting with output window upper left at origin −360,+120) is converted from pixel units to degrees (SFy). Each vertical pixel equates to 0.375 degrees (45′/120 lines). The programmable Tilt value is added to each pixel value. The programmable zoom performs a scale (45°×(1/zoom))/120. Therefore, Zoom Scale equals 1.
φ=(π/2−Tilt)−[(Wy+60)×(SFy)] Equation 3
φ=90−Tilt−0.375Wy−22.5 Equation 4
φ=67.5−(Tilt+0.375Wy) Equation 5
Radius =−φ/π/2 Equation 6
Radius=−φ/90 Equation 7 - Next, the input image X,Y (Ixy) pixel is calculated, as follows:
Ix=(Radius×cos(θ)×IRx)+ICxj Equation 8
Iy=(Radius×sin(θ)×IRy)+ICy Equation 9 - The
image buffer 220 is preferably addressed as a 1024 byte by 1024 byte array. The memory location, in the image buffer for the pixel data is calculated by:
PixelData=(Iy×1024)+Ix Equation 10 - PixelData may then be sent to the
encoder 235 for output to the output device such as adisplay monitor 240 and or arecorder 245. Pixel Data for each pixel comprising thepanoramic views - Each
virtual view window panoramic view FIG. 3 , is 0 degrees. Additionally, φ is the Tilt angle around the X-Axis where the positive Z direction, pointing straight down as shown inFIG. 3 , is 0 degrees. Z is the perpendicular distance from the X-Y plane down to the surface of a unity radius hemispheric fisheye model. - The virtual view transformation algorithm entails converting the X and Y output window coordinates to an equivalent proportion of a unity radius hemisphere model using a current Zoom factor and a measured radius-X and radius-Y from an input fisheye circular image. These adjusted values are used to calculate a corresponding Z value. The X, Y and Z coordinates are rotated around the unity radius hemisphere model using the commanded Pan and Tilt from the
user input module 290. The rotated X and Y values are then rescaled back to pixels and lines, based on measured radius-X and radius-Y from the input fisheye circular image. The rescaled X and Y are then converted to an address in the input capture buffer. Data from this address is then output to the encoder for display. The variables Pan, Tilt, and Zoom are the angle of interest commanded by theuser input module 290. - Given:
-
-
- Pan=Commanded Pan angle in degrees
- Tilt=Commanded Tilt angle in degrees
- Zoom=Requested Zoom scale (ie: 1, 2, 3, etc. . . . )
- Wx=Output Window pixel x coordinate (−180 to +180)
- Wy=Output Window pixel y coordinate (+120 down to −120)
- IRx=Image buffer fisheye image X axis radius in pixels
- IRy=Image buffer fisheye image Y axis radius in pixels
- ICx=Image buffer fisheye image center X coordinate in pixels, as conceptually shown in
FIG. 4 - ICy=Image buffer fisheye image center Y coordinate in pixels, as conceptually shown in
FIG. 4 .
- The vertical and horizontal components of the virtual view window (center at origin 0,0) are converted from pixel units to unit values from 0 to less than one by dividing pixel number by the respective image radius. One is the maximum length from the center of the virtual output image, to the edge of the image. By scaling to the image radius much of the 3-dimensional transformation math can be simplified.
Wx=WindowX/IRx Equation 11
Wy=WindowY/IRy Equation 12 - To scale the Wx and Wy components by Zoom, the following equations are used.
Wx=Wx×1/Zoom Equation 13
Wy=Wy×1/Zoom Equation 14 - The desired radius squared is calculated from the vertical and horizontal components using the Pythagorean theorem.
Wr 2 =Wx 2 +Wy 2 Equation 15 - For each pixel, to convert the two dimensional radius squared and scaled components to a three dimensional value with a unit radius of 1, the following equations are used.
- Given:
Ix 2 +Iy 2 +Iz 2=1 Equation 16
Then:
Iz={square root}{square root over (1−Wr 2 )} Equation 17 - Assuming the acquired image center is perpendicular to the Z axis, a rotation of the desired window to a desired spot on the image can be performed using the Pan and Tilt values to simulate a movable dome.
- To rotate around the X axis for tilt, the following equations are used.
Iz′=(Iz×cos(Tilt))−(Wy×sin(Tilt)) Equation 18
Wy′=(Wy×cos(Tilt))+(Iz×sin (Tilt)) Equation 19 - To rotate around the Z axis for pan, the following equations are used.
Wx′=(Wx×cos(Pan))−(Wy′×sin (Pan)) Equation 20
Wy′=(Wx×sin(Pan))+(Wy′×cos(Pan)) Equation 21 - To convert unit values to Image Buffer pixel units, the following equations are used.
Ix=Wx′×IRx Equation 22
Iy=Wy′×IRy Equation 23 - To align the origin relative to upper left of input image buffer, the following equations are used.
Ix=Ix+ICx Equation 24
Iy=Iy+ICy Equation 25 - The
image buffer 220 may be addressed as a 1024 byte by 1024 byte array. The memory location, in theimage buffer 220, for the pixel data is calculated using the
PixelData=(Iy×1024)+Ix Equation 26 - PixelData may then be sent to the
encoder 235 for output to the display window. - Advantageously, the
panoramic views virtual views virtual views reference windows - Referring now to
FIG. 5 , a high level flow chart for a process of creating a video user interface in accordance with an exemplary implementation of the invention is shown. The steps may be implemented using a system with a programmed processor as described above.FIG. 5 may equally represent a high level block diagram of an exemplary system constructed in accordance with the invention, implementing the steps thereof. - In
step 510, wide-angle data (e.g., fisheye lens image data) is buffered. The data may be supplied from a video camera system with a decoder or from a source of pre-recorded data. - Next, in
step 520, pan, tilt and zoom angles are determined to specify regions of interest in the panoramic views for display in virtual views. Pan, tilt and zoom angles may be supplied by a programmed algorithm, a table or a user input module. Those skilled in the art will appreciate that values other than pan, tilt and zoom (e.g., Cartesian coordinate values) may be specified to define areas of interest in the panoramic views. - Next, in
step 530, reference data for reference windows are determined. The reference data preferably corresponds to rectangular windows overlaying the virtual view regions of interest within the panoramic views. Preferably, the rectangular windows move according to a programmed algorithm, a table or user commands from a user input module. As the rectangular windows move, the virtual views correspondingly change. - Next, in step 540, buffered wide-angle data are transformed using a panoramic transformation algorithm for panoramic view data, and using a virtual view transformation algorithm for virtual view data.
- Next, in
step 550, the transformed data and reference data are encoded for output. The transformed data defines any other virtual views and the panoramic views. The reference data defines the reference windows. The encoded output may be sent to a display monitor and/or to a recorder. - Those skilled in the art will appreciate that the order in which some of the steps may vary from the order shown in the flowchart without departing from the scope of the invention. For example, reference data may be determined before, after or during transformation of the buffered data. Additionally, a pre-programmed pan, tilt and zoom may be determined before wide-angle data is initially buffered.
- Referring now to
FIG. 6 , a high level flow chart for another process of creating video user interface in accordance with an exemplary implementation of the invention is shown. As with the process illustrated by the flowchart ofFIG. 5 , the steps may be implemented using a system with a programmed processor as described above.FIG. 6 may equally represent a high level block diagram of an exemplary system constructed in accordance with the invention, implementing the steps thereof. - In
step 610, wide-angle data (e.g., fisheye lens image data) is buffered. The data may be supplied from a video camera system with a decoder or from a source of pre-recorded data. - Next, in
step 620, pan, tilt and zoom angles are determined to specify regions of interest in the panoramic views for display in virtual views. Pan, tilt and zoom angles may be supplied by a programmed algorithm, a table or a user input module. Those skilled in the art will appreciate that values other than pan, tilt and zoom (e.g., Cartesian coordinate values) may be specified to define areas of interest in the panoramic views. - Next, in
step 630, reference data for reference windows are determined. The reference data preferably corresponds to rectangular windows overlaying the virtual view regions of interest within the panoramic views. Preferably, the rectangular windows move according to a programmed algorithm, a table or user commands from a user input module. As the rectangular windows move, the virtual views correspondingly change. - Next, in
step 640, position data is communicated to a dome camera system. The communicated position data will cause the dome camera to aim at and zoom to the region of the monitored area corresponding to the determined pan, tilt and zoom values. - In
step 650, dome camera data is buffered. The buffered data may be encoded, without transformation, for output to a display monitor and/or to a recorder as a virtual view. - Next, in
step 660, buffered wide-angle data are transformed using a panoramic transformation algorithm for panoramic view data, and using a virtual view transformation algorithm for virtual view data. - Next, in
step 670, buffered dome camera data are obtained for encoding. The dome camera data may be obtained form a dome camera buffer by a processor and communicated with transformed wide-angle data for encoding. - Next, in step 680, the transformed data, dome camera data and reference data are encoded for output. The dome camera data defines one virtual view. The transformed data defines any other virtual views and the panoramic views. The reference data defines the reference windows. The encoded output may be sent to a display monitor and/or to a recorder.
- Those skilled in the art will appreciate that the order in which some of the steps are performed may not be important. For example, the reference data may be determined before, after or during transformation of the buffered data. Additionally, by way of example, a pre-programmed pan, tilt and zoom may be determined before the wide-angle data is initially buffered. As another example, dome position data may be sent before reference data is determined. As yet another example, dome data may be buffered after transformation of wide-angle data. Also, dome camera data may be obtained by the processor before completion of transformation. Such variations in order are intended to come within the scope of the invention.
- Although the foregoing description is directed to preferred embodiments of the invention, it is noted that other variations and modifications will be apparent to those skilled in the art, and may be made without departing from the spirit or scope of the invention.
Claims (22)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/802,451 US20050062845A1 (en) | 2003-09-12 | 2004-03-17 | Video user interface system and method |
US13/946,279 US9955073B2 (en) | 2003-09-12 | 2013-07-19 | Video user interface system and method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US50272803P | 2003-09-12 | 2003-09-12 | |
US10/802,451 US20050062845A1 (en) | 2003-09-12 | 2004-03-17 | Video user interface system and method |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/946,279 Continuation US9955073B2 (en) | 2003-09-12 | 2013-07-19 | Video user interface system and method |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050062845A1 true US20050062845A1 (en) | 2005-03-24 |
Family
ID=34135389
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/802,451 Abandoned US20050062845A1 (en) | 2003-09-12 | 2004-03-17 | Video user interface system and method |
US13/946,279 Active 2024-09-17 US9955073B2 (en) | 2003-09-12 | 2013-07-19 | Video user interface system and method |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/946,279 Active 2024-09-17 US9955073B2 (en) | 2003-09-12 | 2013-07-19 | Video user interface system and method |
Country Status (4)
Country | Link |
---|---|
US (2) | US20050062845A1 (en) |
EP (1) | EP1515546B1 (en) |
CA (1) | CA2465045C (en) |
ES (1) | ES2748129T3 (en) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050226463A1 (en) * | 2004-03-31 | 2005-10-13 | Fujitsu Limited | Imaging data server and imaging data transmission system |
US20060015554A1 (en) * | 2004-07-14 | 2006-01-19 | Fujitsu Limited | Image distributing apparatus |
US20060033813A1 (en) * | 2004-08-06 | 2006-02-16 | Provinsal Mark S | Immersive surveillance system interface |
CN101827260A (en) * | 2010-05-06 | 2010-09-08 | 东莞光阵显示器制品有限公司 | Implementation method of video transmission and real time monitoring and system device |
US20120045149A1 (en) * | 2010-03-18 | 2012-02-23 | Panasonic Corporation | Omnidirectional image processing device and omnidirectional image processing method |
US20120206565A1 (en) * | 2011-02-10 | 2012-08-16 | Jason Villmer | Omni-directional camera and related viewing software |
US20160191797A1 (en) * | 2014-12-26 | 2016-06-30 | Casio Computer Co., Ltd. | Image Processing Device and Image Playback Device Which Control Display Of Wide-Range Image |
US20190005613A1 (en) * | 2015-08-12 | 2019-01-03 | Sony Corporation | Image processing apparatus, image processing method, program, and image processing system |
US10225511B1 (en) | 2015-12-30 | 2019-03-05 | Google Llc | Low power framework for controlling image sensor mode in a mobile image capture device |
US10732809B2 (en) | 2015-12-30 | 2020-08-04 | Google Llc | Systems and methods for selective retention and editing of images captured by mobile image capture device |
CN113542848A (en) * | 2021-06-22 | 2021-10-22 | 惠州市德赛西威汽车电子股份有限公司 | Vehicle-mounted image window self-defining adjusting method and vehicle-mounted system |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2006131759A (en) * | 2006-09-04 | 2008-03-10 | Николай Иванович Пальченко (RU) | METHOD AND SYSTEM OF MODELING, REPRESENTATION AND FUNCTIONING OF A UNIFIED VIRTUAL SPACE AS A UNIFIED INFRASTRUCTURE FOR IMPLEMENTATION OF REAL AND VIRTUAL ECONOMIC AND OTHER HUMAN ACTIVITIES |
FR2937208B1 (en) | 2008-10-13 | 2011-04-15 | Withings | METHOD AND DEVICE FOR TELEVISIONING |
US9792012B2 (en) | 2009-10-01 | 2017-10-17 | Mobile Imaging In Sweden Ab | Method relating to digital images |
US9544498B2 (en) | 2010-09-20 | 2017-01-10 | Mobile Imaging In Sweden Ab | Method for forming images |
CN105472344A (en) * | 2015-12-28 | 2016-04-06 | 北京极图科技有限公司 | Four-way splicing type real-time panoramic imaging device |
KR102473815B1 (en) | 2016-10-18 | 2022-12-05 | 주식회사 바텍 | X-ray image display device and method |
EP3311750A1 (en) * | 2016-10-18 | 2018-04-25 | Vatech Co. Ltd. | Producing panoramic radiograph |
CN109151295B (en) * | 2017-06-16 | 2020-04-03 | 杭州海康威视数字技术股份有限公司 | Target object snapshot method and device and video monitoring equipment |
Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5359363A (en) * | 1991-05-13 | 1994-10-25 | Telerobotics International, Inc. | Omniview motionless camera surveillance system |
US5563650A (en) * | 1992-11-24 | 1996-10-08 | Geeris Holding Nederland B.V. | Method and device for producing panoramic images, and a method and device for consulting panoramic images |
US5870135A (en) * | 1995-07-27 | 1999-02-09 | Sensormatic Electronics Corporation | Image splitting forming and processing device and method for use with no moving parts camera |
USRE36207E (en) * | 1991-05-13 | 1999-05-04 | Omniview, Inc. | Omniview motionless camera orientation system |
US6034716A (en) * | 1997-12-18 | 2000-03-07 | Whiting; Joshua B. | Panoramic digital camera system |
US20020005896A1 (en) * | 2000-05-23 | 2002-01-17 | Kiyoshi Kumata | Surround surveillance system for mobile body, and mobile body, car, and train using the same |
US20020080017A1 (en) * | 2000-10-11 | 2002-06-27 | Kiyoshi Kumata | Surround surveillance apparatus for mobile body |
US6624846B1 (en) * | 1997-07-18 | 2003-09-23 | Interval Research Corporation | Visual user interface for use in controlling the interaction of a device with a spatial region |
US20040047623A1 (en) * | 2002-09-10 | 2004-03-11 | Top Arnold N. | Rugged miniature pan/tilt dome camera assembly |
US20040257436A1 (en) * | 1997-04-21 | 2004-12-23 | Sony Corporation | Controller for photographing apparatus and photographing system |
US20050141607A1 (en) * | 2003-07-14 | 2005-06-30 | Michael Kaplinsky | Multi-sensor panoramic network camera |
US20060136972A1 (en) * | 2003-02-11 | 2006-06-22 | Raymond Metzger | System for a plurality of video cameras disposed on a common network |
US7103232B2 (en) * | 2001-03-07 | 2006-09-05 | Canon Kabushiki Kaisha | Storing and processing partial images obtained from a panoramic image |
US7206017B1 (en) * | 1998-11-13 | 2007-04-17 | Olympus Corporation | Image reproduction apparatus with panoramic mode based on aspect ratio |
US20070182819A1 (en) * | 2000-06-14 | 2007-08-09 | E-Watch Inc. | Digital Security Multimedia Sensor |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE3919265A1 (en) * | 1989-06-13 | 1990-12-20 | Krupp Atlas Elektronik Gmbh | OPTO-ELECTRONIC OUTLOOK ASSEMBLY |
CA2155719C (en) * | 1994-11-22 | 2005-11-01 | Terry Laurence Glatt | Video surveillance system with pilot and slave cameras |
US6459451B2 (en) * | 1996-06-24 | 2002-10-01 | Be Here Corporation | Method and apparatus for a panoramic camera to capture a 360 degree image |
US6215519B1 (en) * | 1998-03-04 | 2001-04-10 | The Trustees Of Columbia University In The City Of New York | Combined wide angle and narrow angle imaging system and method for surveillance and monitoring |
US20020147991A1 (en) * | 2001-04-10 | 2002-10-10 | Furlan John L. W. | Transmission of panoramic video via existing video infrastructure |
US6833843B2 (en) * | 2001-12-03 | 2004-12-21 | Tempest Microsystems | Panoramic imaging and display system with canonical magnifier |
JP3925299B2 (en) * | 2002-05-15 | 2007-06-06 | ソニー株式会社 | Monitoring system and method |
-
2004
- 2004-03-17 US US10/802,451 patent/US20050062845A1/en not_active Abandoned
- 2004-04-22 CA CA2465045A patent/CA2465045C/en not_active Expired - Lifetime
- 2004-04-29 ES ES04010139T patent/ES2748129T3/en active Active
- 2004-04-29 EP EP04010139.6A patent/EP1515546B1/en active Active
-
2013
- 2013-07-19 US US13/946,279 patent/US9955073B2/en active Active
Patent Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5359363A (en) * | 1991-05-13 | 1994-10-25 | Telerobotics International, Inc. | Omniview motionless camera surveillance system |
USRE36207E (en) * | 1991-05-13 | 1999-05-04 | Omniview, Inc. | Omniview motionless camera orientation system |
US5563650A (en) * | 1992-11-24 | 1996-10-08 | Geeris Holding Nederland B.V. | Method and device for producing panoramic images, and a method and device for consulting panoramic images |
US5870135A (en) * | 1995-07-27 | 1999-02-09 | Sensormatic Electronics Corporation | Image splitting forming and processing device and method for use with no moving parts camera |
US20040257436A1 (en) * | 1997-04-21 | 2004-12-23 | Sony Corporation | Controller for photographing apparatus and photographing system |
US6624846B1 (en) * | 1997-07-18 | 2003-09-23 | Interval Research Corporation | Visual user interface for use in controlling the interaction of a device with a spatial region |
US6034716A (en) * | 1997-12-18 | 2000-03-07 | Whiting; Joshua B. | Panoramic digital camera system |
US7206017B1 (en) * | 1998-11-13 | 2007-04-17 | Olympus Corporation | Image reproduction apparatus with panoramic mode based on aspect ratio |
US20020005896A1 (en) * | 2000-05-23 | 2002-01-17 | Kiyoshi Kumata | Surround surveillance system for mobile body, and mobile body, car, and train using the same |
US20070182819A1 (en) * | 2000-06-14 | 2007-08-09 | E-Watch Inc. | Digital Security Multimedia Sensor |
US20020080017A1 (en) * | 2000-10-11 | 2002-06-27 | Kiyoshi Kumata | Surround surveillance apparatus for mobile body |
US7103232B2 (en) * | 2001-03-07 | 2006-09-05 | Canon Kabushiki Kaisha | Storing and processing partial images obtained from a panoramic image |
US20040047623A1 (en) * | 2002-09-10 | 2004-03-11 | Top Arnold N. | Rugged miniature pan/tilt dome camera assembly |
US20060136972A1 (en) * | 2003-02-11 | 2006-06-22 | Raymond Metzger | System for a plurality of video cameras disposed on a common network |
US20050141607A1 (en) * | 2003-07-14 | 2005-06-30 | Michael Kaplinsky | Multi-sensor panoramic network camera |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050226463A1 (en) * | 2004-03-31 | 2005-10-13 | Fujitsu Limited | Imaging data server and imaging data transmission system |
US20060015554A1 (en) * | 2004-07-14 | 2006-01-19 | Fujitsu Limited | Image distributing apparatus |
US7561187B2 (en) * | 2004-07-14 | 2009-07-14 | Fujitsu Limited | Image distributing apparatus |
US20060033813A1 (en) * | 2004-08-06 | 2006-02-16 | Provinsal Mark S | Immersive surveillance system interface |
US7750936B2 (en) * | 2004-08-06 | 2010-07-06 | Sony Corporation | Immersive surveillance system interface |
US8744216B2 (en) * | 2010-03-18 | 2014-06-03 | Panasonic Corporation | Omnidirectional image processing device and method using warp processing for enhanced object visibility |
US20120045149A1 (en) * | 2010-03-18 | 2012-02-23 | Panasonic Corporation | Omnidirectional image processing device and omnidirectional image processing method |
CN101827260A (en) * | 2010-05-06 | 2010-09-08 | 东莞光阵显示器制品有限公司 | Implementation method of video transmission and real time monitoring and system device |
US20120206565A1 (en) * | 2011-02-10 | 2012-08-16 | Jason Villmer | Omni-directional camera and related viewing software |
US9930225B2 (en) * | 2011-02-10 | 2018-03-27 | Villmer Llc | Omni-directional camera and related viewing software |
US20160191797A1 (en) * | 2014-12-26 | 2016-06-30 | Casio Computer Co., Ltd. | Image Processing Device and Image Playback Device Which Control Display Of Wide-Range Image |
US20190005613A1 (en) * | 2015-08-12 | 2019-01-03 | Sony Corporation | Image processing apparatus, image processing method, program, and image processing system |
US10867365B2 (en) * | 2015-08-12 | 2020-12-15 | Sony Corporation | Image processing apparatus, image processing method, and image processing system for synthesizing an image |
US10225511B1 (en) | 2015-12-30 | 2019-03-05 | Google Llc | Low power framework for controlling image sensor mode in a mobile image capture device |
US10728489B2 (en) | 2015-12-30 | 2020-07-28 | Google Llc | Low power framework for controlling image sensor mode in a mobile image capture device |
US10732809B2 (en) | 2015-12-30 | 2020-08-04 | Google Llc | Systems and methods for selective retention and editing of images captured by mobile image capture device |
US11159763B2 (en) | 2015-12-30 | 2021-10-26 | Google Llc | Low power framework for controlling image sensor mode in a mobile image capture device |
CN113542848A (en) * | 2021-06-22 | 2021-10-22 | 惠州市德赛西威汽车电子股份有限公司 | Vehicle-mounted image window self-defining adjusting method and vehicle-mounted system |
Also Published As
Publication number | Publication date |
---|---|
US20130300822A1 (en) | 2013-11-14 |
ES2748129T3 (en) | 2020-03-13 |
EP1515546B1 (en) | 2019-08-07 |
CA2465045A1 (en) | 2005-03-12 |
US9955073B2 (en) | 2018-04-24 |
EP1515546A2 (en) | 2005-03-16 |
EP1515546A3 (en) | 2010-01-20 |
CA2465045C (en) | 2018-05-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9955073B2 (en) | Video user interface system and method | |
CA2465087C (en) | Imaging system and method for displaying and/or recording undistorted wide-angle image data | |
US7161615B2 (en) | System and method for tracking objects and obscuring fields of view under video surveillance | |
US10091408B2 (en) | Imaging apparatus and method of recognizing target object | |
KR101270893B1 (en) | Image processing device, image processing method, program thereof, recording medium containing the program, and imaging device | |
US8855441B2 (en) | Method and apparatus for transforming a non-linear lens-distorted image | |
JP4243767B2 (en) | Fisheye lens camera device and image extraction method thereof | |
JP4268206B2 (en) | Fisheye lens camera device and image distortion correction method thereof | |
JP3012142B2 (en) | Full-view still camera monitoring system | |
JP5235910B2 (en) | Camera head system | |
JPH11261868A (en) | Fisheye lens camera device and image distortion correction method and image extraction method thereof | |
JP2011142419A5 (en) | ||
JP2006245793A (en) | Imaging system | |
US8860780B1 (en) | Automatic pivoting in a wide-angle video camera | |
KR101778744B1 (en) | Monitoring system through synthesis of multiple camera inputs | |
US6876387B1 (en) | Digital zoom-out processing apparatus | |
JP3594225B2 (en) | Wide-field camera device | |
KR102009988B1 (en) | Method for compensating image camera system for compensating distortion of lens using super wide angle camera and Transport Video Interface Apparatus used in it | |
WO1996008105A1 (en) | Method for creating image data | |
EP4009622B1 (en) | Method for capturing and processing a digital panoramic image | |
KR20180022327A (en) | Image transforming device of a 360-degree revolution in CCTV camera | |
JP2019009574A (en) | Image processing apparatus | |
JP2019008494A (en) | Image processing device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SENSORMATIC ELECTRONICS CORPORATION, FLORIDA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MILLS, LAWRENCE R.;REEL/FRAME:015118/0911 Effective date: 20040317 |
|
AS | Assignment |
Owner name: SENSORMATIC ELECTRONICS, LLC,FLORIDA Free format text: MERGER;ASSIGNOR:SENSORMATIC ELECTRONICS CORPORATION;REEL/FRAME:024213/0049 Effective date: 20090922 Owner name: SENSORMATIC ELECTRONICS, LLC, FLORIDA Free format text: MERGER;ASSIGNOR:SENSORMATIC ELECTRONICS CORPORATION;REEL/FRAME:024213/0049 Effective date: 20090922 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION |