WO2006087524A1

WO2006087524A1 - A driver assistance system

Info

Publication number: WO2006087524A1
Application number: PCT/GB2006/000493
Authority: WO
Inventors: David Matthew James Stark
Original assignee: Trw Limited
Priority date: 2005-02-18
Filing date: 2006-02-14
Publication date: 2006-08-24
Also published as: EP1848611A1; GB0503388D0; WO2006087524B1

Abstract

A driver assistance system for a vehicle which has a mirror unit (1) fixed thereto in a position such that a driver of the vehicle looking into the mirror unit (1) is provided with a view outside of the vehicle comprising a projector (14) adapted to project an image onto the surface of the mirror (12), the image being visible to the driver when looking into the mirror (12), the image overlaying the view seen by the driver in the mirror (12) thereby to assist the driver in identifying salient features within the view visible in the mirror (12).

Description

A DRIVER ASSISTANCE SYSTEM

This invention relates to a driver assistance system which is provided on a vehicle and which assists a driver of a vehicle.

When driving a vehicle, such as a car or lorry, the driver constantly makes decisions based upon their interpretation of the surroundings of the vehicle. When driving straight along a road, for example, the driver constantly checks for obstacles which need to be avoided, or lane markings showing the direction of the road.

Several systems have been developed to assist the driver. These can generally be divided into passive and active systems. In a passive system the driver is presented with information to assist them. For example, a sound may be emitted to warn that the driver is departing from a lane on a carriageway or that a collision with an obstacle is likely. In an active system, control of the vehicle is at least partially taken away from the driver and passed to the system. For example, the brakes may automatically be applied if a collision is likely, or to counter the onset of understeer or oversteer.

According to a first aspect the invention provides a driver assistance system for a vehicle which has a mirror fixed thereto in a position such that a driver of the vehicle looking into the mirror is provided with a view outside of the vehicle, the system comprising: projection means adapted to project an image onto the surface of the mirror, the image being visible to the driver when looking into the mirror, the image overlaying the view seen by the driver in the mirror thereby to assist the driver in identifying salient features within the view visible in the mirror. The provision of a projecting means which provides an overlay onto the image that would otherwise be seen by the driver when looking in a mirror provides many possible advantages to the driver. The information assists the driver in deciding what they can see. The type of image which can be projected onto the surface of the mirror, and the source of the image, can vary for many embodiments within the scope of the present invention.

The image may comprise a representation of any lane markings which the driver may be able to see when looking in the mirror. These should be projected onto the mirror in such a way that the driver sees an "enhanced" or otherwise augmented view of the lane markings. The projected image could appear to the driver as a solid or broken line overlaying the actual view of the boundaries, making them easier to see.

It may alternatively or additionally display an image that helps the driver identify a vehicle which the driver can view in the mirror. A cross-hair or other icon located at the centre of a vehicle image could be provided, or perhaps an outline of the vehicle. This assists the driver in identifying vehicles in the mirror, or perhaps obstacles such as bollards when reversing.

Providing the overlay image may therefore ensure that important features that the driver will be looking for in the mirror are enhanced. This can provide a useful level of extra safety when driving in low visibility conditions such as rain or at night.

The image may be also or alternatively be adapted to indicate to the driver the proximity of an object that can be seen in the mirror. For example, an icon or outline of a vehicle could be shown in different colours, or at different intensities, or may flash at different rates dependent on how close it is.

The system may include a camera which may be attached to the vehicle. It may be located on, within or proximal to the housing and may view an area outside of the vehicle which at least approximates the area that is viewed by the driver when looking in the mirror. It may, in fact, view a wider or alternative area outside of the vehicle.

The camera may capture one or more images of the scene which may be passed to a processing means. This processing means may convert the images into the image that is to be projected by the projection means. It may comprise a digital processor. The processor may employ edge detection techniques to identify the boundary of salient features in the images captured by the camera.

The camera may capture images in the visible spectrum. It may also, or alternatively, capture images in the infra-red or near infra-red spectrum. The later may be beneficial in identifying salient features under low light conditions. A source of infra-red illumination may be provided to assist the camera in capturing images.

In a still further alternative, or additionally, other non-camera devices could be provided which capture information from the region that the driver views in the mirror. For example, a radar or Lidar sensor may be provided. The output of these devices may again be passed to a processor for generating an image to be projected onto the mirror.

A driver position determining means may be provided which determines the position of the drivers head relative to a fixed part of the vehicle, and may determine the position and direction of gaze of the drivers eyes. This information may be passed to a processor to modify the image overlayed on the mirrored surface to ensure that the driver perceives the image and the reflection of the scene in the mirror to be in register. Modification may comprise any one or more of scaling, translating, skewing and distorting the projected image.

The driver position determining means may comprise at least one camera.

Preferably it comprises two cameras which may work together to provide a stereoscopic image.

In a simpler system, the driver may be provided with controls which permit them to alter the overlay to suit their head position when driving.

One or more marks could be provided on the mirror and the driver may be presented with a calibration pattern which they can alter until it aligns with the marks.

The system may be arranged to adopt a default setting if the driver position information is lost for any reason.

An additional sensor may be provided which generates an output signal or signals indicative of the angle of orientation of the mirror relative to a fixed portion of the vehicle.

The processor, or a different processor, may combine information from the detector about the driver's head (or eye) position with information about the position and angle of inclination of the mirror to determine any modification of the overlay that is required.

The projecting means may be located in front of or behind the mirror surface or may form a part of the surface. It may project an image through the mirror from the rear towards the driver. Alternatively, it may project an image from the front of the mirror which is reflected towards the driver. The projecting means may include one or more light sources such as LEDS or Laser devices. Where the light is projected from the front of the mirror, in order for the light from the projected device to be reflected towards the eyes of a driver the surface of the mirror may be provided with a suitable treatment to scatter a proportion of the projected light.

The mirror may therefore be provided with a doped or cloudy layer over the whole or part of its surface which scatters incident projector light such that some of the scattered light is reflected towards the driver. This also ensures that the image is visible to a driver when viewing from a range of positions.

The mirror may also display other information to the driver. This could, for example, be used to show the driver the outside temperature. In cold weather an alert could be shown when there is a risk of freezing. Vehicle speed could be shown, or perhaps navigation information.

This additional information may be presented to the driver in a region of the mirror which does not contain any salient features and/or which provides the best contrast for the display of the information. This can be determined by processing the images captured by the camera or information from any other sensors.

The system may also be adapted to determine the relative speeds of salient features viewed in the mirror and the host vehicle and may display an image which depends on the relative speeds. For example, if the processor determines that a vehicle seen by a sensor is approaching rapidly a different type or colour of overlay may be provided than for a vehicle which is receding away. The overlay for an approaching vehicle could flash or have a warning attached to it for example.

In a modification, the system may increase or decrease the intensity of at least a part of the overlay image if there is a risk that it would be obscured by bright lights reflected in the mirror at night.

The system may include control means which enable the driver to switch the overlay image on or off. The controls may also permit the driver to alter the brightness of the overlay image, or the colours or even the style of the overlay image. For instance, the driver could control the types of salient feature they wish to have highlighted in the overlay image. Where an appropriate display screen is provided in the vehicle, this information could be presented through a driver selectable menu.

According to a second aspect the invention provides a mirror assembly for a vehicle which combines a driver information system for a vehicle according to the first aspect and a mirror in a single housing.

The housing may also include a camera for detecting the position of a driver's head or eyes. Alternatively, this camera may be provided outside of the housing, and may be connected thereto by one or more cables.

The mirror may be connected to the housing through an adjustable support and detecting means may be provided for determining the position of the mirror relative to the housing.

The mirror assembly may be suitable for retrofitting to a vehicle. It may comprise a driver or passenger side mirror for mounting to a vehicle on or proximal an A pillar of the vehicle. It may comprise a rear view mirror for attachment to a windscreen or the roof of a vehicle. According to a third aspect the invention provides a driver assistance system comprising means for projecting information onto or through a surface such as a window or mirror through or in which a driver has a view outside of the vehicle, means for determining the view seen by the driver in or through the surface, processing means for processing the view seen by the driver to identify at least one salient feature, and means for projecting an image onto or through the surface whereby the image does not obscure the drivers view of the salient feature in the window or mirror.

Examples of salient features include other vehicles, pedestrians, lane markings, obstacles in the path of the vehicle and road signs. By ensuring that the projected information does not obscure such salient features it will not be obscured from the driver.

The system may project information onto a windscreen for reflection towards a driver in the manner of a head-up display. It may project information to an area of the windscreen which does not contain a salient feature. For example, the information could be moved around depending on where the salient features are.

Alternatively, the system may be adapted to project the image onto or through a portion of the surface which provides the greatest contrast between the image and the view seen in or through the surface.

This alternative may be of use when projecting information during periods of bright light which could obscure the projected information. There will now be described, by way of example only, one embodiment of the present invention with reference to the accompanying drawings of which:

Figure 1 is an overview of a system in accordance with a first aspect of the invention;

Figure 2 is a block diagram illustrating the interaction between each of the components of the system of figure 1 ;

Figure 3 is a view of a part of an alternative system in accordance with the present invention in which a projector is located bin front of a mirror surface;

Figure 4 is a view of a part of a still further alternative system in accordance with the first aspect of the present invention in which a projector is located away from the mirror;

Figure 5 (a) shows a typical scene viewed by a driver in a mirror of a system in accordance with the first aspect of the invention;

Figure 5(b) shows an overlay image produced for the scene shown in Figure 5 (a);

Figure 5(c) shows the combined scene and overlay that can be seen in the mirror by the driver;

Figure 6 shows additional information presented in the mirror giving temperature and a frost warning; and Figure 7 shows alternative additional information giving navigational instructions .

As shown in Figure 1, a driver assistance system comprises a mirror unit 1 secured to a vehicle 2 in a position which permits a driver 3 of the vehicle to view a scene reflected in the mirror unit 1. The unit includes a housing 11 will typically be secured to one side of the vehicle so it is visible through either the drivers side window or passenger side window.

In each case, the driver 3 will observe in the mirror unit a reflection of the scene to the rear and one side of the vehicle. Alternatively, it may be secured to the windscreen or roof of the vehicle 2 to present a scene through the rear window of the vehicle.

The mirror unit 1, an example of which can be seen in more detail in Figure 3 of the accompanying drawings comprises a housing 11 which supports a mirror surface 12, typically rectangular, through an adjustable mounting and control structure 13. The mounting 13 allows the driver 3 to move the mirror surface 12 within the housing 11 to a position which affords the best view. This position will depend on the position of the drivers head 3 whilst driving. A projection means 14 is also attached to the housing 11. In this example this comprises a small light projector.

The projector 14 overlays an image onto the mirror surface 12 which is reflected from the mirror surface 12 into the eyes of the driver 3. The purpose of the projected image is to present to the driver information which helps them process the scene that they see in the mirror.

To produce the images projected onto the mirror surface 12, the system includes a rear facing camera 15 which captures an image approximately corresponding to that which would be seen by the driver 3 when looking in the mirror surface 12. The camera 15 should have sufficient sensitivity to be of use at night. This image, or a sequence of such images, is then passed to a processing means that generates the images to be projected. The processing means extracts salient features from the captured images and from these salient features produces an appropriate overlay image to display in the mirror surface 12.

To ensure that the overlay image is in register with that seen in the mirror surface 12 by the driver 3, the system also includes two cameras 20,30 attached to the front part of the vehicle 2 to either side of the driver 3. The cameras 20,30 capture images of the driver 3. These images are passed to the processing means which determines the position of the drivers head. An additional sensor attached to the mirror as part of the control device 13 determines its relative position relative to the housing 11. This information is then used to ensure that the overlay and actual images are aligned when viewed by the driver.

The relationship between the various components and the processing means is shown in block diagram of Figure 2 of the accompanying drawings. Images from the two cameras 20,30 that detect the drivers head position, and also from the mirror mounted camera 15 are fed to a frame grabber circuit 40 and on to a processor 50. The processor also receives signals from the mirror position sensor, and user controls 60. This information is processed to produce a projected overlay image which is sent to the projector 14.

The features which may be highlighted in the projected image may comprise:

Other vehicles

If located within the mirror's field of view, these can be highlighted either by tracing their outline, or, more simply, placing rectangles in their general position. If their approach speed can be determined this information can be encoded onto the tracing or rectangle. Options for this include but are not limited to:

Colour change, e.g. red for high speed approach green for low Change of line type or flashing line Numeric speed or warning shown in addition

Lane markings/carriageway boundaries

By picking out the edge of the road, and painted markings on it, the driver is provided with a frame of reference. This then allows the driver to accurately place the vehicle on the road and to determine the position of the vehicle relative to others, which assists which joining a motorway using a sliproad, or changing lane when already on the motorway. These markings can be highlighted by tracing over them with a solid line or broken line.

The role of the processing means 40,50 is therefore to receive and process images from the camera to identify salient features, to produce a raw image to be projected onto the mirror surface 12, to receive information about the drivers head position and mirror position, and to apply a transform to the raw overlay image to ensure it is correct for a given head position and mirror position.

As previously described, the projector may be provided as part of the mirror unit. In this arrangement, the projector 14 may be located just in front of the mirror surface with an image being reflected from the mirror. In a modification, the projector may still be located at the mirror unit but instead projects information through the mirrored surface from its rear. This is achieved by providing a mirror surface which is "one-way", reflecting light from its front face only whilst transmitting light from the rear. The projector comprises an LCD panel having dimensions corresponding to those of the mirror surface. Such technology is well known for other applications such as television screens for bathrooms. An alternative would replace the LCD panel with a matrix of LEDs.

A still further alternative is shown in Figure 4 of the accompanying drawings. A projector 14 is secured to the vehicle at a distance from the mirror unit 11. In this example, the mirror unit 11 is attached to the front corner of a drivers door at the foot of the A post of the vehicle and the projector 14 is attached at the foot of the B pillar. This would not, of course, be possible with a simple 100 percent reflective mirror as the projected light would need to be scattered as it strikes the mirrored surface. A solution to this problem can be found by providing a mirror with a slightly cloudy layer on the surface that will scatter a small proportion of light incident upon it, say 1-2 percent. The mirror may therefore comprise a glass plate having a silvering on its rear surface and a cloudy layer on its front surface. It is advantageous to apply this to the surface rather than the back face of the mirror, to avoid any dirt on the surface acting as a second scattering layer and producing a double image of the enhanced projections with an offset between them.

Figures 5 (a) to (c) show the system of Figures 1 to 4 in operation. Figure 5 (a) shows a typical view of a scene in a mirror as seen by a driver. The scene includes several salient features including a lane boundary 100 and a vehicle 101 which is partially obscured. It is a nighttime scene and so it is quite hard to identify the salient features in the dark image.

Figure 5(b) shows the image that is to be projected onto the mirror surface for this scene. The processor identified the lane boundary 100 and has created a dashed line 100a which overlays it. The outline of the vehicle 101 has also been identified and a boundary line 101a which overlays it has been produced. A warning arrow 101b has also been produced to highlight a partially obscured vehicle.

The position of these features in the image is determined with reference to the output of the cameras 20,30 which determine the drivers head position and the sensor that determines mirror alignment. The image is modified to ensure that the features exactly overlay the actual features when viewed by the driver.

The result of projecting the image from Figure 5(b) onto the scene viewed in the mirror from Figure 5 (a) can be seen in Figure 5(c) . The way in which the projected information assists the driver in identifying salient features is clearly visible.

The system may be arranged to show other, additional information. In the examples shown in Figures 6 and 7 outside temperature and navigational information are shown.

Where additional information is presented, this information can be presented in a region of the mirror which will not overlay salient information. A system operating in this manner falls within the scope of the second aspect of the invention.

Claims

1. A driver assistance system for a vehicle which has a mirror fixed thereto in a position such that a driver of the vehicle looking into the mirror is provided with a view outside of the vehicle, the system comprising: projection means adapted to project an image onto the surface of the mirror, the image being visible to the driver when looking into the mirror, the image overlaying the view seen by the driver in the mirror thereby to assist the driver in identifying salient features within the view visible in the mirror.

2. A driver assistance system according to claim 1 in which the projection means projects an image which comprises a representation of any lane markings which the driver may be able to see when looking in the mirror.

3. A driver assistance system according to claim 1 or claim 2 in which the projection means projects an image which helps the driver identify a vehicle which the driver can view in the mirror.

4. A driver assistance system according to claim 1,2 or 3 in which the image is adapted to indicate to the driver the proximity of an object that can be seen in the mirror.

5. A driver assistance system according to any preceding claim which includes a camera attached to the vehicle which views an area outside of the vehicle which at least approximates the area that is viewed by the driver when looking in the mirror.

6. A driver assistance system according to claim 5 in which the camera captures one or more images of the scene which are passed to a processing means which converts the images into the image that is to be projected by the projection means.

7. A driver assistance system according to claim 5 or 6 in which the camera captures images in the visible spectrum or in the infra-red or near infra-red spectrum.

8. A driver assistance system according to any preceding claim which further includes a driver position determining means which determines the position of the drivers head relative to a fixed part of the vehicle.

9. A driver assistance system according to any preceding claim in which a sensor is provided which generates an output signal or signals indicative of the angle of orientation of the mirror relative to a fixed portion of the vehicle.

10. A driver assistance system according to claims 8 and 9 which is adapted to combine information from the detector about the driver's head

(or eye) position with information about the position and angle of inclination of the mirror to determine any modification of the overlay that is required.

11. A driver assistance system according to any preceding claim in which the projecting means is located in front of or behind the mirror surface .

12. A driver assistance system according to any preceding claim which is adapted to determine the relative speeds of salient features viewed in the mirror and the host vehicle and to display an image which depends on the relative speeds.

13. A driver assistance system according to any preceding claim in which the mirror surface is provided with a cloudy or doped layer such that light reflected from the mirror is at least partially scattered.

14. A mirror assembly for a vehicle which combines a driver information system for a vehicle according to any one of claims 1 to 12 and a mirror in a single housing.

15. A driver assistance system comprising means for projecting information onto or through a surface such as a window or mirror through or in which a driver has a view outside of the vehicle, means for determining the view seen by the driver in or through the surface, processing means for processing the view seen by the driver to identify at least one salient feature, and means for projecting an image onto or through the surface whereby the image does not obscure the drivers view of the salient feature in the window or mirror.

16. A driver assistance system substantially as described herein with reference to and as illustrated in the accompanying drawings.