DE19723294C2 - Pattern recognition methods - Google Patents

Description

The invention relates to a pattern recognition method according to the preamble of Patent claim 1.

For pattern recognition methods, in particular for recognizing speech or writing, recognition systems based on Hidden Markov Models have been used for some time, see for example [KCGM 93 ]. In such recognition systems, multi-dimensional feature vectors are obtained from test objects and mapped onto symbols of a multi-dimensional symbol vector space. The list of these symbols associated with different states of the underlying HMM model is called a codebook. These symbols, in turn, can be considered as coefficients of a multidimensional symbol vector.

For the transformation of the feature vectors into the symbol vector space, called Vector quantization is, in principle, between continuous, part differentiated between continuous and discrete models. The continuous models come in most because of the associated processing overhead Not considered.

In the discrete model, there is a so-called hard decision in the Assignment of a feature vector to a symbol instead. Opposite is at the semicontinuous models a feature vector multiple symbols with im Usually assigned a different weighting. For the determination of this For example, weights for the clusters are state-dependent Feature vector space a statistical distribution, in particular a Gauss- or  a gamma distribution, assumed. The part-continuous models show in the In general, a much better recognition rate than the discrete models

The present invention is based on the object, the recognition performance a pattern recognition method of this kind, without the Significant increase in recognition effort.

The invention is described in claim 1. The subclaims contain advantageous embodiments and modifications of the invention.

The invention is based on the finding that the usual partial-continuous Models assumed statistical distribution the actual distribution of Feature vectors in the state-dependent vector space often only very unsatisfactory describes. In contrast, in the present invention made no assumption about a particular distribution function, but it from representative training data the coefficients of a polynomial Classifier determines which one is essential in vector quantization closer approximation to the actual distribution in vector space represents.

In an advantageous embodiment, the polynomial classifier with Moments matrices are also trained in training the HMM models incurred. The vector quantization is preferably carried out in several stages in the Kind, that several polynomial part classifiers are provided, which are the nodes represent the junction and are arranged in a tree-like junction, which has the advantage over a single classifier that a restriction on only the best paths of the tree can be made and thereby significantly fewer polynomials need to be evaluated. Preferably, these are Part classifiers run as binary polynomial classifiers.

Advantageously, in a manner known per se prior to the implementation of Vector quantization the dimension of the feature vectors by means of a linear Discriminant analysis reduced.  

The invention is illustrated below still using a recognition system for recognizing bound font. Such a recognition system is described, for example, in [KCGM 93 ] and for details of such an exemplary recognition system, reference is made to this reference.

Starting from a binary pixel image of a handwritten letter or number sequence, an essential task when using an HMM recognizer is to generate from the binary image a sequence of vectors as input variables of the HMM recognizer. Such preprocessing may comprise, for example, the transition from the pixel representation to a contour description, which may then be followed by normalization steps, such as writing line estimation, oblique script erection, rotation of the typeface, and transition to a skeleton representation. Such preprocessing measures are known and used in various ways. A more detailed illustration of the preprocessing measures of such a recognition system is described, for example, in [CGM 93 ]. The setting of an HMM recognizer is complex and, as well known in various embodiments of the prior art, is not described in detail here. In order to generate feature vectors from the preprocessed image, for example, a narrow window section in the assumed writing direction is moved stepwise over the text image, and at each step features are extracted from the window content which in their entirety form a feature vector. This procedure is applied to typefaces of known content in a training phase as well as to typefaces of unknown content in a subsequent recognition phase.

If the number of training patterns is sufficiently large, they will show from these patterns feature vectors accumulations, so-called clusters, im multidimensional space of feature vectors. These clusters can be on known manner, for example, according to the known and predominantly applied LBG algorithm.  

According to an advantageous procedure known from the prior art When moving the window, not only are the static features a window position, but also the differences of the features to adjacent ones Window positions determined and further processed, resulting in a corresponding higher-dimensional feature vector leads. The following statements apply, without the need for a separate explicit reference, even for such and similar variants of feature extraction.

In a first training phase of the HMM recognizer possible paths through the structures and assignments of individual feature vectors to states of the HMM on the basis of the previously made state-independent Vektorquantisierungstrainings (one or more so-called codebooks) and on the basis of predeterminable structures of the HMM models for the possible object classes Models, eg. B. using the so-called forward-backward method, and determines the coefficients of the HMM models. In this framework, torque matrices were also determined from the feature vectors to be assigned for the various states which form the symbols of a single new state-dependent codebook. The moment matrices are used to estimate a linear discriminant transformation matrix, which in turn serves to reduce the dimension of the feature vectors. These process steps are known and customary and described in detail in (Fuk 90 ).

From the moment matrices can be when using a Vektorquantisierers on the Basis of assumed statistical distributions, e.g. Gaussian distributions, the Parameters of the distribution such. B. center of gravity and covariance can be determined. However, such assumed statistical distributions approximate the actual ones Distributions often insufficient.

According to the present invention, the use of a polynomial Classificator a completely different way followed, in which a priori no Assumptions about a particular statistical distribution are made. The  Coefficients of the polynomial classifier are calculated on the basis of the Training phase determined feature vectors.

Advantageously, instead of the feature vectors, the already previously used formed torque matrices and so in the Coefficient determination anyway required step of forming these matrices avoided the feature vectors. In addition, this way can Advantageously, the explicit classification of the class membership of individual feature vectors are bypassed because the moment matrices anyway belong to certain HMM states and these in the later vector quantizer polynomial class just the symbols to be generated.

For reasons of computational effort, it is also advantageous here with the opposite working on the original feature vector space reduced dimension by the linear discriminant transformation on the feature vectors or the Torque matrices is applied.

The polynomial structure is preferably given as a complete structure, but can also be thinned out to reduce the computational effort and remains unchanged during the training phase. For the discriminant function d (v) of the polynomial to the different object classes

d (v) = A ^T x (v)

with A as a coefficient matrix and x (v) as a function for a vectorial mapping of the original feature vector v onto the polynomial structure list x.

For each node of the path branch tree (trellis), the probability that a feature vector is to be assigned to a particular state of an HMM model can be determined, resulting in new moment matrices E {xx ^T } and E {xy ^T } with y as symbol classes. Vector leads from which the coefficient matrix after

A = E {xx ^T } ^-1 E {xy ^T }

can be determined. A detailed description of these and other aspects of polynomial classifiers can be found in [Sch 96 ].

The use of a polynomial classifier, at least when used the least mean square error as an optimization criterion, the advantage of a mathematically closed solution without iterations.

To reduce the computational effort, vector quantization is preferred not by means of a single single-stage polynomial classifier, but in several stages with successive branching in successive stages using multiple Polynomial part classifiers performed in the manner of a tree structure. The Partial classifiers are particularly advantageous as binary polynomial Classifiers executed.

The setting of the polynomial classifier in a training phase is usually versus a vector quantizer with assumed statistical Distribution function in this training phase with a higher computational effort be connected. However, this only occurs once in the training phase. Of the Calculation effort in the later recognition phase is not or only insignificant higher compared to vector quantization based on statistical Distribution functions. When going through the branch tree of the sub- Classifiers can be pruning by truncating insignificant paths Processing complexity in the vector quantization can be further reduced.

The improved description of the actual distribution in the feature vector Space through the polynomial classifier leads to initial investigations compared to an assumed normal distribution to a significant Reduction of the error rate, otherwise setting the same recognition system for example, an error rate of 5.6% compared to 6.3% if assumed Normal distribution.  

Bibliography

[KCGM93] A. Kaltenmeier, T. Caesar, JM Gloger, and E. Mandler. Sophisticated topology of hidden markov models for cursive script recognition. In Proceedings of the Second International Conference on Document Analysis and Recognition, pp. 139-142, Tsukuba Science City, Japan, 20.-22. October 1993. IEEE Computer Society Press
[CGM93] T. Caesar, JM Gloger, and E. Mandler. Preprocessing and feature extraction for a handwriting recognition system. In Proceedings of the Second International Conference on Document Analysis and Recognition, pp. 408-411, Tsukuba Science City, Japan, 20.-22. October 1993, IEEE Computer Society Press.
[Fuk90] Keinosuke Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press Inc., San Diego, CA, 2nd Edition, 1990
[Sch96] Jürgen Schürmann, Pattern Classification, John Wiley & Sons Inc., New York

1996

Claims (6)

A pattern recognition method based on Hidden Markov models in which feature vectors are transformed into symbol vectors by means of vector quantization, characterized in that the vector quantization is performed by means of a polynomial classifier without assumption over statistical distributions present in the state dependent feature space. 2. The method according to claim 1, characterized in that the Vector quantization is branched in multiple stages, with individual polynomial Part classifiers representing nodes of the branch. 3. The method according to claim 2, characterized in that the polynomial partial Classifiers are binary polynomial classifiers. 4. The method according to any one of claims 1 to 3, characterized in that the Dimension of the feature vectors reduced and the vector quantization based on the dimension-reduced feature vectors is performed. 5. The method according to any one of claims 1 to 4, characterized in that the Coefficients of the polynomial classifier from those in a training phase estimated feature vectors or derived variables. 6. The method according to claim 5, characterized in that in one Training phase of the HMM recognizer from the feature vectors moment Matrices are formed and these moment matrices are also used to estimate the Coefficients of the polynomial classifier or polynomial classifiers be used.