** Up:** HMM Project
** Previous:** Build an HMM Trainer

In order to evaluate how well our HMM trainer does, we need a test to determine
how well our trained model mimics the original model. Please refer to the HMM
document
for a discussion. In it, I define a
quantity called
( ) that measures the accuracy
to which the trained model mimics the original model .
We wish to know how various parameters of the training process affect
( ).

Our HMM trainer has a number of parameters. Two parameters refer to the Markov
model that we use as the ``actual'' process, i.e., the one being simulated.

*N*:
- The number of states in the Markov model being modeled.
*M*:
- The number of observations in the Markov model being modeled.

There are at least three parameters controlling the training process.

*T*:
- The length of each observation sequence.
*L*:
- The number of observation sequences.
*C*:
- The maximum number of iterations allowed for the trained model to
converge. This may be irrelevant if convergence is fast.

For your evaluation of HMM training, I would like to see a 4-dimensional (maybe
5) table of results showing statistics on
( )
for various values of
*N*, *M*, *T* and *L* (and maybe *C*).
For each table entry, you must do a number of trainings against a number of
's that are generated randomly. In this way, we will have some
measure of the HMM training process for those parameters. For example, set
*N* = 3, *M* = 4, *T* = 5
and *L* = 20 and run, say, a dozen experiments. In each experiment, generate
a new rendomly, train a and calculate
( ).
For these, say, dozen values of
( ), report the
mean and standard deviation. Now do this for a large variety of values of the
variables explained above. With luck, we will see some trends in the data.

** Up:** HMM Project
** Previous:** Build an HMM Trainer
Webmaster: Jim Schmolze, schmolze@eecs.tufts.edu. [Get postscript version]