Sunday, May 30, 2010

Common steps in machine learning

I will summary one way to solve a machine learning problem. These abstract steps fit many problems.

  1. Understand the task. See how to measure the performance.

    A computer program is learning if its performance improves with more experience. We are going to design such a program.

  2. Choose the source of training experience.

  3. Decide what will be input and output.

  4. Choose a set of models to approximate the output function.

    For example, the set could be a set of linear functions or a set of neural networks with N hidden neurons. We should use our knowledge of the problem to restrict the set. The set only needs to contain the true output function or its good approximation.

  5. Choose a learning algorithm.

    The algorithm will select one model from the given set of models. The model is selected based on high probability under the given training data (i.e., P(model|data)).

    We may also select multiple models and use a weighted vote from them. The models should be mutually exclusive. Every model should have just one vote.

Resources

Chapters 1, 2 and 6 of Mitchell's Machine Learning.