Common steps in machine learning

I will summary one way to solve a machine learning problem. These abstract steps fit many problems.

1. Understand the task. See how to measure the performance.

   A computer program is learning if its performance improves with more experience. We are going to design such a program.

2. Choose the source of training experience.

3. Decide what will be input and output.

4. Choose a set of models to approximate the output function.

   For example, the set could be a set of linear functions or a set of neural networks with N hidden neurons. We should use our knowledge of the problem to restrict the set. The set only needs to contain the true output function or its good approximation.

5. Choose a learning algorithm.

   The algorithm will select one model from the given set of models. The model is selected based on high probability under the given training data (i.e., P(model|data)).

   We may also select multiple models and use a weighted vote from them. The models should be mutually exclusive. Every model should have just one vote.

Resources

Chapters 1, 2 and 6 of Mitchell's Machine Learning.