Saturday, January 23, 2010

Many layers are needed

I have read an interesting paper on limitations of machine learning models: Scaling Learning Algorithms towards AI. It mentions limitation of two-layer neural networks and other two-layer models (SVMs). These shallow models are unable to learn some functions without an exponential number of components. For example, to learn the parity function over N input bits, they would need 2N hidden neurons.

On the other hand, a deep model with N layers could compute the parity with just N components.

Tuesday, January 19, 2010

Humane utility function

It will be hard to design a utility function for a strong AI. The utility function should express what the AI should maximize. Humans still cannot decide what weight to assign to lives. Especially if you have to decide between lives and the restoration of order in a society.

Saturday, January 2, 2010

AI for real life: Understanding autonomy

There are talks about autonomy and other forms of motivations. But I realized the meaning of autonomy only after reading about it in an AI book.

Let's first define how we want an agent or an employee to behave. We want him to try his best to maximize our score assigned to him. When doing "his best", he can only use his prior knowledge or his senses. We should not blame a deaf person for not running in reaction to the sound "fire". We should also not blame a person without the prior knowledge of the English word "fire". They are acting rationally under the given conditions.

And autonomy is the ability to enhance or correct the prior knowledge. An autonomous agent does not need to follow the rules defined in the prior knowledge. He could override them if it makes sense to him. For example, he does not have to run immediately after hearing "fire". He can grab the deaf person's hand first.