tag:blogger.com,1999:blog-1367197553634210691.post1268588591312160451..comments2017-04-23T13:42:30.588+02:00Comments on Lessoned Code: Intro to Sum-Product NetworksIvo Danihelkanoreply@blogger.comBlogger6125tag:blogger.com,1999:blog-1367197553634210691.post-61565300942478299222013-12-17T14:53:13.828+01:002013-12-17T14:53:13.828+01:00Thank you !!!Thank you !!!karine mirashttps://www.blogger.com/profile/01597219167832846513noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-73042836297046178192012-12-11T12:31:04.925+01:002012-12-11T12:31:04.925+01:00I will try to explain some unmentioned things.
In...I will try to explain some unmentioned things.<br /><br />Input to the network<br />====================<br />At the bottom layer, each pixel represents a cell in a grid.<br />Each pixels is modeled by 4 models: white, whiteGrey, blackGrey, black.<br />So the cell holds 4 values:<br /> P(pixel|PixelModel=white), P(pixel|PixelModel=whiteGrey),<br />P(pixel|PixelModel=blackGrey), P(pixel|PixelModel=black)<br /><br />For the images, these probabilities are modeled by a 4 Gaussians.<br />Each Gaussian has a different mean. E.g., the mean of the "white"<br />model corresponds to intensity of white pixels.<br /><br />Product nodes:<br />Each product node joins two regions together. At the lowest layer,<br />a region is just a pixel position. The joined probability is:<br /> P(pixel1, pixel2|ThisProductNode).<br /><br />For example, one product node can be joining the often-white pixel1<br />and the often-black pixel2:<br /> P(pixel1, pixel2|ThisProductNode) = P(pixel1|PixelModel=white) *<br />P(pixel2|PixelModel=black)<br /><br />Different product nodes can be joining different combinations of pixel values.<br /><br />In higher layers, bigger and bigger regions are joined together.<br />In the SPN Java code, they allow to join adjacent regions together.<br /><br />SPN is not limited to joining just adjacent regions.<br />But the space of possible combinations to check needs to small.<br />The hard EM training is searching for product nodes with high likelihood:<br />P(region|ProductNode).<br /><br />About Hard EM training<br />======================<br />You know EM for a mixture of Guassians, right?<br />The "hard EM" refers to using k-means like training instead.<br />Only one centroid is selected for each example.<br />The "centroid" is a branch in the SPN tree.<br />The weights on the selected branch are increased,<br />to increase the P(example|whole_net).<br /><br />The best branch to select, is the branch which would produce<br />the highest P(example|whole_net), <b>after increasing the branch weights</b>.<br />So when looking for the best branch:<br />1) The network is evaluated from bottom up.<br />2) When evaluating a sum node,<br />all possible increments are considered. A consideration: increment the weight<br />for one of the children and keep the other weights unchanged.<br /> => Find the child with the highest P(covered_region|childNode)<br />3) The best found P(covered_region|ThisSumNode_with_increment) is reported up.<br />4) The best branch has the highest P(example|ThisBranch_with_increment)<br />at the top.<br /><br />BTW, I recommend the <a href="https://www.coursera.org/course/ml" rel="nofollow">ML</a> and <a href="https://www.coursera.org/course/pgm" rel="nofollow">PGM</a> classes at Coursera.Ivo Danihelkahttps://www.blogger.com/profile/12732454351811400017noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-12050235164614631412012-12-07T04:44:11.135+01:002012-12-07T04:44:11.135+01:00Clear and concise. Much easier to understand than ...Clear and concise. Much easier to understand than the original paper...:). Thanks, dude.Ziheng Jianghttps://www.blogger.com/profile/00174819667928528411noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-33795660268354591752011-12-14T20:36:27.000+01:002011-12-14T20:36:27.000+01:00Very nice blog. Keep the good work! Sum-products n...Very nice blog. Keep the good work! Sum-products networks seem to be promising.vlasdashttps://www.blogger.com/profile/15175375940929880763noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-90050580176218107262011-10-04T09:49:03.286+02:002011-10-04T09:49:03.286+02:00There is a Java implementation by the original aut...There is a Java implementation by the original authors:<br />http://alchemy.cs.washington.edu/spn/<br /><br />You can run it on your laptop. See the user guide. It uses MPI to distribute the computation to multiple cores or machines.Ivohttps://www.blogger.com/profile/12732454351811400017noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-66684421056464457772011-10-04T09:40:12.066+02:002011-10-04T09:40:12.066+02:00Looks interesting. Is there any known implementati...Looks interesting. Is there any known implementation or is it just DIY?Guidohttps://www.blogger.com/profile/12147991495120816879noreply@blogger.com