tag:blogger.com,1999:blog-1367197553634210691.comments2017-04-23T13:42:30.588+02:00Lessoned CodeIvo Danihelkanoreply@blogger.comBlogger15125tag:blogger.com,1999:blog-1367197553634210691.post-34301488379857970212017-04-23T13:42:30.588+02:002017-04-23T13:42:30.588+02:00I realize that this post is very old and dead, but...I realize that this post is very old and dead, but I just wanted to share the observation that it is possible to compute the class probabilities for each leaf once we have trained the tree. <br />Then inference can be done in roughly the same time as for a classical decision tree. <br /><br />However, I'm not sure how to make CTW scale well in the number of classes. Perhaps at each node we should simply predict the most likely class (or the most likely few) vs the rest rather than assigning probabilities to each class.Sean Burtonhttps://www.blogger.com/profile/04717225197734852252noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-61565300942478299222013-12-17T14:53:13.828+01:002013-12-17T14:53:13.828+01:00Thank you !!!Thank you !!!karine mirashttps://www.blogger.com/profile/01597219167832846513noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-73042836297046178192012-12-11T12:31:04.925+01:002012-12-11T12:31:04.925+01:00I will try to explain some unmentioned things.
In...I will try to explain some unmentioned things.<br /><br />Input to the network<br />====================<br />At the bottom layer, each pixel represents a cell in a grid.<br />Each pixels is modeled by 4 models: white, whiteGrey, blackGrey, black.<br />So the cell holds 4 values:<br /> P(pixel|PixelModel=white), P(pixel|PixelModel=whiteGrey),<br />P(pixel|PixelModel=blackGrey), P(pixel|PixelModel=black)<br /><br />For the images, these probabilities are modeled by a 4 Gaussians.<br />Each Gaussian has a different mean. E.g., the mean of the "white"<br />model corresponds to intensity of white pixels.<br /><br />Product nodes:<br />Each product node joins two regions together. At the lowest layer,<br />a region is just a pixel position. The joined probability is:<br /> P(pixel1, pixel2|ThisProductNode).<br /><br />For example, one product node can be joining the often-white pixel1<br />and the often-black pixel2:<br /> P(pixel1, pixel2|ThisProductNode) = P(pixel1|PixelModel=white) *<br />P(pixel2|PixelModel=black)<br /><br />Different product nodes can be joining different combinations of pixel values.<br /><br />In higher layers, bigger and bigger regions are joined together.<br />In the SPN Java code, they allow to join adjacent regions together.<br /><br />SPN is not limited to joining just adjacent regions.<br />But the space of possible combinations to check needs to small.<br />The hard EM training is searching for product nodes with high likelihood:<br />P(region|ProductNode).<br /><br />About Hard EM training<br />======================<br />You know EM for a mixture of Guassians, right?<br />The "hard EM" refers to using k-means like training instead.<br />Only one centroid is selected for each example.<br />The "centroid" is a branch in the SPN tree.<br />The weights on the selected branch are increased,<br />to increase the P(example|whole_net).<br /><br />The best branch to select, is the branch which would produce<br />the highest P(example|whole_net), <b>after increasing the branch weights</b>.<br />So when looking for the best branch:<br />1) The network is evaluated from bottom up.<br />2) When evaluating a sum node,<br />all possible increments are considered. A consideration: increment the weight<br />for one of the children and keep the other weights unchanged.<br /> => Find the child with the highest P(covered_region|childNode)<br />3) The best found P(covered_region|ThisSumNode_with_increment) is reported up.<br />4) The best branch has the highest P(example|ThisBranch_with_increment)<br />at the top.<br /><br />BTW, I recommend the <a href="https://www.coursera.org/course/ml" rel="nofollow">ML</a> and <a href="https://www.coursera.org/course/pgm" rel="nofollow">PGM</a> classes at Coursera.Ivo Danihelkahttps://www.blogger.com/profile/12732454351811400017noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-12050235164614631412012-12-07T04:44:11.135+01:002012-12-07T04:44:11.135+01:00Clear and concise. Much easier to understand than ...Clear and concise. Much easier to understand than the original paper...:). Thanks, dude.Ziheng Jianghttps://www.blogger.com/profile/00174819667928528411noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-30498497049799633772012-08-30T16:35:36.176+02:002012-08-30T16:35:36.176+02:00More posts please!More posts please!Dan Haiduchttps://www.blogger.com/profile/15276894431071453667noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-76328466624941862852012-03-15T08:43:41.541+01:002012-03-15T08:43:41.541+01:00Hello Ivo,
I am trying to understand the CTW algor...Hello Ivo,<br />I am trying to understand the CTW algorithm and I got some resources too..problem is, I do not understand all the symbolic notations. Can you suggest some material which explain it in simple.<br />or, can suggest some tutorial for understanding the statistical notations.<br /><br />thanksPremhttps://www.blogger.com/profile/04612069257182904623noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-33795660268354591752011-12-14T20:36:27.000+01:002011-12-14T20:36:27.000+01:00Very nice blog. Keep the good work! Sum-products n...Very nice blog. Keep the good work! Sum-products networks seem to be promising.vlasdashttps://www.blogger.com/profile/15175375940929880763noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-31918171717187660512011-12-14T17:59:35.690+01:002011-12-14T17:59:35.690+01:00Thanks for telling me. The link is fixed now.Thanks for telling me. The link is fixed now.Ivohttps://www.blogger.com/profile/12732454351811400017noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-41537773166554920882011-12-14T16:57:58.556+01:002011-12-14T16:57:58.556+01:00SVM link is not ok.SVM link is not ok.rodrigobhttps://www.blogger.com/profile/05583769088304975293noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-90050580176218107262011-10-04T09:49:03.286+02:002011-10-04T09:49:03.286+02:00There is a Java implementation by the original aut...There is a Java implementation by the original authors:<br />http://alchemy.cs.washington.edu/spn/<br /><br />You can run it on your laptop. See the user guide. It uses MPI to distribute the computation to multiple cores or machines.Ivohttps://www.blogger.com/profile/12732454351811400017noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-66684421056464457772011-10-04T09:40:12.066+02:002011-10-04T09:40:12.066+02:00Looks interesting. Is there any known implementati...Looks interesting. Is there any known implementation or is it just DIY?Guidohttps://www.blogger.com/profile/12147991495120816879noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-90113036667504198832010-04-17T19:49:58.123+02:002010-04-17T19:49:58.123+02:00MeggieSort :-DMeggieSort :-DGuidohttps://www.blogger.com/profile/12147991495120816879noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-68635106467826146342009-11-27T18:43:21.789+01:002009-11-27T18:43:21.789+01:00Good introduction, although a little bit common. I...Good introduction, although a little bit common. I'm looking forward to next posts.Guidohttps://www.blogger.com/profile/12147991495120816879noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-85582660896006600222009-11-27T10:03:39.581+01:002009-11-27T10:03:39.581+01:00Yes you are right. I should not write that join() ...Yes you are right. I should not write that join() is slower.<br />A more correct statement would be:<br />Using .append() followed by .join() is slower than using += directly.Ivohttps://www.blogger.com/profile/12732454351811400017noreply@blogger.comtag:blogger.com,1999:blog-1367197553634210691.post-78516798075131587092008-07-24T12:05:00.000+02:002008-07-24T12:05:00.000+02:00This is not a fair comparison as you are building ...This is not a fair comparison as you are building up a list before you combine by joining. Just comparing concatenation vs. joining yields different results: <BR/><BR/>In [20]: text=""<BR/>In [21]: a=['aaaaa']*1000000<BR/>In [22]: %time for c in a: text+=c<BR/>CPU times: user 0.67 s, sys: 0.03 s, total: 0.70 s<BR/>Wall time: 0.71 s<BR/>In [24]: %time text="".join(a) <BR/>CPU times: user 0.10 s, sys: 0.01 s, total: 0.10 s<BR/>Wall time: 0.10 srobhttps://www.blogger.com/profile/04183271899326957094noreply@blogger.com