WEKO3
アイテム
Statistical Learning Theory by Boosting Method
https://ir.soken.ac.jp/records/763
https://ir.soken.ac.jp/records/763e7c7849a91604d7c9420b07ac1923e06
名前 / ファイル  ライセンス  アクション 

要旨・審査要旨 / Abstract, Screening Result (281.7 kB)

Item type  学位論文 / Thesis or Dissertation(1)  

公開日  20100222  
タイトル  
タイトル  Statistical Learning Theory by Boosting Method  
タイトル  
タイトル  Statistical Learning Theory by Boosting Method  
言語  en  
言語  
言語  eng  
資源タイプ  
資源タイプ識別子  http://purl.org/coar/resource_type/c_46ec  
資源タイプ  thesis  
著者名 
竹之内, 高志
× 竹之内, 高志 

フリガナ 
タケノウチ, タカシ
× タケノウチ, タカシ 

著者 
TAKENOUCHI, Takashi
× TAKENOUCHI, Takashi 

学位授与機関  
学位授与機関名  総合研究大学院大学  
学位名  
学位名  博士（学術）  
学位記番号  
内容記述タイプ  Other  
内容記述  総研大甲第739号  
研究科  
値  数物科学研究科  
専攻  
値  15 統計科学専攻  
学位授与年月日  
学位授与年月日  20040324  
学位授与年度  
値  2003  
要旨  
内容記述タイプ  Other  
内容記述  We deal with statistical learning theory, especially classification problems, by Boosting method. In the context of Boosting method, we can use only a set of weak learners which output statistical discriminant functions having low performance for a given set of examples. Aim of Boosting method is to construct a strong learner by combining a lot of weak learners and a typical boosting algorithm is AdaBoost. AdaBoost can be derived from a sequential minimization of the exponential loss function for a statistical discriminant function. This minimization problem is equivalent to the minimization of the extended KullbackLeibler divergence between an empirical distribution of given examples and an extended exponential model. Statistical properties of AdaBoost have been investigated and the relationship between the exponential loss function of AdaBoost and the logistic model was revealed. In this thesis, we obtain two main results:<br /><br />1. AdaBoost is extended to general UBoost by using the statistical form of the Bregman divergence, which contains the KullbackLeibler divergence as an example and consider a geometrical interpretation of UBoost in terms of information geometry.<br /><br />2. We propose a new Boosting algorithm ηBoost, which is a robustified version of AdaBoost.<br /><br />The UBoost is derived from a sequential minimization of the Bregman divergence between the empirical distribution and Umodel. A geometric interpretation for UBoost is given in terms of information geometry. From the Pythagorean relation associated with the Bregman divergence, we derive two special versions of UBoost, the normalized UBoost and the unnormalized UBoost. We define the normalized version of Umodel on the probability space and derive normalized UBoost from this model. The normalized UBoost corresponds to usual statistical classification methods, for example, logistic discriminant analysis. The unnormalized UBoost is derived from an unnormalized version of Umodel defined on the extended nonnegative measure space and has not been seen in the previous statistical context. Especially, unnormalized UBoost has a beautiful geometrical structure related to the Pythagorean relation and the flatness. Its algorithm is interpreted as a pile of right triangles which leads to a mild convergence property of UBoost algorithm as seen in the EM algorithm. Based on a probabilistic assumption for a training data set, statistical discussion for consistency, efficiency and robustness of UBoost is given. <br /><br />An algorithm of AdaBoost implements the learning process by exponentially reweighting examples according to classification results. Then weight distribution is often too sharply tuned, so that AdaBoost has a weak point on the robustness and overlearning. As a special example of UBoost, we propose ηBoost which aims to robustify AdaBoost to avoid an overlearning. The statistical meaning of ηBoost is discussed and ηBoost is associated with a probabilistic model of mislabeling which is a contaminated logistic model. As a general UBoost algorithm, ηBoost also has a normalized and unnormalized version. A loss function of the normalized version of ηBoost is a minus loglikelihood of a contaminated logistic model in which mislabeling probability is constant and does not depend on the input. The unnormalized version of ηBoost is a slight modification of AdaBoost and is derived from a loss function which is defined by a mixture of the exponential loss of AdaBoost and naive error loss functions. A probabilistic model of unnormalized version is also a contaminated logistic model and its mislabeling probability depends on the input. In an algorithm of unnormalized version of ηBoost, a weight distribution of AdaBoost is moderated by an uniform weight distribution and a way of combining a weak learners is adjusted by a naive error rate. As a result, ηBoost incorporates the effect of forgetfulness into AdaBoost. For both versions, a tuning parameter ηis associated with a degree of the contamination of the model and we can choose it by the minimization of naive error rate. We theoretically investigated the robustness of ηBoost and confirmed it with computer experiments. Also, we applied ηBoost to real datasets and compared it with previously proposed Boosting method. The ηBoost outperformed the other method in term of robustness.  
所蔵  
値  有 