WEKO3
アイテム
Statistical Learning Theory by Boosting Method
https://ir.soken.ac.jp/records/763
https://ir.soken.ac.jp/records/763e7c7849a-9160-4d7c-9420-b07ac1923e06
| 名前 / ファイル | ライセンス | アクション |
|---|---|---|
要旨・審査要旨 / Abstract, Screening Result (281.7 kB)
|
| Item type | 学位論文 / Thesis or Dissertation(1) | |||||
|---|---|---|---|---|---|---|
| 公開日 | 2010-02-22 | |||||
| タイトル | ||||||
| タイトル | Statistical Learning Theory by Boosting Method | |||||
| タイトル | ||||||
| タイトル | Statistical Learning Theory by Boosting Method | |||||
| 言語 | en | |||||
| 言語 | ||||||
| 言語 | eng | |||||
| 資源タイプ | ||||||
| 資源タイプ識別子 | http://purl.org/coar/resource_type/c_46ec | |||||
| 資源タイプ | thesis | |||||
| 著者名 |
竹之内, 高志
× 竹之内, 高志 |
|||||
| フリガナ |
タケノウチ, タカシ
× タケノウチ, タカシ |
|||||
| 著者 |
TAKENOUCHI, Takashi
× TAKENOUCHI, Takashi |
|||||
| 学位授与機関 | ||||||
| 学位授与機関名 | 総合研究大学院大学 | |||||
| 学位名 | ||||||
| 学位名 | 博士（学術） | |||||
| 学位記番号 | ||||||
| 内容記述タイプ | Other | |||||
| 内容記述 | 総研大甲第739号 | |||||
| 研究科 | ||||||
| 値 | 数物科学研究科 | |||||
| 専攻 | ||||||
| 値 | 15 統計科学専攻 | |||||
| 学位授与年月日 | ||||||
| 学位授与年月日 | 2004-03-24 | |||||
| 学位授与年度 | ||||||
| 値 | 2003 | |||||
| 要旨 | ||||||
| 内容記述タイプ | Other | |||||
| 内容記述 | We deal with statistical learning theory, especially classification problems, by Boosting method. In the context of Boosting method, we can use only a set of weak learners which output statistical discriminant functions having low performance for a given set of examples. Aim of Boosting method is to construct a strong learner by combining a lot of weak learners and a typical boosting algorithm is AdaBoost. AdaBoost can be derived from a sequential minimization of the exponential loss function for a statistical discriminant function. This minimization problem is equivalent to the minimization of the extended Kullback-Leibler divergence between an empirical distribution of given examples and an extended exponential model. Statistical properties of AdaBoost have been investigated and the relationship between the exponential loss function of AdaBoost and the logistic model was revealed. In this thesis, we obtain two main results:<br /><br />1. AdaBoost is extended to general U-Boost by using the statistical form of the Bregman divergence, which contains the Kullback-Leibler divergence as an example and consider a geometrical interpretation of U-Boost in terms of information geometry.<br /><br />2. We propose a new Boosting algorithm η-Boost, which is a robustified version of AdaBoost.<br /><br />The U-Boost is derived from a sequential minimization of the Bregman divergence between the empirical distribution and U-model. A geometric interpretation for U-Boost is given in terms of information geometry. From the Pythagorean relation associated with the Bregman divergence, we derive two special versions of U-Boost, the normalized U-Boost and the unnormalized U-Boost. We define the normalized version of U-model on the probability space and derive normalized U-Boost from this model. The normalized U-Boost corresponds to usual statistical classification methods, for example, logistic discriminant analysis. The unnormalized U-Boost is derived from an unnormalized version of U-model defined on the extended non-negative measure space and has not been seen in the previous statistical context. Especially, unnormalized U-Boost has a beautiful geometrical structure related to the Pythagorean relation and the flatness. Its algorithm is interpreted as a pile of right triangles which leads to a mild convergence property of U-Boost algorithm as seen in the EM algorithm. Based on a probabilistic assumption for a training data set, statistical discussion for consistency, efficiency and robustness of U-Boost is given. <br /><br />An algorithm of AdaBoost implements the learning process by exponentially reweighting examples according to classification results. Then weight distribution is often too sharply tuned, so that AdaBoost has a weak point on the robustness and over-learning. As a special example of U-Boost, we propose η-Boost which aims to robustify AdaBoost to avoid an over-learning. The statistical meaning of η-Boost is discussed and η-Boost is associated with a probabilistic model of mislabeling which is a contaminated logistic model. As a general U-Boost algorithm, η-Boost also has a normalized and unnormalized version. A loss function of the normalized version of η-Boost is a minus log-likelihood of a contaminated logistic model in which mislabeling probability is constant and does not depend on the input. The unnormalized version of η-Boost is a slight modification of AdaBoost and is derived from a loss function which is defined by a mixture of the exponential loss of AdaBoost and naive error loss functions. A probabilistic model of unnormalized version is also a contaminated logistic model and its mislabeling probability depends on the input. In an algorithm of unnormalized version of η-Boost, a weight distribution of AdaBoost is moderated by an uniform weight distribution and a way of combining a weak learners is adjusted by a naive error rate. As a result, η-Boost incorporates the effect of forgetfulness into AdaBoost. For both versions, a tuning parameter ηis associated with a degree of the contamination of the model and we can choose it by the minimization of naive error rate. We theoretically investigated the robustness of η-Boost and confirmed it with computer experiments. Also, we applied η-Boost to real datasets and compared it with previously proposed Boosting method. The η-Boost outperformed the other method in term of robustness. | |||||
| 所蔵 | ||||||
| 値 | 有 | |||||
