GBM

Full name Microsoft.VisualBasic.MachineLearning.XGBoost.train.GBM Assembly Microsoft.VisualBasic.MachineLearning.XGBoost Members 2

Tiny implement of Gradient Boosting tree

It is a Tiny implement of Gradient Boosting tree, based on XGBoost's scoring function and SLIQ's efficient tree building algorithm. TGBoost build the tree in a level-wise way as in SLIQ (by constructing Attribute list and Class list). Currently, TGBoost support parallel learning on single machine, the speed and memory consumption are comparable to XGBoost.

TGBoost supports most features As other library:

Built-in loss , Square error loss for regression task, Logistic loss for classification task
Early stopping, evaluate On validation Set And conduct early stopping
Feature importance, output the feature importance after training
Regularization , lambda, gamma
Randomness, subsample，colsample
Weighted loss Function , assign weight To Each sample

Another two features are novel:

Handle missing value, XGBoost learn a direction For those With missing value, the direction Is left Or right. TGBoost take a different approach: it enumerate missing value go To left child, right child And missing value child, Then choose the best one. So TGBoost use Ternary Tree.
Handle categorical feature, TGBoost order the categorical feature by their statistic (Gradient_sum / Hessian_sum) On Each tree node, Then conduct split finding As numeric feature.

00 Remarks

https://github.com/wepe/tgboost

01 Syntax

Microsoft.VisualBasic.MachineLearning.XGBoost.train.GBM

02 Methods

Name	Overloads	Summary
.ctor	1	load model from file
fit	1	do model training

03 Members

method .ctor #

#ctor(List(Of Tree), Loss, Double, Double)

load model from file

Parameters

Name	Type	Description
`trees`	`List(Of Tree)`	-
`loss`	`Loss`	-
`first_round_pred`	`Double`	-
`eta`	`Double`	-

method fit #

fit(TrainData, ValidationData, Int32, Boolean, Metrics, String, Double, Int32, Int32, Double, Double, Double, Double, Int32, Double, Double, Int32)

do model training

Parameters

Name	Type	Description
`trainset`	`TrainData`	-
`valset`	`ValidationData`	-
`early_stopping_rounds`	`Int32`	-
`maximize`	`Boolean`	-
`eval_metric`	`Metrics`	`Metrics.mse` for regression problem
`loss`	`String`	logloss: `LogisticLoss` for classify problem squareloss: `SquareLoss` for regression problem
`eta`	`Double`	[learning_rate] Step size shrinkage used in update to prevents overfitting. After each boosting step, we can directly get the weights of new features, and eta shrinks the feature weights to make the boosting process more conservative. range: [0,1]
`num_boost_round`	`Int32`	-
`max_depth`	`Int32`	Maximum depth of a tree. Increasing this value will make the model more complex and more likely to overfit. 0 indicates no limit on depth. Beware that XGBoost aggressively consumes memory when training a deep tree. exact tree method requires non-zero value. range: [0,∞]
`scale_pos_weight`	`Double`	-
`rowsample`	`Double`	-
`colsample`	`Double`	-
`min_child_weight`	`Double`	Minimum sum of instance weight (hessian) needed in a child. If the tree partition step results in a leaf node with the sum of instance weight less than min_child_weight, then the building process will give up further partitioning. In linear regression task, this simply corresponds to minimum number of instances needed to be in each node. The larger min_child_weight is, the more conservative the algorithm will be. range: [0,∞]
`min_sample_split`	`Int32`	-
`lambda`	`Double`	[reg_lambda] L2 regularization term on weights. Increasing this value will make model more conservative.
`gamma`	`Double`	[min_split_loss] Minimum loss reduction required to make a further partition on a leaf node of the tree. The larger gamma is, the more conservative the algorithm will be. range: [0,∞]
`num_thread`	`Int32`	-