Gradient Boosted Trees for Regression Explained

With video explanation | Data Series | Episode 11.5

Building Gradient Boosted Trees for Regression

To understand how gradient boosted trees for regression works, let us go through step-by-step how to build gradient boosted trees on the following fake test score data to predict a student’s test score.

1. Calculate the mean of our target variable, test score.

This forms our initial prediction:

2. Calculate the residuals

The residuals are given by the following formula:

3. Build a decision tree to predict the residuals

Here we use Weekly Physical Exercise and Previous Test Score/100 to predict Residual.