12 Ensemble Decision Tree Models

12.1 Overcoming Limitations of a Single Decision Tree

A decision tree recursively splits the data into increasingly homogeneous groups, both for regression models and classification models. Decision tree models are among the most interpretable machine learning models, easily explainable to managers without statistical background. Further, they can represent nonlinear relationships and interactions among predictors without requiring the analyst to identify and specify those patterns in advance.

However, there are limitations discussed of a decision tree model that have led to the development of more general decision tree models called ensemble models, the primary topic of this section.

12.1.1 Overfitting

As discussed earlier in the general treatment of overfitting and the decision-tree discussion, any predictive model can overfit the training data, but decision trees are especially susceptible. A decision tree overfits when it learns details specific to the training sample rather than patterns that generalize to new data. If allowed to grow too deeply, the tree may continue splitting until it captures minor irregularities, unusual observations, or random noise.

For example, a tree may discover a sequence of splits that predicts the training data well but does not represent a stable pattern in the broader population. The model may appear highly accurate when evaluated on the same observations used to construct it, yet perform substantially worse on new data. A large difference between training and test performance is a warning sign of overfitting.

Overfitting is especially easy to induce in a decision tree because almost every additional split improves its fit to the training data. As the tree grows, its terminal nodes become smaller and more homogeneous. Beyond some point, however, the added detail no longer represents general structure. Instead, the tree begins to memorize the particular training sample.

12.1.2 Instability and High Variance

A related limitation of a single decision tree is instability. A small change in the training data can produce a substantially different tree. One sample may lead the tree to split first on one predictor, whereas a slightly different sample may lead it to split first on another. Because each split determines the observations available for subsequent splits, a change near the top of the tree can alter much of the structure that follows.

This instability reflects high variance. In this context, high variance means that the fitted model is highly sensitive to the particular sample used for training. The tree learns not only the general relationship between the predictors and the target variable, but also some of the incidental characteristics of that sample. As a result, a single tree may be easy to explain yet produce predictions that change noticeably when fitted to a new sample or to a slightly modified version of the original data.

12.1.3 Ensemble Methods: From One Tree to Many Trees

To address the limitations of a single decision tree, ensemble methods combine the predictions from many trees. Two widely used examples are random forests and XGBoost. Both methods build on the decision tree, but they generally produce more accurate and stable predictions than a single tree. Understanding how one decision tree works provides the foundation for understanding these more advanced methods.

These combined models are called ensemble models.

Ensemble model: A predictive model that combines the predictions from multiple component models to produce a single prediction that is usually more accurate and stable.

The main idea is simple: many imperfect models can often be combined to form a stronger model. The individual trees do not need to be highly accurate on their own. Instead, each tree contributes information to the final prediction. Random forests and XGBoost both combine many decision trees, but they do so in different ways. A random forest builds many trees largely independently and combines their predictions through averaging or voting. XGBoost builds trees sequentially, with each new tree designed to reduce the prediction errors left by the existing ensemble.

Historically, the concept of a decision tree was invented in the early 1960s, but it was in the 1980s that decision trees became established regression and classification methodologies. The development of ensemble methods applied to decision trees began in the mid-1990s to early 2000s. Leo Breiman introduced bagging in 1996 and the random forest in 2001, drawing in part on Tin Kam Ho’s earlier work on randomly selecting predictors. Tianqi Chen and Carlos Guestrin described XGBoost in 2016. By the 2010s, better software, faster computing, larger datasets, and perceived success of achieving better predictive models moved tree ensembles from research into routine organizational practice. Python users could apply random forests directly through scikit-learn beginning around 2011. XGBoost arrived several years later as an independent package, with a scikit-learn-compatible interface available by about 2015 and widespread adoption accelerating around 2015–2016.

12.2 Random Forests: Many Decision Trees Working Together

12.2.1 Definition

A random forest is an ensemble model composed of many decision trees. Like a single decision tree, each tree makes a prediction by directing an observation through a sequence of splits until it reaches a terminal leaf node. The important difference is that a random forest does not rely on the prediction from only one tree. Instead, it combines the predictions from many trees to produce a final result.

For a regression problem, the forest usually averages the numerical predictions from all the trees. For a classification problem, each tree predicts a category, and the forest usually selects the category that receives the most votes.

The word forest refers to the many trees in the model. The word random refers to the random procedures used to make those trees different from one another. Each tree is trained on a different random sample of the training observations. In addition, each split considers only a random subset of the available predictor variables. These two sources of randomness together produce a collection of trees that do not all make the same splits or the same prediction errors.

12.2.2 How Random Forests Work

12.2.2.1 Constructing Many Trees

The instability of a single decision tree helps explain the value of a random forest. Because one tree may depend heavily on the particular training sample, a small change in the data can produce a different sequence of splits and different predictions.

A random forest reduces this sensitivity by building many trees and combining their predictions. Its strength comes from using trees that are individually useful but not identical. Each tree may capture somewhat different patterns in the data or perform better for different observations. Averaging or voting across the trees limits the influence of the errors made by any one tree and produces a more stable overall prediction.

Differences among the trees are essential. If every tree were built from the same observations and considered the same predictors at each split, the trees would be nearly identical. Combining many identical trees would add little value. A random forest instead creates a diverse collection of trees and combines their predictions into a single result.

This design leads to a result that may at first seem backward. The individual trees in a random forest are usually grown deep and are not pruned. Each tree is allowed to develop the very instability described earlier for a single decision tree. The forest tolerates that instability because averaging is what removes it. A deep tree captures detailed structure in the data but is highly sensitive to its particular training sample. Averaging many such trees retains the structure they have in common and cancels much of the sample-specific detail they do not share. The protection against overfitting in a random forest comes primarily from combining the trees rather than from restricting any one of them.

12.2.2.2 Two Sources of Randomness

A random forest creates differences among its trees through two sources of randomness.

The first source is the set of observations used to train each tree. Each tree is fitted to a different random sample of the original training data. This process is called bagging, short for bootstrap aggregating.

Bagging (bootstrap aggregating): An ensemble method that builds multiple models from different bootstrap samples of the training data and then combines their predictions.

A bootstrap sample is created by randomly sampling observations from the training data with replacement. Because sampling is performed with replacement, an observation may appear more than once in a particular sample, while another observation may not appear at all. Each tree is trained on a somewhat different collection of observations.

The second source of randomness concerns the predictors considered at each split. A tree does not evaluate every available predictor whenever it searches for the next split. Instead, it evaluates only a random subset of the predictors.

This restriction helps prevent the same strong predictor from dominating every tree. For example, if all predictors were considered at every split, many trees might choose the same especially powerful predictor near the top and develop similar structures. Restricting each split to a random subset of predictors encourages the trees to follow different sequences of splits and capture different patterns in the data.

The reason this restriction matters follows from how averaging works. Averaging reduces error only to the extent that the errors are independent of one another. Trees that make the same mistakes on the same observations average to nearly the same result as any one of them, so little is gained. The value of the forest depends on the trees being genuinely different, and considering only a random subset of predictors at each split is what keeps a single dominant predictor from making them alike.

The number of predictors considered at each split is a model setting the analyst can adjust, named max_features in scikit-learn. A traditional guideline is the square root of the number of available predictors for classification and one third of them for regression, although these values are starting points rather than requirements. The default for RandomForestClassifier follows the square-root guideline, but the default for RandomForestRegressor is to consider every predictor at each split. A regression forest left at its defaults therefore draws on only one source of randomness rather than two, the bootstrap samples, and its trees are correspondingly more alike. Setting max_features explicitly restores the second source.

Together, these two sources of randomness produce a collection of related but distinct trees. The trees are trained on different bootstrap samples, and their splits are selected from different random subsets of predictors. Combining their predictions reduces the influence of any one tree and helps lower the high variance associated with a single decision tree.

12.2.2.3 Combining the Trees

After the trees are built, their predictions are combined. For a regression problem, suppose the forest contains 500 trees. A new observation is passed through each tree, producing 500 numerical predictions. The random forest prediction is usually the average of those values. For a classification problem, each tree assigns the observation to a category, also called a class. The forest then counts the votes for each class, and the class receiving the most votes becomes the final prediction.

Most implementations refine this procedure slightly. Rather than counting one vote per tree, they average the class probabilities estimated by the individual trees, which uses more of the information each tree provides. Scikit-learn’s random forest classifier works this way. The practical consequence is that a random forest returns an estimated probability of class membership, not only a predicted class. That probability is often the more useful output for a business decision. A model that estimates the probability that a customer will leave allows the analyst to rank customers by risk and to set a threshold for action according to the cost of intervening and the cost of losing the customer, rather than accepting whatever threshold the software applies by default.

In either case, the prediction reflects the collective result from many trees rather than the result from any one tree. Averaging or voting limits the influence of an individual tree that makes an unusual prediction. Because the trees are different from one another, some of their errors may offset one another, usually producing a more stable prediction than a single decision tree.

12.2.3 Advantages

Random forests often perform well with relatively little tuning. Some machine learning methods require extensive adjustment of their model settings, whereas a random forest can frequently produce strong results with default or near-default values. The analyst can still tune parameters such as the number of predictors considered at each split and the maximum depth of the trees. Even before extensive tuning, however, random forests often provide dependable predictive performance.

The number of trees is a setting of a different kind. Adding trees to a random forest does not cause overfitting. Because the trees are built independently and their predictions are averaged, the predictions become more stable as trees are added and eventually settle down, after which further trees change the result very little. The number of trees is chosen for stability and computational cost rather than balanced against the risk of overfitting. Enough trees are needed for the predictions to stabilize, and beyond that point additional trees mainly consume time and memory. This property distinguishes a random forest from the boosting methods described later, in which adding trees is one of the ways a model can be pushed into overfitting.

A random forest also provides its own estimate of predictive accuracy without requiring a separate holdout sample. Because each tree is fitted to a bootstrap sample, roughly a third of the observations are left out of any particular tree. These are called the out-of-bag observations for that tree. Each observation can be predicted using only the trees that did not see it during training, and those predictions can be compared to the observed values.

Out-of-bag error: An estimate of predictive accuracy computed from the observations excluded from each bootstrap sample, using only the trees that were not trained on them.

The out-of-bag error is a useful check because it is obtained as a byproduct of fitting the model. It is particularly convenient when the data set is small enough that setting aside a separate validation sample is costly. It does not replace a final test set held out from the entire modeling process, but it provides an inexpensive early indication of how well the forest predicts observations it has not seen.

Random forests can also represent complex relationships among variables. A linear regression model represents predictors through a specified equation, usually based on additive and linear effects unless the analyst explicitly adds nonlinear terms or interactions. Decision trees can discover nonlinear and conditional patterns through their sequence of splits. For example, the relationship between income and purchasing behavior may differ across age groups, or the relationship between square footage and home price may depend on the neighborhood. A random forest extends this flexibility by combining many trees that capture different patterns in the data.

Random forests can also provide measures of predictor importance. These measures summarize how much each predictor contributes to the model’s predictions across the collection of trees. A predictor that repeatedly helps create useful splits may receive a higher importance value than one that contributes little. Predictor importance can help identify which variables the fitted model relies on most strongly, although it does not by itself establish that those variables have a causal effect on the target.

12.2.4 Tradeoffs

The main tradeoff of a random forest is reduced interpretability. A single decision tree can be drawn and explained as a sequence of rules, but no one tree represents an entire random forest. A forest may contain hundreds or thousands of trees, making it impractical to display the complete model in one diagram. The general logic remains intuitive—build many different trees and combine their predictions—but the full fitted model is less transparent.

Random forests also require more computation than a single tree or a linear model. A random forest must grow many trees, store their results, and combine their predictions. Modern computers can usually perform these tasks efficiently for standard business and data science applications, but the additional computation is part of the cost of obtaining greater predictive stability. The fitted model is also a large object. Hundreds of deep trees must be stored and then traversed each time a prediction is made, which can matter when a model is deployed in a system that scores many records or must respond quickly.

A random forest is also unable to predict outside the range of the target values it observed during training. Every prediction is an average of values taken from the training data, so a regression forest cannot produce a prediction higher than the highest value it has seen or lower than the lowest. If home prices in the training data reach two million dollars, the forest cannot predict three million, however extreme the predictors of a new property may be. Instead, its predictions flatten out near the edge of the observed range. A linear regression model behaves differently, because its fitted line continues indefinitely in both directions. This limitation is easy to overlook and matters whenever a model is applied to conditions more extreme than those in the data used to build it.

Feature-importance measures should also be interpreted carefully. They can indicate which predictors the fitted forest relies on most strongly, but they do not provide the same direct interpretation as regression coefficients. For example, a regression coefficient may indicate that, holding the other predictors constant, a one-unit increase in a predictor is associated with a specific average change in the target variable. A random forest does not provide one equation or one slope coefficient that summarizes a predictor’s effect. Its predictions result from many conditional splits across many trees. The analyst may know that a predictor is important to the model without being able to describe its effect with one simple numerical statement.

12.2.5 Revisiting the Limitations

This chapter began with some limitations of a single decision tree. A random forest addresses them to different degrees, and the differences are worth stating directly.

Instability is the limitation the method addresses most completely. A single tree may change substantially when the training data change slightly. A forest averages across many trees fitted to different bootstrap samples, so the influence of any one sample is limited and the predictions vary far less from one sample to the next.

Overfitting is reduced but not eliminated. As described above, the individual trees are grown deep and are allowed to overfit their own bootstrap samples. Averaging cancels much of that sample-specific detail, which is why a random forest generalizes better than any of the trees it contains. The forest can still overfit when the data are noisy or the sample is small, so its performance must still be evaluated on data not used to build it.

12.2.6 Summary

A random forest is an ensemble method that improves on a single decision tree by building many trees and combining their predictions. Each tree is trained on a different bootstrap sample of the training data, and each split considers a random subset of the available predictors. These two sources of randomness make the trees different from one another.

For regression, the trees’ predictions are averaged. For classification, their predictions are combined by voting, or in most software by averaging the class probabilities the trees estimate. The resulting model is usually more stable and more accurate on new data than a single decision tree. Random forests are widely used because they combine flexibility, strong predictive performance, and relative ease of use.

12.3 XGBoost: Decision Trees That Learn From Their Mistakes

12.3.1 Definition

XGBoost is an ensemble method that builds many decision trees sequentially. Instead of relying on one tree, it adds trees one at a time, with each new tree designed to improve the predictions produced by the trees already in the model.

Boosting: An ensemble method in which models are trained sequentially, with each new model designed to reduce the errors left by the existing ensemble.

Each new model depends on the performance of the models already constructed. The improvement from one model to the next is directed by a gradient, which identifies the direction in which the errors decrease most rapidly. Boosting that proceeds in this way is called gradient boosting, the form of boosting that XGBoost implements. XGBoost, short for Extreme Gradient Boosting, is a widely used and computationally efficient implementation.

12.3.2 How XGBoost Works

XGBoost begins with an initial prediction and then builds a sequence of trees that gradually improve that prediction. The initial prediction is not a tree but a single constant value, ordinarily the mean of the target variable for a regression problem. Every observation receives that same starting prediction. After each stage, the algorithm evaluates the errors that remain and constructs another tree to help reduce them.

For a regression analysis, the errors can be understood as differences between the observed and predicted values, the residuals. For classification, the same general logic applies, although the algorithm evaluates error through a loss function of the classification errors rather than through ordinary numerical residuals. In both cases, the remaining errors provide information about patterns the current model has not yet learned.

Because that initial constant is the same for every observation, it is wrong for nearly all of them. Some predictions are too high and others too low. The first tree searches for patterns in the predictor variables that help explain those errors. Its prediction is not used independently. Instead, its contribution is added to the prediction the model has already made.

The contribution from each new tree is usually reduced by a learning rate. Rather than allowing one tree to make a large correction, the learning rate causes the model to improve through smaller steps. A learning rate of 0.1, for example, adds only one tenth of the correction the tree recommends. A second tree is then built to reduce the errors that remain after the constant and the first tree have been combined. This process continues for a specified number of rounds or until additional trees provide little improvement.

The learning rate and the number of trees must be chosen together, because one compensates for the other. A smaller learning rate takes smaller steps, so more trees are required to travel the same distance. A rate of 0.01 may need several thousand trees to reach what a rate of 0.3 reaches in a few hundred. Neither setting is meaningful without the other. Reporting that a model uses 1,000 trees says little until the learning rate is also known.

What matters is the pairing, not the size of the learning rate by itself. A small rate is not better in its own right. Paired with too few trees it produces a model that underfits, because the sequence stops before it has traveled far enough to reach the structure in the data. Nor does lowering the rate improve predictions indefinitely. Reducing it from 0.3 to 0.1 often yields a visible gain, whereas reducing it from 0.1 to 0.01 usually yields a slight one at ten times the computation.

Each individual tree in a boosting model is small by design. Where the trees in a random forest are grown deep and left unpruned, the trees in a boosting model are usually restricted to a depth of roughly three to six splits. Such a tree is called a weak learner.

Weak learner: A model that performs only somewhat better than chance or than a simple baseline, used as a component of a boosting ensemble.

The restriction is a design choice rather than a compromise. Because each tree contributes a small correction that the learning rate shrinks further, no single tree can pull the model far in a wrong direction, and the ensemble improves through many small adjustments rather than a few large ones. Tree depth also governs how complicated a relationship the model can represent. A tree limited to three splits can combine at most three predictors along any path from the root, so it can capture interactions among three variables but no more. Increasing the depth allows more elaborate interactions, and also increases the risk that the model will fit accidental combinations that appear only in the training sample. Depth is one of the most consequential settings in a boosting model.

12.3.2.1 Regularization

Regularization is a central feature of XGBoost.

Regularization: A penalty added to the quantity a model minimizes, discouraging complexity that is not justified by a sufficient improvement in fit.

Without regularization, an algorithm seeks only to reduce residuals from applying the model to the training data. Adding complexity almost always reduces error on the training data. Regularization changes what is being minimized. XGBoost adds to the loss function a penalty for the number of leaves in a tree and a penalty for the size of the adjustments made at those leaves. A tree that adds several leaves for a small improvement in fit now increases the penalized objective rather than decreasing it, so the algorithm declines to grow it. Added complexity must be earned by a reasonable increase in fit.

This mechanism is a formal version of a familiar idea. A tree is not permitted to add detail simply because the detail helps on the training data. It must help enough to outweigh the cost the penalty assigns to the added complexity. The analyst controls how severe that cost is according to the tuning parameters that can be adjusted to specify the amount of regularization.

12.3.2.2 Randomness in Boosting

Although boosting and bagging are usually presented as opposites, XGBoost borrows from bagging as well. The analyst can direct each tree to be built from a random subset of the training observations and from a random subset of the predictors, much as a random forest does. Using a random portion of the data for each tree makes the trees less similar and reduces the risk of fitting patterns peculiar to the full training sample. However, the essential difference remains that a random forest builds its trees independently, whereas XGBoost builds each tree in response to the errors of those already constructed.

12.3.3 Advantages

The main advantage of XGBoost is its potential for strong predictive accuracy. By combining many small decision trees, the model can capture complex patterns that may be missed by simpler methods. Each tree contributes an additional correction to the existing ensemble, allowing predictive performance to improve gradually across the sequence.

XGBoost can represent nonlinear relationships and interactions among predictors without requiring the analyst to specify those patterns in advance. This flexibility is especially useful for structured business data, in which the relationship between one predictor and the target may depend on the values of other predictors. For example, the relationship between income and purchasing behavior may differ across age groups, or the relationship between prior customer activity and churn may depend on the type of product or service.

XGBoost also provides several mechanisms for controlling overfitting. The analyst can limit the depth and complexity of the individual trees, reduce the contribution of each new tree through the learning rate, apply regularization, and use early stopping to end training when performance on validation data no longer improves. These safeguards do not eliminate overfitting, but they provide considerable control over the complexity of the fitted model.

Early stopping deserves particular attention, because it answers a question the analyst would otherwise have to guess at: how many trees to build.

Early stopping: Ending the sequence of boosting rounds when performance on a validation sample has failed to improve for a specified number of consecutive rounds.

The procedure sets aside a validation sample that is not used to fit the trees, or simulate a formal validation sample by applying cross-validation of the training data. After each boosting round, the current ensemble is evaluated on that sample. Early in training, performance on the validation sample improves along with performance on the training data. At some point the two diverge: the model continues to fit the training data more closely while its predictions on the validation sample stop improving and may begin to deteriorate. That divergence marks the point at which the model has begun to learn features of the training sample rather than general structure. Training halts, and the number of trees from the best round is retained.

Early stopping is convenient because the analyst can specify a generous number of rounds and allow the algorithm to determine how many are actually useful. It also resolves the difficulty raised earlier, that the learning rate and the number of trees have to be chosen in relation to each other. The practical approach is to fix the learning rate at a moderate value, commonly between 0.05 and 0.1, and let early stopping determine the number of trees. The number of trees then follows from the data rather than from a guess, and the two settings are matched without the analyst having to balance them by hand.

Early stopping carries one caution. Because the validation sample influences the choice of the number of trees, it has participated in building the model and no longer provides an unbiased assessment of performance. A separate test set, untouched throughout model development, is still required for the final evaluation.

A further practical advantage is that XGBoost accepts missing values directly. Business data frequently contain them, whether an unreported income, a blank survey item, or a field that does not apply to every customer. Many methods require the analyst to fill in such values before fitting, and any choice of filled-in value is an assumption imposed on the data. XGBoost instead treats missingness as information. At each split, it learns which direction the observations with a missing value should take, choosing whichever assignment better reduces the loss. If customers who decline to report income behave like high-income customers, the algorithm can discover this and route them accordingly.

12.3.4 Tradeoffs

The predictive power of XGBoost comes with additional complexity. XGBoost has more tuning parameters than a single decision tree or a standard regression model. The analyst can control the number of trees, the depth of each tree, the learning rate, and the strength of regularization. These settings can substantially affect performance. A poorly tuned model may overfit the training data or provide little improvement over a simpler method.

As always, consider the bias–variance tradeoff. A model that is too simple may have high bias because it cannot represent important patterns in the data. A highly flexible model may reduce bias by capturing more complicated relationships, but it may also have higher variance and become too sensitive to the particular training sample.

The objective is not to fit the training data as closely as possible, but to obtain predictions that generalize well to new data.

Careful model evaluation and tuning help manage this tradeoff. Training and test sets, cross-validation, regularization, limits on tree complexity, and early stopping can all help determine whether additional boosting improves prediction on new data. These methods reduce the risk of overfitting, but they do not remove the need for thoughtful model development.

Boosting is also more sensitive than a random forest to unusual or erroneous observations. Each new tree is directed toward the observations the model currently predicts worst, so an observation with a mistaken target value, a data-entry error, or a genuinely extreme value attracts repeated attention. Successive trees may devote much of their capacity to accommodating a small number of observations that do not represent any general pattern. A random forest treats such observations as a few values among many in an average and is affected far less. Careful examination of the data before fitting matters more for boosting than for bagging, and a low learning rate together with a limit on tree depth restrains how far the model can go in pursuing them.

Another tradeoff is reduced interpretability. A single decision tree can be drawn and followed from its root node to a terminal leaf node. XGBoost may contain hundreds or thousands of small trees whose contributions are combined into one prediction. Feature-importance measures and other interpretation tools can help explain the fitted model, but they do not provide the same transparency as a single tree or the direct coefficient interpretation of a regression model.

For a business application, the most appropriate model is not necessarily the newest or most complex one. XGBoost is a strong candidate when predictive accuracy is the primary objective and the data contain complex patterns. A simpler regression model or decision tree may be preferable when transparency, ease of explanation, limited data, or implementation requirements are especially important. The practical goal is to choose a model that predicts new data well and is appropriate for the decision in which it will be used.

12.3.5 Revisiting the Limitations

The limitations of a single decision tree can be reconsidered for boosting as they were for the random forest, and the answers differ noticeably.

Overfitting is the limitation boosting handles most deliberately, and also the one it most readily creates. A random forest resists overfitting largely as a consequence of averaging, without the analyst doing anything in particular. XGBoost provides an assortment of instruments for the purpose, including the learning rate, limits on tree depth, regularization, subsampling, and early stopping, but they must be used. Left unrestrained, a boosting model will continue adding trees that fit the training data ever more closely, because reducing the remaining error is precisely what the algorithm is built to do. Control over overfitting in XGBoost is available but not automatic.

Instability is reduced, though by a different route. A boosting model does not average away the peculiarities of a sample. It relies instead on the learning rate to keep any single tree from imposing much influence, so that the fitted model reflects the accumulation of many small corrections rather than a few decisive ones. The result is more stable than a single tree, but a boosting model remains more sensitive than a random forest to unusual observations, for the reason described above.

12.3.6 Summary

A single decision tree builds one tree to make predictions. XGBoost builds many trees sequentially, with each new tree helping correct what the existing ensemble does not yet predict well. This step-by-step improvement is the central idea of boosting and a major source of XGBoost’s predictive power.

12.4 Model Comparison and Practical Guidance

12.4.1 Random Forests vs. XGBoost

Underlying the difference in construction of Random forests and XGBoost models is a difference in purpose. The two methods begin with opposite kinds of component trees and correct opposite deficiencies. A random forest begins with trees that are individually accurate but unstable, and averaging reduces that instability, so bagging addresses variance. XGBoost begins with trees that are individually too simple to predict well. The accumulated boosting sequence gradually builds a model flexible enough to capture the structure in the data, so boosting addresses bias. Recognizing which kind of error a method is designed to reduce explains much of how the two behave in practice.

This distinction affects how the methods are used in practice. Random forests are generally more forgiving and less sensitive to tuning. They often provide strong predictive performance with default or near-default settings, making them a natural first ensemble method after learning about a single decision tree. A well-performing random forest also provides a useful standard against which more extensively tuned models can be compared.

XGBoost is often considered when predictive accuracy is the highest priority and the analyst is willing to devote more attention to tuning and validation. Parameters such as the learning rate, number of trees, tree depth, and regularization can substantially affect the results. With thoughtful tuning, XGBoost may outperform a random forest. Without careful tuning, however, it may provide little improvement or may overfit the training data.

Both methods sacrifice some of the simplicity and interpretability of a single decision tree. In return, they often produce more accurate and stable predictions. Both are especially useful for structured, tabular data, in which observations are stored as rows and variables as columns. Customer records, employee data, financial transactions, loan applications, product information, and sales histories are common examples.

Tree-based ensembles work well with this type of data because they can represent nonlinear relationships and interactions without requiring the analyst to specify those patterns in advance. Random forests are often preferred when the goal is a strong and dependable model with limited tuning. XGBoost is often preferred when the analyst wants to determine whether careful boosting and tuning can improve predictive performance further.

The differences discussed in this chapter are collected in Table 12.1, along with the setting that governs each one. The names are those of RandomForestClassifier and RandomForestRegressor in scikit-learn and of XGBClassifier and XGBRegressor, the estimators through which XGBoost is ordinarily used from Python.

Table 12.1: Random forests and XGBoost compared, with the setting that controls each.

	Random forest	XGBoost
Trees built	Independently, in any order	Sequentially, each in response to the errors of those before it
Number of trees	n_estimators. Predictions stabilize as trees are added, with no added risk of overfitting	n_estimators. Fit to the training data keeps improving, so overfitting becomes possible
Individual tree	Deep and unpruned, as max_depth is unset by default	Shallow, ordinarily three to six splits, through max_depth
Error addressed	Variance	Bias
Sources of randomness	Bootstrap samples through bootstrap and max_samples; predictors per split through max_features	Optional rather than intrinsic, through subsample and colsample_bytree
Contribution of each tree	Every tree counts equally in the average	Scaled down by learning_rate, so smaller steps require more trees
Tuning required	Little; defaults are often adequate	Substantial; learning_rate and n_estimators must be chosen together
Overfitting control	Largely automatic, through averaging	Deliberate, through learning_rate, max_depth, min_child_weight, the regularization settings reg_lambda, reg_alpha, and gamma, and early_stopping_rounds
Unusual observations	Diluted by averaging	Attract repeated attention from later trees
Missing values	Require imputation before fitting	Accepted directly, the value named by missing; a direction is learned at each split
Built-in validation	Out-of-bag error, through oob_score	Early stopping, through early_stopping_rounds with an eval_set
Computation	Trees can be built simultaneously; n_jobs sets the cores used	Trees must be built in order; n_jobs parallelizes only within a tree
Reproducibility	random_state	random_state

12.4.2 In Practice

Before even fitting the simplest fitted model, establish what can be achieved with no model at all. For a quantitative target, predict the mean of the target for every observation. For a classification problem, predict the most common category for every observation, the baseline classification described earlier. Any fitted model should be judged against that reference. The comparison is particularly important for classification when one category is much more common than the other, a frequent situation in business data. If only 4% of customers default on a loan, a model that predicts no default for everyone is 96% accurate while identifying not one of the customers the analysis was undertaken to find. Recording the accuracy of the trivial prediction at the outset prevents a model that has learned nothing from appearing successful.

In applied work, begin model construction with the simplest reasonable model and use its performance as a baseline for evaluating more complex methods. For a regression problem with a quantitative target variable, ordinary least-squares regression is often a reasonable starting point. For a binary classification problem, logistic regression is often appropriate. These models are fast to estimate, relatively easy to explain, and useful for establishing how well a straightforward model predicts the target variable.

Before concluding that a more complex method is necessary, ask whether a simpler model already predicts well enough and use its performance as a baseline for comparison.

A baseline is useful both practically and conceptually. Least-squares regression and logistic regression require relatively little computation and represent the relationship between the predictors and the target with one fitted equation. If a simple model predicts nearly as well as a more complex model, it may be preferable because it is easier to explain, diagnose, and implement.

A single decision tree provides a logical next step. Unlike a standard linear model, a tree can represent nonlinear relationships and interactions without requiring the analyst to specify them in advance. Its predictions can also be explained as a sequence of decisions from the root node to a terminal leaf node. However, one tree may be unstable and may overfit the training data.

A random forest addresses much of this instability by combining many different decision trees. It is often a useful first ensemble model because it can provide strong predictive performance without extensive tuning. A fitted random forest also establishes a meaningful standard against which more complex ensemble methods can be compared.

XGBoost may then be used to determine whether sequential boosting can improve predictive accuracy further. It often performs well with structured, tabular data, but generally requires more attention to tuning, validation, and overfitting than a random forest.

XGBoost is available from its own package. Scikit-learn (sklearn) also provides its own gradient-boosting estimators, including GradientBoostingClassifier and HistGradientBoostingClassifier. Two further implementations are installed from separate packages. LightGBM, from the lightgbm package, is often faster than XGBoost on large data sets. CatBoost, from the catboost package, can work especially well with categorical predictors. All of these methods implement the same underlying idea, and the choice among them ordinarily rests on speed, convenience, and the character of the data rather than on any fundamental difference in approach.

This sequence is useful both pedagogically and analytically:

0. Establish the baseline level before any model is constructed
1. Fit a simple regression or logistic regression model.
2. Fit a single decision tree.
3. Fit a random forest composed of many independently constructed trees.
4. Fit XGBoost, in which trees are added sequentially to improve the existing ensemble.

Each step adds flexibility but also introduces additional complexity. Regression and logistic regression are fast and interpretable but may miss nonlinear patterns and interactions. A single tree can represent those patterns but may be unstable. A random forest reduces that instability by combining many trees. XGBoost may improve predictive accuracy further, but it usually requires more tuning and is less transparent.

12.4.2.1 Preparing the Data

The four models in this sequence do not all require the data prepared in the same way, a point easily overlooked when moving from one step to the next.

Predictor variable measurement scale. Regression models are sensitive to the scale and shape of the predictors. Tree-based models are not. A tree searches for a value at which to divide a predictor, and that division is unaffected by the units in which the predictor is expressed. Converting dollars to thousands of dollars, or replacing a variable by its logarithm, changes where the split point falls but not which observations end up on either side of it, so the fitted tree makes the same predictions. Standardizing predictors, which can matter a great deal for other methods, accomplishes nothing for a tree. Neither do the transformations discussed for regression, which are applied to satisfy the assumptions of a linear model that a tree does not make.

Categorical predictors. Categorical predictors require attention in the opposite direction. Scikit-learn’s tree-based estimators require numeric input, so a categorical predictor must be converted to indicator (dummy) variables before fitting, exactly as for a regression model. This requirement belongs to the software rather than to the method, as tree algorithms in other languages accept categorical variables directly. One caution applies when a categorical predictor has many categories. Converting it produces many indicator (dummy) variables, each of which is examined separately at every split, which both slows the fitting and dilutes the importance attributed to the original variable.

Missing values. Missing values are handled differently at each of the last three steps. A single tree and a random forest, as implemented in scikit-learn, require that missing values be filled in beforehand. XGBoost accepts them directly. A model comparison in which missing values are imputed for some methods and passed through for others is not a comparison of the methods alone, because the treatment of the missing data differs as well. Applying the same imputation throughout keeps the comparison clean, even though doing so gives up one of the conveniences XGBoost offers.

Whatever preparation is chosen, it must be determined from the training data alone and then applied unchanged to the validation and test data. A mean used for imputation, or the set of categories used to construct indicator variables, is a quantity estimated from data. Computing it from the full data set before splitting allows information from the test observations to influence the fitted model, the data leakage described earlier, and produces a test result that overstates how well the model will perform.

12.4.2.2 Choosing the Final Model

The amount of data available bears on which methods are worth considering at all. A flexible model requires enough observations to distinguish a genuine pattern from an accidental one, and the more flexible the model, the more observations it requires. With a few hundred observations, a regression model or a modest decision tree is usually the appropriate limit, and a boosting model with a dozen tuning parameters has more freedom than such a sample can support. A random forest is reasonable across a wide range of sample sizes and is a sensible choice when the data are limited, because it requires little tuning and its out-of-bag estimate makes efficient use of a small sample. XGBoost repays its added complexity most reliably when there are many thousands of observations, enough to support a separate validation sample for early stopping and a search across tuning parameters. These are rough guides rather than thresholds, and the number of predictors and the strength of the relationships matter as well, but the general principle is dependable: a more flexible model needs more data to be trusted.

In some applications, a simple regression model may be the best choice because it predicts well and is easy to explain. In others, a random forest may provide the best balance between predictive performance and ease of use. XGBoost may be worth the additional complexity when it produces a meaningful improvement in prediction.

And, as always, when comparing the predictor performance of different models, follow the procedure of a three-way data split of the full data table: training data, validation data, and testing data. The validation data can exist as a formal third data set created before the analysis begins. Or, particularly when there is not sufficient data or as a matter of preference, as a cross-validation procedure that creates random hold-out samples that perform the role of validation data in the analysis of the predictive performance of each model.

A practical workflow establishes the performance of the simplest reasonable model and moves to more flexible methods only when they provide a useful improvement. The final model should be selected because it best serves the purpose of the analysis, not because it is the most complex method available.

12.4.3 Summary of Ensemble Terms

Term	Meaning	Relationship
Ensemble method	Combines the predictions from multiple component models into one predictive system	Broad umbrella term
Bagging	Builds multiple models independently from different random samples of the training data and then combines their predictions	A major type of ensemble method
Boosting	Builds models sequentially, with each new model designed to improve the current ensemble	A major type of ensemble method
Random forest	Builds many decision trees using bagging and random subsets of predictors at each split	A specific bagging-based ensemble method
Gradient boosting	Adds models sequentially to reduce prediction error according to a loss function	A specific type of boosting
XGBoost	Provides an efficient and widely used implementation of gradient-boosted decision trees	A specific gradient-boosting method