Time series and classifier ensembles

Quick guide

The common idea behind both blocks is model combination. In time series, the final forecast is built from transformations, trend, seasonality and a seasonal model. In classification, the final decision is built from several learners that reduce variance, bias or both depending on the ensemble strategy.

1. Time series with AirPassengers

The notebook begins with the AirPassengers dataset, a classic monthly series of airline passengers. The first plots already show the three ingredients that drive the rest of the exercise: heteroscedasticity, trend and seasonality.

The analysis is developed in stages. First, the variance is stabilized to remove heteroscedasticity. Then the long-term trend is estimated and separated, and finally the seasonal behavior is studied in order to identify the period and fit a seasonal forecasting model.

1.1 Components of the series

• Heteroscedasticity: transform the series so that seasonal amplitude remains more stable over time.
• Trend: isolate the long-term linear growth component.
• Seasonality: detect the periodic behavior and model it explicitly.
• SARIMA: use a seasonal ARIMA model once the period has been identified.

The notebook therefore treats forecasting as a structured decomposition problem rather than a single black-box prediction step.

1.2 Forecasting the next two years

After fitting the model, the assignment predicts the next two years and compares the forecast with the real continuation of the series. The exercise then rebuilds the final prediction step by step: first the SARIMA output, then the trend component, and finally the inverse transformation needed to restore the original heteroscedastic behavior.

This makes the forecasting pipeline transparent: each transformation is justified and later reversed so that the prediction can be interpreted in the original scale of the problem.

2. Combination of classifiers

The second half of the assignment changes domain and works with a supervised dataset about whether a car is likely to skid in a curve. From there, the notebook compares several ensemble strategies instead of relying on a single model.

2.1 Parallel combinations

The notebook starts from a decision tree baseline and then studies parallel ensemble methods such as bagging and boosting. The point is to measure how performance changes when multiple weak or moderate learners are aggregated.

This section is useful as a controlled comparison: the same classification problem is kept fixed while the combination strategy becomes more sophisticated.

2.2 Sequential combinations

The final block studies sequential combinations of different base classifiers. The notebook introduces KNN and SVM as component models and then builds higher-level strategies such as stacking and cascading.

In other words, the assignment does not limit itself to ensemble voting. It also explores workflows where the output of one classifier becomes part of the input or decision process of another.

Overall objective

Together, both parts of the notebook show two complementary ideas: forecasting improves when the temporal structure of the data is modeled explicitly, and classification often improves when several learners are combined with the right architecture. The full notebook with code, plots and outputs remains below in Spanish.

How to evaluate these models

A good review should not stop at the final accuracy or a visually plausible forecast. For the time-series block, inspect residuals, compare the forecast against a simple baseline and verify that transformations are reversed correctly before interpreting the prediction in the original scale. For the classifier block, compare train and test results, use cross-validation and check whether the ensemble improves the baseline enough to justify the extra complexity.

This is especially important for educational notebooks because it is easy to present many algorithms without a clear decision criterion. The useful question is not “which model is more advanced?”, but “which model gives a better error profile for this dataset and remains understandable enough to maintain?”.

Temporal validation: never shuffle the future into the past

A random train/test split is inappropriate for forecasting because it lets observations from the future influence model selection. Keep the final months as an untouched test period and tune the model with expanding-window or rolling-origin validation. At every fold, the training dates must be earlier than the validation dates.

Compare SARIMA against simple baselines such as the last observed value and the value from the same month one year earlier. Report MAE or RMSE together with a scale-aware metric and prediction intervals. A sophisticated model is only useful when it improves the baseline consistently, not just in one favorable split.

from sklearn.metrics import mean_absolute_error
from statsmodels.tsa.statespace.sarimax import SARIMAX

train, test = series.iloc[:-24], series.iloc[-24:]
model = SARIMAX(
    train,
    order=(1, 1, 1),
    seasonal_order=(1, 1, 1, 12),
    enforce_stationarity=False,
    enforce_invertibility=False,
).fit(disp=False)

forecast = model.get_forecast(steps=len(test))
mae = mean_absolute_error(test, forecast.predicted_mean)
intervals = forecast.conf_int()

Residual diagnostics before trusting a forecast

Plot residuals over time and inspect their autocorrelation. Remaining seasonal structure means the model has not captured all predictable information. Also review variance changes, large outliers and the coverage of forecast intervals. Residuals do not need to look aesthetically perfect, but systematic patterns are evidence that the model specification or data preparation still needs work.

Keep every transformation inside the reproducible pipeline. If a logarithm, Box-Cox transform or differencing is applied during training, predictions and confidence intervals must be returned carefully to the original scale. Otherwise an apparently good error on the transformed series can produce a misleading business forecast.

Fair evaluation of classifier ensembles

Use one preprocessing pipeline per candidate so scaling, encoding and feature selection are learned only from each training fold. Compare every ensemble with its base learners under the same stratified folds and random seed. Accuracy alone can hide poor minority-class behavior, so include precision, recall, macro F1, confusion matrices and, when probabilities matter, calibration or ROC-AUC.

Stacking needs special care: the meta-model must train on out-of-fold predictions, not predictions generated by base models that already saw those rows. That separation is what prevents leakage. Complexity is justified only when the improvement survives cross-validation and remains stable on the untouched final test set.

from sklearn.model_selection import StratifiedKFold, cross_validate

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_validate(
    estimator,
    X,
    y,
    cv=cv,
    scoring=["accuracy", "precision_macro", "recall_macro", "f1_macro"],
    return_train_score=True,
)

Related learning path

If you are building a broader machine learning path, read this article after dataset selection and preparation and before deeper examples such as MuZero reinforcement learning. The sequence is useful: prepare the data, evaluate classical supervised models and then move toward more complex learning strategies.

The same evaluation discipline applies across the path. Keep baselines, write down assumptions and avoid changing several parts of the pipeline at the same time. If accuracy improves after transforming variables, changing the model and changing the split strategy, it becomes difficult to know which decision actually helped.

In a production project, this notebook would also need reproducible environments, fixed random seeds, saved metrics and clear separation between training, validation and final test data. Those details are less visible than the algorithms, but they determine whether a result can be trusted.