← Back to list
AI & Data
#시계열#ARIMA#지수평활#딥러닝 예측#파운데이션 모델
Last updated · 2026-11-20

Time Series Analysis and Forecasting

1. Overview

A. Definition

Time series forecasting is an analytical technique that learns the trend, seasonality, and autocorrelation structure embedded in data observed at regular intervals in time order ($y_1, y_2, \dots, y_t$) with statistical or machine-learning models, and infers the values of as-yet-unobserved future points ($y_{t+1}, \dots, y_{t+h}$) in the form of a point estimate or a probability distribution (prediction interval).

The essence of time series forecasting is to capture the temporal dependency the data reveals on its own, on the assumption that "the rules of the past continue to some degree into the future." Unlike ordinary regression, which handles mutually independent observations, a time series is strongly correlated between neighboring points (today's sales resemble yesterday's), so the very order of the observations becomes information. Accordingly, traditional statistical techniques that assume "independent and identically distributed (i.i.d.)" data cannot be used as is, and autocorrelation, non-stationarity, and seasonal patterns must be modeled explicitly. This is what makes time series forecasting a distinct methodological domain.

B. Background and necessity

When you can know in advance indicators that change over time—demand, sales, electric load, stock prices, traffic, sensor readings—you can proactively allocate inventory, staff, equipment, and budget. A large share of corporate decisions reduces to the forecasting question "how much will be needed going forward," and in industries where a 1 percentage-point forecasting error leads to tens of billions of won in inventory cost or stockout loss (retail, semiconductors, energy), forecasting accuracy itself is competitiveness. In the public sector, too, forecasting traffic volume, electricity demand, tax revenue, and infectious-disease spread becomes the basis for infrastructure investment and policy-making, with large social repercussions. In the past, statistical models such as ARIMA and exponential smoothing were mainstream, but as large-scale, multivariate, nonlinear environments requiring the simultaneous handling of thousands to millions of series became common, machine-learning and deep-learning-based techniques spread rapidly. Recently, pretrained time series foundation models have appeared, reaching the stage of predicting even brand-new series with almost no training data in a zero-shot manner. In short, time series forecasting is a domain of very large practical demand, moving from traditional econometrics to the frontier of modern AI.

Another point that distinguishes time series forecasting from other prediction tasks is the feedback loop. The forecast result changes decisions (ordering, production, staffing), and those decisions in turn affect future observed values. For example, if demand is forecast high and inventory is increased, stockouts fall and observed sales rise, which again pushes up the next forecast. Because of this self-fulfilling cycle, time series forecasting must be designed not as pure "observation" but as part of an operational system, and unless the closed loop of forecast-decision-outcome is considered together, accuracy on metrics does not translate into real business value.

C. Characteristics

The nature of time series forecasting is summarized in four points. First, order dependency: shuffling the observations destroys information, so random-shuffle cross-validation cannot be used. Second, non-stationarity: the mean, variance, and seasonal structure change over time, so preprocessing (differencing, log, stabilization) is essential. Third, multiple time scales: overlapping cycles such as daily, weekly, and annual exist simultaneously. Fourth, exogenous-variable dependency: external factors such as weather, promotions, and holidays swing the values strongly. Because of these four traits, time series forecasting is, before it is "a problem of choosing a model," "a problem of dissecting the temporal structure of the data," and preprocessing and validation design matter as much as model selection.

2. Components of a Time Series and Stationarity

flowchart TB
  Y["Raw time series y(t)"] --> DEC["Time series decomposition(Decomposition)"]
  DEC --> T["Trend: long-term up/down direction"]
  DEC --> S["Seasonality: fixed-period repetition"]
  DEC --> C["Cycle: non-fixed long-term wave"]
  DEC --> R["Irregular/Residual: unexplained variation"]
  T --> MODEL["Model selection after grasping structure"]
  S --> MODEL
  C --> MODEL
  R --> MODEL

To forecast a time series, you first decompose the series into components to identify which pattern dominates. Trend is the directionality of values rising or falling over the long term, and seasonality is variation that repeats on a fixed period, such as "a weekly Monday traffic surge" or "an annual December sales peak." Cycle is a long-term wave whose period is not constant, like the business cycle, and the remaining unexplained variation is the irregular (residual) component. Decomposition splits into an additive model that sums components ($y_t = T_t + S_t + R_t$) and a multiplicative model that multiplies them ($y_t = T_t \times S_t \times R_t$); when the seasonal amplitude grows in proportion to the trend (the larger the sales, the wider the December peak), the multiplicative model fits. In practice the robust STL (Seasonal-Trend decomposition using Loess) decomposition is widely used.

The core premise required by forecasting models—especially the ARIMA family—is stationarity. A stationary time series is one whose mean, variance, and autocovariance do not change over time. If there is a trend the mean changes, and if there is seasonality the value differs systematically at certain points, so it is non-stationary. The representative technique for turning a non-stationary series into a stationary one is differencing: taking the difference between adjacent points, as in $y_t - y_{t-1}$, removes the trend. When the variance grows over time, it is stabilized with a log or Box-Cox transform. Stationarity is tested with the ADF (Augmented Dickey-Fuller) test; for example, if the p-value of the ADF statistic is 0.35, greater than 0.05, the null hypothesis that "a unit root exists (non-stationary)" cannot be rejected, a signal that differencing is needed.

After stationarizing by differencing or transformation, the appropriate model orders are read with the ACF (autocorrelation function) and PACF (partial autocorrelation function). If the ACF cuts off abruptly at a certain lag it suggests an MA (moving-average) component, and if the PACF cuts off it suggests an AR (autoregressive) component. For instance, in monthly sales data, if a large peak repeats in the ACF every 12 lags, it means there is annual seasonality, handled with seasonal differencing ($y_t - y_{t-12}$) or the seasonal term of SARIMA. In this way the work of Section 2 is the stage of diagnosing "which model the data wants," and it determines the quality of the subsequent model selection.

Real data often has not a single period but multiple seasonality. Hourly electric load has a daily period (24), a weekly period (168), and an annual period simultaneously, and such overlapping cycles are hard to contain in a single seasonal ARIMA. So one adds Fourier terms that approximate multiple periods with trigonometric bases as exogenous variables, or uses models that explicitly support multiple seasonality, such as TBATS or Prophet. In particular, irregular events whose date differs each year, such as holidays and long weekends, must be handled separately with calendar variables to prevent pattern distortion.

The preprocessing stage must also handle missing values and outliers. Because order is information in a time series, filling in missing values with a simple mean damages the trend and seasonal structure. Use linear/spline interpolation or seasonal imputation that reflects seasonality, and for outliers such as sensor errors or event-driven spikes, judge with domain knowledge whether it is noise or a true signal before removal. For example, a Black Friday sales surge is not an outlier but a seasonal signal that must be learned. In this way the diagnostic and preprocessing quality of Section 2 determines the ceiling of every model in later stages.

3. Major Forecasting Techniques

flowchart LR
  subgraph STAT["Statistical models"]
    ES["Exponential smoothing/Holt-Winters"]
    AR["ARIMA/SARIMA/SARIMAX"]
  end
  subgraph ML["Machine-learning models"]
    GBM["Gradient Boosting(feature engineering)"]
  end
  subgraph DL["Deep-learning models"]
    RNN["RNN/LSTM/GRU"]
    SEQ["DeepAR/N-BEATS/TFT/PatchTST"]
  end
  subgraph FM["Foundation models"]
    TFM["Chronos/TimesFM/Moirai(pretrained)"]
  end
  DATA["Preprocessed time series"] --> STAT
  DATA --> ML
  DATA --> DL
  DATA --> FM
  STAT --> OUT["Forecast value/prediction interval"]
  ML --> OUT
  DL --> OUT
  FM --> OUT

A. Exponential smoothing and Holt-Winters

The most classical yet still powerful approach is Exponential Smoothing. The core idea is to estimate the mean by "giving larger weight to recent observations and reducing the weight exponentially toward the past." Simple exponential smoothing handles only the level, but Holt captures the trend and Holt-Winters captures trend and seasonality simultaneously. The smoothing coefficients $\alpha$ (level), $\beta$ (trend), and $\gamma$ (seasonality) control the reaction speed of each component; a large $\alpha$ reacts sensitively to recent values, while a small one changes gently. The ETS (Error-Trend-Seasonal) framework, generalized into a statistical state-space form, automatically searches combinations of components to pick a suitable exponential-smoothing model. Light to compute and easy to interpret, it is widely used as a baseline in environments that must run thousands of series quickly, such as store-level retail demand forecasting.

The reason exponential smoothing is still important is that simplicity is robustness. In large-scale forecasting competitions such as M3 and M4, it has been repeatedly confirmed that complex techniques do not easily beat simple exponential-smoothing combinations, which refutes the belief that "a sophisticated model is always better." However, exponential smoothing cannot naturally accommodate exogenous variables and cannot capture nonlinear relationships, so its limits are clear in situations where explanatory variables are decisive, such as promotions and price elasticity. Therefore, in practice it is set as a baseline that must be surpassed, and the superiority of a more complex model is demonstrated relative to this baseline.

B. ARIMA, SARIMA, SARIMAX

ARIMA (AutoRegressive Integrated Moving Average) is a model combining autoregression (AR, a linear combination of past values), differencing (I, stationarization), and moving average (MA, a linear combination of past errors), specified by the three orders $(p,d,q)$. Adding a seasonal term $(P,D,Q)_m$ to this gives SARIMA, and adding exogenous variables (promotions, temperature, etc.) gives SARIMAX. ARIMA's strength lies in statistical interpretability and the theoretical basis of its prediction intervals. For example, fitting daily call-center arrivals with SARIMA$(1,1,1)(1,1,1)_7$ yields forecasts that reflect weekly seasonality while accompanying a 95% confidence interval. Its weaknesses are that it is single-series-centric, struggles to capture nonlinearity and long-term dependency, and incurs a large cost of choosing orders for each of many series. To mitigate this, auto_arima-type tools that automatically search orders are used, but limits remain when the number of series explodes.

ARIMA modeling traditionally follows the Box-Jenkins procedure. It is a repetition of ① securing stationarity (differencing/transformation), ② identifying orders with ACF/PACF, ③ estimating parameters by maximum likelihood, and ④ residual diagnostics. In particular, step ④ checks with the Ljung-Box test whether the residuals are white noise; if autocorrelation remains in the residuals, it means there is still unexplained structure, so the orders are adjusted. Selection among models uses the AIC/BIC information criteria, which look at goodness of fit and complexity together and curb overfitting. This systematic procedure is the key that makes ARIMA not a mere black box but a diagnosable statistical model, and the fact that one can trace the cause of a forecasting failure is why it is still preferred in public and financial domains where interpretability is required.

C. Machine learning (Gradient Boosting) and feature engineering

Applying gradient boosting such as LightGBM and XGBoost to time series is especially powerful in recent practice. The core is the feature engineering that reframes time series forecasting as a supervised regression problem. When you create features such as lag variables, moving averages/moving standard deviations, day-of-week/month/holiday flags, and promotion flags and feed them as input, the boosting model flexibly learns nonlinear interactions and exogenous effects. A representative basis is that LightGBM-based solutions swept the top ranks in the M5 Competition (2020, Walmart data), an international competition in demand forecasting. A single model can learn tens of thousands of series simultaneously (global model), giving higher data efficiency than per-series models (ARIMA), but it has the drawbacks of requiring expertise in feature design and being weak at extrapolation (trends beyond the observed range).

Another key design element when using boosting for time series is the multi-step forecasting strategy. The method of predicting $h$ steps ahead splits into the recursive approach, which predicts one step at a time and feeds that value back as input, and the direct approach, which trains a separate model for each horizon. Recursive is simple with a single model but propagates forecasting errors cumulatively, becoming less accurate the farther into the future; direct has no error accumulation but the number of models grows with the horizon, raising cost. In practice a compromise strategy or a multi-output model is used. The extrapolation weakness of boosting comes from the structural limit that tree-based models cannot predict values beyond the training range, so for series with a strong upward trend, a hybrid that first removes the trend (differencing/detrend) and then learns the residuals with boosting is effective.

D. Deep learning—from RNN/LSTM to the Transformer family

Deep learning shows strength in learning long-term dependency and multivariate, nonlinear patterns. Recurrent neural networks such as LSTM/GRU maintain long-term memory with a gate structure and process sequential data. Later, DeepAR is an autoregressive RNN that directly outputs a probability distribution (prediction interval) and learns many series together, and N-BEATS performs interpretable trend/seasonal decomposition through the basis expansion of pure fully-connected blocks. Temporal Fusion Transformer (TFT) integrates static and time-varying exogenous variables with attention and variable-selection gates, and reveals important time points interpretably. PatchTST divides the time series into patches to input to a Transformer and processes channels independently, raising long-horizon forecasting accuracy. Deep learning excels in large-scale, multivariate environments but demands large data, computation, and tuning, so when series are short or few it is often worse than simple statistical models.

What is interesting is that even though the Transformer conquered natural language and vision, it is not unconditionally superior in time series. As research emerged that simple linear models (DLinear) outperform complex Transformers on several benchmarks, a debate arose over "whether a Transformer is really needed for time series," and in response PatchTST, introducing channel-independence and patching techniques, raised performance again. The lesson of this debate is that the flashiness of the architecture is not proportional to forecasting accuracy, because time series have a weak signal relative to noise and relatively little data, carrying a high risk of overfitting. Therefore, when adopting deep learning, one must compare fairly against simple statistical/linear baselines and exercise the restraint of adopting it only when there is substantive improvement commensurate with the complexity.

4. Model Evaluation and Validation

Forecasting performance is measured with error metrics, but since each metric has a different nature, it must be chosen to fit the purpose. MAE (mean absolute error) and RMSE (root mean squared error) are intuitive because their units are the same as the raw data, but comparison is hard when scales differ across series. MAPE (mean absolute percentage error) is scale-independent as a percentage, but it explodes when the actual value is near zero and is asymmetric toward under-forecasting. The corrected sMAPE, and the MASE (Mean Absolute Scaled Error), which measures relative performance against a naive forecast, are used as standards in competitions and papers. A MASE below 1 means "better than a seasonal naive forecast," which is good for judging a model's real value. In probabilistic forecasting, instead of point error, the quality of the interval is also evaluated with quantile/pinball loss or CRPS.

Metric Nature Strength Caution
MAE Mean absolute error Intuitive, robust to outliers Scale-dependent
RMSE Squared-error based Sensitive to large errors Over-sensitive to outliers
MAPE Percentage Scale-independent Explodes/asymmetric near 0
sMAPE Symmetric percentage Corrects MAPE Still unstable near 0
MASE Ratio vs. naive Scale/series-independent Needs a baseline (naive) defined

Along with metric choice, how the forecasting horizon is set is also part of evaluation. Short-term forecasts such as next-day demand and long-term forecasts such as quarterly or annual require different accuracy and fit different models, and usually the farther the horizon the greater the uncertainty, widening the prediction interval. Therefore, rather than reporting performance as a single number, present it as an error curve by horizon to tell the decision-maker up to which point it can be trusted. Also, when there are many series, looking only at the average metric hides a few extreme-error series, so it is practically important to examine the distribution of the metric (median/quantiles) together to find weak segments.

Validation design is the point most commonly gotten wrong in time series. Because observations are in time order, one must prevent data leakage (look-ahead leakage) where future information leaks into past training, and therefore random K-fold cross-validation must not be used. Instead, use rolling/expanding-window cross-validation (time series split). Train on a past segment, validate on the immediately following segment, then push the window forward and repeat this to average performance over multiple points. For example, starting with 36 months of data as "train on 24 months → validate on the next 3 months" and advancing by 3 months yields 4–5 independent evaluations, filtering out overfitting. It is also a practical principle to always set the naive/seasonal naive that "copies the past value directly into the future" as a baseline, to verify whether a complex model really earns its complexity.

Finally, a point to note is the overfitting of validation itself. If you choose hyperparameters/models only by performance on a single validation segment, a model tuned to that segment may collapse in the actual future. To prevent this, average rolling evaluations over multiple points to pick a model that is consistently good, and report final performance only once on a holdout (test) segment never used for tuning. In particular, before real service, it is safe to simulate with a backtest that reproduces the past—"what decision would have been made and what would the outcome have been had this model been used then"—to confirm that metric improvement leads to real business benefit.

5. Comparison and Cases

Before comparing techniques, the first thing is to gauge "forecastability" itself. Not every series is equally forecastable. Electric load with a steady, clear trend and seasonality is easy to forecast, but series with almost no history, such as intermittent demand (mostly zero with occasional spiking parts demand) or new products, have clear limits for any sophisticated model. For intermittent demand, dedicated techniques such as Croston's method are used, and for series with no history, a cold-start strategy that borrows the pattern of similar products is used. In other words, diagnosing "how forecastable this series is in the first place, before choosing a model" is the starting point of resource allocation.

Technique selection is not "which is the best" but the problem of "which is optimal in this data situation." When series are few and long and interpretation and confidence intervals matter, ARIMA/ETS is advantageous. When there are tens of thousands of series and exogenous variables are abundant, global boosting captures both data efficiency and accuracy. When series are very many and long and complex nonlinear, multivariate interactions dominate, deep learning leads. The fundamental reason this difference arises lies in the balance of model complexity against the amount of data. The more complex the model, the more data it requires, so using deep learning when data is scarce increases variance and the bias-variance trade-off of losing to a simple model appears as is.

Technique Suitable situation Strength Limitation
Exponential smoothing/ETS Short, many series; fast baseline Light and stable Weak at nonlinearity/exogenous
ARIMA/SARIMAX Long, single series; interpretation-focused Statistical intervals/interpretability Weak at scalability/nonlinearity
Gradient Boosting Many series, rich features Data efficiency/exogenous effects Weak extrapolation; needs feature engineering
Deep learning/Transformer Large-scale, multivariate, long Complex patterns/probabilistic forecasts Data/computation/tuning cost

Three concrete cases can be cited. First, in electricity-demand forecasting, the Korea Power Exchange and others forecast short-term (next-day) demand within a few percent error with hybrid models that include temperature, day of week, and holidays as exogenous variables, used for generation planning and reserve-margin estimation. Because a 1°C temperature change strongly swings heating/cooling load, combining exogenous variables is decisive. Second, in retail demand forecasting, as in Walmart's M5 competition case, learning over 30,000 product-store series with a single LightGBM global model improved both accuracy and operational efficiency over per-series ARIMA. Third, in semiconductor/manufacturing demand forecasting, because the bullwhip effect amplifies demand variation the further upstream one goes, accurate demand forecasting governs the stability of inventory and production planning. These cases commonly show that "rather than the model itself, exogenous variables, validation design, and business integration decide success or failure."

Another strategy often adopted in practice is the hybrid/ensemble. A combination that captures linear trend/seasonality with ARIMA and learns its residuals with a neural network, or an ensemble that weights-averages the forecasts of several heterogeneous models, produces more stably lower error than a single model. The fact that the M4 competition's winning solution was a hybrid combining a statistical model and a neural network supports this. The reason an ensemble is effective is that each model has a different bias, so averaging them cancels individual errors and reduces variance. However, since an ensemble raises operational and interpretive complexity, a design that restrains the number of constituent models by weighing the accuracy gain against maintenance cost is needed.

6. Deep Dive: Time Series Foundation Models and Recent Trends

The biggest recent change is that, like the large language models of natural language, foundation models pretrained on vast time series have appeared and forecast new series zero-shot without training. Amazon's Chronos quantizes continuous values into tokens and trains in a language-model structure, and Google's TimesFM, Salesforce's Moirai, the lightweight model Lag-Llama, and Nixtla's commercial API TimeGPT are competing in open and commercial forms. Comparison studies in 2024–2025 report that "on noisy periodic data Chronos and TimesFM lead other models, but can be beaten by simple statistical models such as linear AR depending on the situation," suggesting they are data- and task-dependent, not a panacea. That is, foundation models are especially useful as "an immediate baseline for new series with almost no data," and when data is sufficient, an ensemble or fine-tuning with domain-specialized models is the realistic choice.

Along with the technical appeal of foundation models, practical constraints are also clear. These are the cost of running hundreds of millions of parameters at inference, a structure that struggles to reflect domain-specific exogenous variables, and the licensing and data-provenance transparency issues of open weights. The 2024–2025 "break-even analysis" studies point out that "once enough data has accumulated, a lightly specialized model surpasses a giant pretrained model in cost efficiency." Therefore, it is reasonable for organizations to take a staged strategy: use a foundation model as a temporary solution for the cold-start segment, then switch to a specialized model as data accumulates, or fine-tune with a small amount of data.

Another major trend is the shift from local models to global models. Instead of building a separate model for each series, learning all series together to share common patterns across series (cross-learning) improves the forecasts of series with little data. In addition, beyond point estimates, probabilistic forecasting is becoming the standard, because inventory, reserve-margin, and financial-risk decisions care more about "how far it could go in the worst case" than "what the mean is." At this point, techniques like the conformal prediction discussed earlier, which guarantee coverage with a prediction interval regardless of the model, are combining with time series and raising reliability in safety-critical demand, energy, and finance domains. Likely exam directions include "comparison of statistical models with deep-learning/foundation models," "demand-forecasting strategy to mitigate the bullwhip effect," and "time series cross-validation and prevention of data leakage" as strong essay topics. When composing an answer, a high-scoring strategy is to develop the flow of preprocessing (stationarity/seasonality diagnosis) → model-group comparison (statistical/ML/DL/foundation) → evaluation and validation design → operations and decision integration, describing at each stage, with evidence, "why that choice fits the data characteristics." Only by infusing the bias-variance trade-off and cost-benefit perspective, rather than merely listing techniques, can one reveal professional-engineer-level depth.

7. Considerations and Implications

From a professional engineer's perspective, adopting time series forecasting must synthesize the following. First, the consistency of data volume with model complexity. A trendy deep-learning or foundation model is not always the answer; when series are short or few, an ETS/ARIMA baseline is more accurate and cheaper to operate. The organization must weigh by cost-benefit "whether a 1 percentage-point accuracy improvement justifies the increase in operational/maintenance cost." Second, validation design and data governance. Without time-ordered cross-validation, leakage prevention, and a backtest system, flashy performance at the development stage collapses in operation. Data quality management such as missing-value/outlier handling, scale normalization, and exogenous-variable consistency is a task that precedes model selection.

Third, operational response to distribution shift. Because model performance drops sharply when past patterns break from a structural shock like COVID-19 or a new-product launch, drift detection and a periodic retraining pipeline must be automated with MLOps. Fourth, quantification of uncertainty and decision integration. Providing only point forecasts makes risk management impossible, so value is realized only by producing prediction intervals/scenarios together and linking them to decision rules such as inventory policy and reserve-margin estimation. Fifth, organizational capability and operational maturity. However good a model is, without the organization and pipeline to continuously retrain, monitor, and improve it, it ends as a one-off project. Only by institutionally linking forecasts to business processes such as demand planning (S&OP), inventory management, and generation planning, and by having a feedback system between forecasters and the field, does a model evolve. Also, trust accumulates only when there is exception-handling/human-in-the-loop design for "who responds, and how, when the model is wrong." Technology adoption and organizational change are one inseparable task.

Sixth, interpretability and accountability. In domains with strict regulation, such as finance, energy, and the public sector, one must be able to explain "why it was predicted that way," so for black-box models a strategy of combining attention interpretation from TFT or explanation techniques such as SHAP, or interpretable ETS/N-BEATS, is needed. In short, the success of time series forecasting depends not on algorithm selection but on end-to-end design encompassing data, validation, operations, and decision-making, and the professional engineer must have the eye to design a balance point suited to the problem's characteristics rather than being swept along by technology trends.

References


In one line: Time series forecasting is an end-to-end decision problem that, after dissecting temporal dependency, non-stationarity, and seasonality through preprocessing, chooses among exponential smoothing, ARIMA, boosting, deep learning, and foundation models the technique matching the data-complexity balance, and designs all the way through time-ordered cross-validation, probabilistic forecasting, and distribution-shift response.