arXiv:2603.16985v2 Announce Type: replace 
Abstract: Transformer-based models have been widely adopted for time-series forecasting due to their high representational capacity and architectural flexibility. However, many Transformer variants implicitly assume stationarity and stable temporal dynamics -- assumptions routinely violated in financial markets characterized by regime shifts and non-stationarity. Empirically, state-of-the-art time-series Transformers often underperform even vanilla Transformers on financial tasks, while simpler architectures with distinct inductive biases, such as CNNs and RNNs, can achieve stronger performance with substantially lower complexity. At the same time, no single inductive bias dominates across markets or regimes, suggesting that robust financial forecasting requires integrating complementary temporal priors. We propose TIPS (Transformer with Inductive Prior Synthesis), a knowledge distillation framework that synthesizes diverse inductive biases -- causality, locality, and periodicity -- within a unified Transformer. TIPS trains bias-specialized Transformer teachers via attention masking, then distills their knowledge into a single student model with regime-dependent alignment across inductive biases. Across four major equity markets, TIPS achieves state-of-the-art performance, outperforming strong ensemble baselines by 55%, 9%, and 16% in annual return, Sharpe ratio, and Calmar ratio, while requiring only 38% of the inference-time computation. Further analyses show that TIPS generates statistically significant excess returns beyond both vanilla Transformers and its teacher ensembles, and exhibits regime-dependent behavioral alignment with classical architectures during their profitable periods. These results highlight the importance of regime-dependent inductive bias utilization for robust generalization in non-stationary financial time series.

تم اقتراح إطار جديد يسمى TIPS (Transformer with Inductive Prior Synthesis) لتحسين توقعات السلاسل الزمنية المالية من خلال دمج انحيازات استقرائية متنوعة عبر تقنيات تقطير المعرفة. تتناول هذه الطريقة القيود المفروضة على نماذج المحولات التقليدية، التي تفترض غالبًا الثبات في الأسواق المالية، مما يؤدي إلى أداء أقل في التطبيقات الواقعية.

Se ha propuesto un nuevo marco llamado TIPS (Transformer with Inductive Prior Synthesis) para mejorar la predicción de series temporales financieras mediante la integración de diversos sesgos inductivos a través de la destilación del conocimiento. Este enfoque aborda las limitaciones de los modelos Transformer tradicionales, que a menudo asumen la estacionariedad en los mercados financieros, lo que lleva a un rendimiento inferior en aplicaciones del mundo real.

Un nouveau cadre appelé TIPS (Transformer with Inductive Prior Synthesis) a été proposé pour améliorer les prévisions de séries temporelles financières en intégrant divers biais inductifs par le biais de la distillation des connaissances. Cette approche répond aux limites des modèles Transformer traditionnels, qui supposent souvent une stationnarité sur les marchés financiers, entraînant une sous-performance dans les applications réelles.

A new framework called TIPS (Transformer with Inductive Prior Synthesis) has been proposed to enhance financial time series forecasting by integrating diverse inductive biases through knowledge distillation. This approach addresses the limitations of traditional Transformer models, which often assume stationarity in financial markets, leading to underperformance in real-world applications.