RANKING TIME SERIES ANOMALY DETECTION ALGORITHMS USING META-LEARNING

ABSTRACT

Time series anomaly detection (TSAD) is crucial for identifying unusual patterns in sequential data across domains such as finance, healthcare, cybersecurity, and predictive maintenance. A key challenge is selecting the most suitable anomaly detection algorithm for a given time series, due to the wide variability in instance characteristics. This paper tackles the algorithm selection problem for univariate TSAD by applying meta-learning to predict the performance ranking of candidate methods. We propose a meta-learning framework that extracts statistical, structural, and temporal meta-features to model algorithm behavior. The study evaluates four TSAD algorithms-two model-based (DeepAnt and Temporal Convolutional Autoencoder) and two discord detection methods (Matrix Profile and Merlin). Experimental results show that meta-learning is effective for unsupervised TSAD algorithm selection. A Random Forest meta-model achieved strong results, with a Spearman correlation of 0.61, Kendall tau of 0.52, top-1 accuracy of 64%, and top-2 accuracy of 89%, demonstrating its reliability even when performance differences between algorithms are small.

Keywords:
time series anomaly detection; meta-learning; algorithm selection

1 INTRODUCTION

Time series data is prevalent in applications where measurements vary over time and where the order and interrelation of observations are crucial in defining data patterns. This data paradigm is present in numerous domains, spanning economic and financial realms, environmental studies, demographics, health management, and sensor monitoring. Within this broad spectrum, several time series analysis tasks have emerged. Among them, time series anomaly detection (TSAD) stands out as particularly popular. It focuses on identifying unusual or abnormal patterns or events across time, leveraging the particularities of the time series structure.

Time series data is very particular in the sense that it may exhibit unique behaviour according to the application domain, such as seasonality, trends, and cycles. Due to this diversity of behaviours, various anomaly detection techniques and algorithms exist for TSAD. In a recent review, Blázquez-García et al. Blázquez-García et al. (2021) proposed a taxonomy of TSAD techniques, classifying the algorithms based on the input data (i.e., univariate or multivariate time series), the type of outlier they were designed to detect, and the nature of the method. From all the classes, model-based and discord detection are the two most common methods found in the literature. Model-based approaches work by fitting a model to predict or estimate the expected value of a time series, thus detecting the anomalies based on the difference between the expected value and the predicted one. Discord detection techniques focus on identifying subsequences or segments within the time series that exhibit unusual or dissimilar behaviour compared to the rest of the data. Thus, discord detection aims to find rare and anomalous patterns rather than detecting anomalies regarding the expected values.

Given the existing number of time series anomaly detection algorithms and models, selecting the most suitable method for a particular application becomes a non-trivial task. In addition, due to the heterogeneity of time series data, no algorithm consistently outperforms others across all contexts. Hence, it is necessary to find which features better describe the data concerning the algorithms’ performance and tools that estimate the algorithm performance prediction based on the data characteristics, which is indeed challenging.

Meta-learning, also referred to as “learning to learn”, constitutes a subfield within machine learning dedicated to developing algorithms and methodologies for enhancing the efficiency of learning systems in acquiring new knowledge or skills (Vanschoren, 2019). The tasks performed by meta-learning techniques include configuration recommendation, transfer learning, few-shot learning, and algorithm performance prediction. Meta-learning has also emerged as an effective approach for algorithm selection across several domains, including combinatorial optimization (Kanda et al., 2016; Smith-Miles & Tan, 2012), data anomaly detection (Kandanaarachchi et al., 2020), time series forecasting (Prudêncio & Ludermir, 2004; Talagala et al., 2018), and data clustering (Fernandes et al., 2021). In these domains, meta-learning acts as an algorithm selector using a learning model to map meta-features extracted from data instances to recommend the most appropriate algorithm for the target task.

Despite its extensive application across various domains like classification and regression tasks, to our knowledge, its exploration in the context of time series anomaly detection still needs to be explored. In this work, we devise a meta-learning model to select the most appropriate algorithm for univariate time series anomaly detection in offline configuration. To train our meta-learning model, we gathered data from the literature, corresponding to benchmark datasets containing time series instances with annotated anomalies. We extracted a variety of meta-features to comprehensively characterize these instances, including basic statistics, information theory measures, time series-specific attributes, and additional relevant features. The algorithm selection is performed concerning a pool of anomaly detection algorithms for time series, formed by two model-based approaches and two discord detection algorithms from the literature, namely Deep-Ant (Munir et al., 2018), Temporal Convolutional Autoencoder (TCN-AE) (Thill et al., 2021), Matrix Profile (STUMPy) (Law, 2019), and Merlin (Nakamura et al., 2020).

From an operations research perspective, the algorithm selection problem (Rice, 1976) can be naturally interpreted as a decision-making problem under uncertainty, where each time series instance defines a decision context, the candidate algorithms represent competing alternatives, and the objective is to select the alternative that maximizes a performance measure. Since the true performance of each algorithm is unknown a priori, the decision-maker must rely on predictive models to estimate outcomes. In this sense, meta-learning can be viewed as a data-driven decision support mechanism, which leverages historical performance information to guide the selection process. Our proposed framework does not only predict algorithm performance, but also prescribes decisions by ranking alternatives according to their expected effectiveness, aligning the algorithm selection task with decision-making paradigms in operations research.

The main contributions of our paper are:

• We engineered a set of meta-features encompassing statistical, structural, spectral analysis, and model-related features for time series anomaly detection.

• By evaluating both model-based and discord detection techniques, our study aims to provide a detailed analysis of their effectiveness and limitations in detecting anomalies for heterogeneous time series data.

• We developed a meta-learning approach for time series anomaly detection with an average accuracy of 80% in classifying good/bad performance. In the label ranking task, our method achieves an average Spearman correlation of 0.60 and a Kendall rank correlation of 0.52. We also achieved a top-1 accuracy of 64%, and 89% on the top-2 accuracy in the algorithm selection task.

The rest of this paper is organized as follows. Section 2 describes the algorithm selection problem, focusing on existing meta-learning frameworks that perform such a task. Section 3 introduces key concepts of anomaly detection in time series, presenting different types of algorithms. Section 4 introduces the methodology used to develop the framework proposed in this paper. Section 5 presents the introduced framework at length. In Section 6, we present our computational experiments and examine both their outcomes and their significance. Section 7 concludes the paper by presenting the main contributions and future directions.

2 META-LEARNING AND THE ALGORITHM SELECTION PROBLEM

Meta-learning, often described as the ability to learn how to learn, draws inspiration from the remarkable human capacity to swiftly adapt to new tasks by leveraging previously acquired knowledge. According to Brazdil et al. Brazdil et al. (2008), meta-learning is the organized exploration of techniques that use prior knowledge to create effective models and solutions, adjusting how machine learning and data mining work. The overarching objective of meta-learning is to capitalize on prior knowledge, enhancing training efficiency and algorithmic performance, as highlighted by Vanschoren (2019).

Within the diverse landscape of meta-learning frameworks and their applications, a foundational contribution to this field was made by Rice (1976). The author presented the Algorithm Selection Problem (ASP), which involves determining the most appropriate algorithm from a collection of algorithms to address a particular problem instance.

Figure 1 illustrates the framework proposed in (Rice, 1976) to address the ASP. In this framework, we are presented with a problem instance i ∈ 𝒫, for which a feature vector f (i) ∈ ℱ is computed. The goal is to determine the function S(f(i)) that associates an algorithm a ∈ 𝒜 with an instance i to optimize a performance metric y(a, i) ∈ 𝒴 when solving instance i.

Figure 1
Framework of Rice (1976) for the ASP, source: Smith-Miles & Lopes (2012).

Before presenting the basic components of our meta-learning model for TSAD, we present in the next section key concepts and algorithmic approaches associated with this task.

3 ANOMALY DETECTION IN TIME SERIES

Researchers have categorized anomalies into three types based on their characteristics (Blázquez-García et al., 2021; Chandola et al., 2009; Audibert et al., 2022; Correia et al., 2024).

Point anomalies indicate individual data points in time that deviate significantly from the expected behaviour of the time series.

Subsequence anomalies are segments of the time series that exhibit unusual behaviour over a period of time.

Anomalous time series can only be detected in multivariate time series, where an entire time series exhibits unusual behaviour compared to the rest of the data.

Our study focuses on univariate time series anomaly detection, addressing both point and subsequence anomalies. The literature presents a variety of algorithms based on different principles to handle the detection of such anomalies in univariate time series data. Among the numerous strategies, this paper investigates two primary approaches: model-based and discord-based. Model-based methods are prevalent in TSAD, employing predictive and reconstructive models to understand data patterns. Discord detection represents a more straightforward yet effective approach for identifying anomalies, where the algorithm compares subsequences within the time series to identify regions that significantly deviate from the rest of the data.

Before discussing these two algorithmic approaches, it is important to provide some definitions:

Definition 1.A time series X is a collection of observations x _t , indexed by a set of time points T ⊂ ℤ⁺ . Each observation x _t represents the value of the time series at time t.

Definition 2.Let X = {x ₁ , x ₂ , . . . , x _L } be a time series of length L. A$\mathrm{subsequence}{S}_p^{\alpha }$of X is a contiguous segment of X, with length α < L, starting at index p. Mathematically, it is defined as:

$S_p^{\alpha }=\{x_p,x_{p+1},\dots ,x_{p+\alpha -1}\}$

where 1 ≤ p ≤ L − α + 1.

Definition 3. The sliding window technique involves moving a fixed-size window iteratively over the time series. This window captures a specific portion of the data at each position, allowing for the sequential analysis of different time series subsequences.

3.1 Model-based algorithms

Model-based algorithms construct models to represent the expected behaviour of the time series, thereby facilitating the identification of behaviour deviations. These algorithms are composed of two main modules: (i) the model and (ii) the discriminator. The model is responsible for learning and capturing the characteristics of the time series data. It aims to understand and represent the data’s usual patterns, trends, and dependencies. The discriminator module then detects anomalies by comparing the predicted values generated by the model to the actual observed values.

Model-based algorithms can be further categorized into prediction and reconstruction models. As the name suggests, prediction-based models apply prediction models to forecast future values based on historical data. In other words, the model takes as input a subsequence $S_p^{\alpha }$ and is configured to predict the following subsequence $S_{p+\alpha }^{\beta }$. Note that the lengths α and β might differ. Time series prediction models can involve statistical analysis such as AutoRegressive Integrated Moving Average (ARIMA) models (Zhou et al., 2018), or deep learning techniques such as Convolutional Neural Networks (CNNs) (Munir et al., 2018), Long Short-Term Memory (LSTM) (Hundman et al., 2018), and Transformers (Kim et al., 2023).

In contrast, reconstruction models aim to learn compact representations or encodings of input sequences to reconstruct them later (Thill et al., 2021). Autoencoders are the most popular of these models. In the time series context, they strive to capture the intrinsic patterns and dependencies in the data, facilitating a comprehensive understanding of their temporal dynamics. Anomaly detection is performed by an autoencoder applied over subsequences of the entire time series. This process compares each reconstructed subsequence ${\hat{S}}_t^{\alpha }$ with its original counterpart $\left(S_t^{\alpha }\right)$.

3.2 Discord-based algorithms

The key idea of discord detection is to identify subsequences within a time series that stand out as significantly different from the rest of the data, denoted discords. Typically, discord detection techniques require the user to specify the subsequence size (often referred to as the “window”) that indicates the granularity that will be examined for discords within the time series.

The most intuitive discord-based algorithm is the Matrix Profile algorithm (Law, 2019; Zimmerman et al., 2019), where each subsequence $S_t^{\alpha }$ of the time series X , for t = 1, . . . , L − α + 1, is compared to all other subsequences regarding a dissimilarity measure. The result is a matrix where each entry stores the dissimilarity between two subsequences of X . A time series discord is identified as the subsequence having the greatest dissimilarity to its nearest neighbouring subsequence.

A naive implementation of the Matrix Profile algorithm computes the dissimilarity matrix in time O(L ²). Other methods have been proposed in the literature to reduce this time complexity. The Heuristically Ordered Time series using Symbolic Aggregate ApproXimation (HOT-SAX) (Keogh et al., 2005) introduces a heuristic that orders the examination of specially-purposed discretized time series subsequences to prioritize those most likely to be anomalous. Another approach is the algorithm proposed by Yankov et al. (2008), called Discord Range Aware Gathering (DRAG) which optimizes the search for discords using search and refinement strategies. More recently, Nakamura et al. (2020) proposed the Merlin algorithm, which relies on a heuristic approach to find a suitable value for the DRAG hyperparameter r that balances runtime efficiency and accurate detection. The selection of r is crucial in the DRAG method. If it is too small, the performance of DRAG deteriorates to O(L ²) complexity. If this parameter is too large, the algorithm may fail to detect anomalies properly. Merlin’s strategy of fine-tuning r for each window length achieves a balance between computational efficiency and improving the likelihood of detecting anomalies within the data.

There is no consensus in the literature regarding whether model-based or discord-based methods are superior, as each method offers distinct advantages and drawbacks. For instance, the performance of model-based algorithms depends heavily on how accurately the model can represent the underlying patterns in the data. This process involves training the model and fine-tuning a range of hyper-parameters to optimize its predictive accuracy. The effectiveness of model-based methods is thus contingent on the availability of high-quality training data and the selection of appropriate model parameters. Conversely, discord-based algorithms identify anomalies by analysing dissimilarities observed between subsequences of the entire time series. Instead of relying on a pre-trained model, these algorithms focus on the data, looking for subsequences that significantly differ from their nearest neighbours. Nonetheless, discord-based methods are not well suited for time series containing similar anomalies, such as the “twin freak problem” (He et al., 2020).

The following sections introduce the concept of meta-learning for algorithm selection (Section 4), and present the meta-learning framework proposed in this paper for the anomaly detection task in time series data (Section 5).

4 META-LEARNING FOR ANOMALY DETECTION IN TIME SERIES DATA

In alignment with the ASP framework proposed by Rice (1976), and guided by the meta-learning taxonomy delineated by Vanschoren (2019), our objective is to develop a meta-learning model for algorithm selection in the context of univariate time series anomaly detection. With that purpose, we follow the methodological steps presented by Karimi-Mamaghan et al. (2022) for the ASP, namely: (i) meta-data extraction, which determines the feature and performance spaces called meta-data, and (ii) meta-learning and meta-model creation, which uses the meta-data to develop a meta-model capable of predicting the performance of each algorithm for specific problem instances, thereby determining the optimal mapping between problem instances and algorithms.

4.1 Meta-features extraction

Meta-features are quantifiable attributes of datasets or tasks that are crucial for meta-learning (Jomaa et al., 2021). The main objective of the meta-features is to characterize the problem instances to enable the meta-model to discriminate about the best algorithm for a given instance. These features are domain-specific, meaning that they depend on the particular characteristics of the dataset.

Meta-features have already been explored in the literature in the context of time series analysis. These features serve as core descriptors that capture the inherent characteristics of the time series data, facilitating meaningful comparisons. In a recent work, Lubba et al. (2019) introduced a feature-based summary of an interdisciplinary time series analysis literature for use in a time series classification task, the CAnonical Time series CHaracteristics (catch22). This set of features was extracted from over 4791 time series features through a systematic, data-driven process, exhibiting a similar classification accuracy across the 93 classification tasks hosted in the UEA/UCR repository. This approach ensures that the selected features are not only powerful in terms of classification performance but also exhibit minimal redundancy.

Particularly in time series forecasting, Prudêncio & Ludermir (2004) utilized 10 diverse metrics encompassing statistics and information theory meta-features. Building on this seminal work, Wang et al. (2009) introduced a novel set of meta-features, which included statistical, information-theoretic, model-based, and landmarking meta-features developed, which were later used in other studies by Widodo & Budi (2013) and Talagala et al. (2018).

4.2 Performance evaluation

To evaluate the detection performance of each anomaly detection algorithm regarding the time series meta-features space, we utilize an anomaly score. This score measures how probable it is for a data point in the time series to be identified as an anomaly. Therefore, data points with higher anomaly scores are more likely to be considered anomalies.

Typically, anomaly detection algorithms for time series differ in how anomaly scores are computed. Hereafter, we use two different approaches for computing the anomaly score of an observation x _t within a time series X depending on the model forecast, estimate or compared subsequence length. The first focuses on single points (i.e., unitary subsequences). In this case, the anomaly score s _t of x _t is given by a dissimilarity measure:

$s_t=diff\left(x_t,{\hat{x}}_t\right),$ (1)

where ${\hat{x}}_t$ represents the estimated or predicted value of ${\hat{x}}_t$ as computed by the algorithm, and diff is dissimilarity measure between the two values (e.g. absolute difference). The second used approach consists of comparing subsequences. We apply the moving average function of the dissimilarities between S and $\hat{S}$ over α time periods. Thus,

$s_t=\frac{\sum _{i=0}^{\alpha }d\left(S_{t-i}^{\alpha },{\hat{S}}_{t-i}^{\alpha }\right)}{\alpha },$ (2)

where d is a dissimilarity (or distance) measure (e.g. Euclidean, Mahalanobis). Therefore, the anomaly score is given by the average of the computed dissimilarity between each pair of subsequences that contain x _t for a given length α. This approach mitigates the impact of transient or noise spikes in the time series.

In the sequel, we assess the performance of the algorithm to detect anomalies by calculating the Area Under the Receiver Operating Characteristic Curve (AUC) for the different anomaly scores computed across the time series. For instance, for X = {x ₁ , x ₂ , x ₃ , x ₄ , x ₅}, we compute the anomaly scores s ₁ , s ₂ , s ₃ , s ₄ and s ₅. Each value serves then as a threshold to compute true positive (TP) and false positive (FP) rates along with the Receiver Operating Characteristic (ROC) curve. The AUC metric provides a comprehensive measure of the algorithm’s performance in distinguishing between normal and anomalous observations, being widely used in the anomaly detection literature (Munir et al., 2018; Francisquini et al., 2022; Kandanaarachchi et al., 2020).

4.3 Meta-model creation

In this stage, learning algorithms are applied to map the relationship between the meta-features and the meta-target performance space. The meta-model is intended to inform a decision about which algorithm we expect to perform better in a given instance.

Our meta-learning model for ASP of TSAD algorithms considers the time series meta-data presented in Section 4.1 as the feature space ℱ, and aims to determine an anomaly detection method from a set 𝒜 of available algorithms. In this work, the set 𝒜 is composed of four TSAD algorithms, namely:

• DeepAnt (Munir et al., 2018), a model-based prediction algorithm;

• TCN-AE (Thill et al., 2021), a model-based reconstruction algorithm;

• Matrix profile (Law, 2019), a discord detection algorithm with a fixed window length;

• Merlin (Nakamura et al., 2020), a discord detection algorithm with variable window lengths.

To create our meta-learning model, we needed to annotate multiple time series for training. Let M: 𝒜 × ℱ → {0, 1} be a function that maps from the Cartesian space ?? × ℱ to a binary label such that:

$M\left(a,f\left(i\right)\right)=\left\{\begin{array}{l}1\text{if}y\left(a,i\right)\ge \tau ,\\ 0\text{otherwise}.\end{array}\right.$ (3)

Here, y(a, i) corresponds to the AUC obtained by the TSAD algorithm a ∈ 𝒜 when detecting anomalies in a time series instance i ∈ 𝒫. The threshold parameter τ in (3) plays a central role in converting the continuous AUC performance measure into a binary label, thereby defining what constitutes a “good” algorithm for a given instance. In this sense, τ determines the decision boundary used by the meta-model and directly affects the class distribution in the meta-dataset. Its choice introduces a trade-off: lower values of τ lead to a more permissive definition of good performance and typically result in more balanced classes, whereas higher values impose stricter performance requirements but may increase class imbalance.

Our meta-learning framework trains one binary classifier for each algorithm in 𝒜, aiming to predict the value of M for unseen time series. Thus, one classifier c _a: ℱ → [0, 1] is learned for each TSAD algorithm a ∈ 𝒜. The ranking of the algorithms in 𝒜 for a particular time series instance i ∈ 𝒫 is finally determined according to c _a in non-increasing order.

5 PROPOSED FRAMEWORK

This section provides an overview of the proposed meta-learning model, summarised graphically in Figure 2. We can split the framework into the training step and the ranking step, described in the next subsections.

Figure 2
Training step of our meta-learning model.

5.1 Training

The training step of our framework is responsible for constructing and optimizing the binary classifiers that will later be used to predict the performance of TSAD algorithms on new instances.

The process of training can be explained through the following steps:

• Benchmark Dataset: To guarantee the diversity of the meta-learning framework, we selected a collection of three benchmark datasets containing annotated instances (𝒫) that encompass a wide variety of time series and anomalies. They cover an extensive range of anomaly detection scenarios in the univariate setting. The database is described in Section 6.1.

• Meta-features: To characterize the time series instances, we extracted a diverse set of meta-features (ℱ) from the data. These include statistical measurements, spectral analysis metrics, time series-specific characteristics, error-based evaluations, fractal analysis, entropy measurements, and other relevant features.

• Algorithm Performance: The proposed framework considers four distinct TSAD algorithms (𝒜), comprising two model-based approaches (DeepAnt and TCN-AE) and two discord detection methods (Matrix Profile and Merlin). This selection ensures a diverse representation of techniques, allowing us to thoroughly assess the effectiveness of model-driven and pattern-based anomaly detection strategies within the proposed system. To evaluate the performance of the algorithms, we selected the AUC as the primary metric. Although AUC has certain limitations, mainly when applied to imbalanced datasets (Kandanaarachchi et al., 2020), it remains a valuable measure to assess the ability of an algorithm to identify anomalies correctly. The AUC effectively captures the balance between true and false positive rates, providing a strong indicator of the algorithm’s anomaly detection capability.

• Data cleaning and transformation: Using the meta-features (ℱ) and the performance of the TSAD algorithms (𝒴), we constructed the meta-dataset. To enhance the performance of the meta-learner, we cleaned and transformed the data. This involved applying Z-score normalization to standardize the meta-features, binarising the AUC performance of the TSAD algorithms into categories of “good” and “bad”, and employing a graph correlation method for feature selection.

• Meta-learner: The meta-learner is responsible for mapping the meta-features to the binary performance of the TSAD algorithms. To achieve this, we evaluated three classification algorithms: Linear Regression, Random Forest, and XGBoost.

5.2 Ranking

In the ranking phase, the trained meta-learner is applied to rank the algorithms in new time series instances. The previously selected meta-features are extracted and normalized for each new instance using the same Meta-Features Extractor and normalizer as in the training phase. The normalized meta-features are then fed into the trained meta-learner, which computes the predicted AUC performance for each TSAD algorithm. Finally, the TSAD algorithms are ranked based on the computed probabilities, allowing for the selection of the most appropriate algorithm for the new dataset.

In the following section, we present a series of computational experiments that demonstrate the effectiveness of this framework.

6 COMPUTATIONAL EXPERIMENTS

This section presents the computational experiments designed to evaluate the performance of our meta-learning model for anomaly detection in time series. The experiments are designed to provide a comprehensive understanding of the model’s efficacy and robustness across different datasets.

In Section 6.1, we describe the benchmark datasets used in our study, detailing their characteristics and relevance. Next, in Section 6.2, we outline the configuration settings for the time series anomaly detection algorithms used in the algorithm space 𝒜, ensuring a standardized and fair comparison. Then, in Section 6.3, we detail the meta-feature extraction and selection processes, along with the configuration of the meta-models used in the meta-learner. Finally, we discuss the experimental results, highlighting key findings, performance metrics, and insights into the effectiveness of the proposed approach.

6.1 Description of benchmark datasets

Our study used three well-known benchmark datasets in the time series anomaly detection literature.

• Numenta Anomaly Benchmark (NAB) (Ahmad et al., 2017): Benchmark composed of 58-time series containing from 1127 to 22695 manually annotated points. The NAB benchmark encompasses two main types of time series: datasets that contain artificially created data and real datasets that contain data extracted from various applications such as taxi demand, Twitter, machine temperature reading, and server data.

• UCR time series anomaly (Wu & Keogh, 2021): Benchmark containing 250 time series in medicine, biology, meteorology, and industry applications. This benchmark has time series instances with 6684 to 900000 annotated points. Each instance is predivided into training and validation splits to facilitate robust model evaluation and testing.

• Yahoo Webscope¹: Benchmark containing 367 time series data, mostly based on computational services at Yahoo!. Each time series contains 741 to 1,680 annotated points.

These benchmark datasets collectively offer a wide range of scenarios and challenges for anomaly detection, ensuring a comprehensive evaluation of the time series anomaly detection algorithms and the meta-learning model proposed here.

6.2 Performance of anomaly detection algorithms

As discussed in Section 5, the training process for our meta-learning model depends on data regarding the AUC metric of four TSAD algorithms. To accomplish this, data collection regarding their performance is essential, with a focus on correctly tuning their parameters and hyper-parameters. In particular, the model-based anomaly detection algorithms in 𝒜 must be trained to learn the patterns of the time series, enabling them to detect anomalies effectively. We recall that discord-based algorithms do not need to be trained.

To prepare the data for training the model-based TSAD algorithms, we split the data into training and test sets using a hold-out split. For the UCR dataset, we used the provided train/evaluation split for each instance. For the Numenta and Yahoo! datasets, which do not have a predefined train/test split, we allocated 40% of the time series data for training and the remaining 60% for testing. Notably, instances lacking anomalies in the test sets were removed to ensure meaningful performance assessment. Finally, data is normalized by minmax scaling.

All the TSAD algorithms in 𝒜 were trained using the hyperparameter values in the referenced works. The only hyperparameter we decided to optimize in this investigation was the window length α, due to the diversity of the time series found in our benchmark datasets, which varied in length and frequency, ranging from 2 to 2269, after employing the Discrete Fourier Transform (DFT) (Ermshaus et al., 2023).

To optimize α, we first extracted the base window lengths for each benchmark time series, thus identifying periods with the most significant amplitudes in each series. In our study, we focused on the top-3 window lengths, ensuring that these lengths are neither multiples nor divisors of each other, thereby avoiding harmonic periods. Given the unique characteristics of the model-based and discord-based algorithms, we employed different approaches for optimizing α according to the type of the TSAD algorithm, as discussed in the next sections.

6.2.1 Window length of model-based algorithms

For each deep learning algorithm used in our study, namely DeepAnt and TCN-AE, we trained nine different models by varying the window length α of the input subsequence according to the top-3 base windows identified for each time series. The nine models were obtained by multiplying the three base windows by 2, 3, and 5 to help the learning models capture seasonal and trend components. They were trained on 85% of the training set and evaluated on the remaining 15% for each time series instance regarding their average AUC performance obtained across the entire set of time series contained in the benchmark datasets.

The impact of the window length on the model-based algorithms can be assessed by Wilcoxon signed-rank tests (Benavoli et al., 2016). Here, the null hypothesis (H ₀) assumes no significant difference in the AUC performance distribution of a model when using two different values for the hyperparameter α. Conversely, the alternative hypothesis (H ₁) proposes that the performance distribution of the first sample (R(α ₁)) is greater than the second (R(α ₂)) over the entire collection of benchmark time series. For each instance, we sort the window length in increasing order of α value. The relation (i, j) corresponds to the p-value of the Wilcoxon test comparing the use of i-th window length of each instance with the use of the j-th window length, where the alternative hypothesis H ₁ is that the results with the i-th window length are better than those with the j-th window length.

Figure 3 presents two heatmaps for the model-based algorithms TCN-AE and DeepAnt, displaying the p-values obtained from the Wilcoxon signed-rank tests. Here, each heatmap cell reports the p-value obtained for one statistical test, where the AUC performance of an algorithm (i.e., DeepAnt or TCN-AE) is compared regarding a pair (α ₁ , α ₂) of distinct α values. The compared α values are sorted in ascending order of window lengths to facilitate interpretation.

Figure 3
Heatmap of p-values from the Wilcoxon signed rank test comparing the distribution of AUC performance,obtained by the TCN-AE and DeepAnt, across various ordered pairs of window sizes.

Window sizes are sorted from 1 (smallest) to 9 (largest) along both axes.

The heatmaps presented for both model-based algorithms clearly indicate that larger window lengths lead to better performance for both algorithms (p-value < 0.05). However, since the computational complexity of deep learning models depends on α, its augmentation leads to significantly longer training times and higher resource demands as discussed by Kahraman et al. (2019).

6.2.2 Window length of discord detection algorithms

The discord detection algorithms Merlin and Matrix Profile do not require a training step. In the case of Merlin, which explores an interval of window lengths [α _min , α _max ], α _min is made equal to the smallest of the top-3 base windows, and α _max equal to the minimum between the largest of the top-3 base windows, and 2000. These values were achieved after preliminary tests where the trade-off between the computational performance and the AUC maximization was prioritized. Regarding the Matrix Profile algorithm, three α values were evaluated, each corresponding to one of the top three base window lengths.

After establishing the values of the hyper-parameters used by the TSAD algorithms in 𝒜, we proceed to the analysis of their performance for the entire set of benchmark time series. It is important to note that the evaluation is conducted exclusively on the test set to ensure a fair comparison with the model-based algorithms. Figure 4 presents the AUC performance distribution for the four TSAD algorithms of our study: DeepAnt, TCNAE, Merlin, and Matrix Profile. We can observe that Matrix Profile exhibits the highest median AUC, followed by DeepAnt. In addition, the Matrix Profile demonstrates a narrow interquartile range, indicating stable performance.

Figure 4
Boxplot representation of AUC performance metrics for the time series anomaly detection algorithms DeepAnt, TCNAE, Merlin, and Matrix Profile (MP).

Figure 5 segments the results presented in Figure 4 by benchmark dataset. We observe that Matrix Profile and Merlin exhibit the best AUC performance distribution in time series instances of the UCR dataset. In contrast Matrix Profile outperformes the other algorithms in the YAHOO dataset. Finally, for the NAB dataset, TCN-AE demonstrated the best overall performance. This varied performance distribution across different benchmark datasets results from the diversity of their time series instances, emphasizing the need for a meta-learning model that can capture the unique features of each time series to select the most suitable anomaly detection algorithm.

Figure 5
Boxplot representation of AUC performance metrics for time series anomaly detection algorithms DeepAnt, TCNAE, Merlin, and Matrix Profile (MP) segmented by benchmark dataset.

6.3 Performance of Meta-learner

In this section, we evaluate the ranking results obtained by our meta-model for anomaly detection in time series. For this, the time series contained in each benchmark dataset are split using 5-fold cross-validation. Thus, five batches containing 80% of the time series instances are composed for training, with the remaining 20% used for evaluation.

We note that all steps of the meta-learning pipeline are performed strictly within each training fold to avoid data leakage. In particular, preprocessing (including z-score normalization and feature selection) is fitted exclusively on the training data of each fold and then applied to the corresponding test set, ensuring that no information from the test instances is used during model training.

6.3.1 Meta-features extraction and selection

For the meta-feature extraction process, we employed a systematic approach to capture the essential characteristics of the time series that would be useful for algorithm selection. We analyzed 73 meta-features relevant to time series data. They consist of (i) the initial 30 meta-features from the R package TSfeatures, (ii) the 9 entropy-based and fractal-based features from the Entropy repository² known as Antropy, and (iii) the 22 attributes from catch22. These meta-features were extracted using established libraries and methodologies drawn from prior studies, ensuring a well-rounded representation of each time series. For further details on the specific tools and methods used in the extraction process, please refer to Appendix A.

In the sequel, the selection of meta-features for the meta-learner followed a method similar to that proposed by Smith-Miles & Muñoz (2023) (see Appendix B for details). This method aimed to eliminate highly correlated meta-features while keeping those that retained the most information from each created cluster. Figure 6 presents a scatter plot of the two principal components derived from the selected meta-features of each time series from the three benchmarks. This visualization allows us to explore the diversity and distribution of the used datasets in a lower-dimensional space. We can observe that the distribution of the benchmark dataset is immediately apparent. Notably, the UCR and YAHOO time series data are well-separated, highlighting their distinct characteristics.

Figure 6
Scatter plot visualization of the time series from the three benchmark datasets - UCR, NAB, and YAHOO - through a PCA projection based on the selected meta-features. Each point represents an instance projected in the first two principal components.

6.3.2 Performance analysis

As described in Section 5, binary classifiers are used as primary models to predict the AUC performance of each TSAD algorithm in our algorithm space 𝒜 for a given time series instance. We define the performance as “good” if y(i, a) ≥ τ, indicating that the AUC performance achieved by algorithm a ∈ 𝒜 on instance i ∈ 𝒫 is equal to or greater than a predefined threshold τ. In this study, we set τ = 0.8 as in Kandanaarachchi et al. (2020). Thus, to establish the relationship between the meta-features and the binary “goodness” performance meta-target for each TSAD algorithm, we trained and compared three distinct binary classification algorithms: Logistic Regression (LR), Random Forest (RF), and XGBoost (XGB).

Table 1 presents the performance of each meta-model when using each of the three binary classifiers (LR, RF or XGB) to predict the goodness of the AUC performance of the TSAD algorithms in 𝒜. The optimization of the hyper-parameters of the binary classifiers was performed using Optuna (Akiba et al., 2019) - more details can be found in Appendix C. We observe that RF and XGB yield the most effective meta-models for this task. The meta-model using XGB slightly outperforms RF for DeepAnt, Matrix Profile, and Merlin, while the latter performs better for TCNAE.

Table 1
Performance of meta-models on the goodness of the AUC performance of each TSAD algorithm.

Algorithm	Meta-Model	Precision	Recall	F1-score
DeepAnt	LR	0.82	0.77	0.79
	RF	0.87	0.85	0.86
	XGB	0.87	0.87	0.87
Matrix Profile	LR	0.92	0.84	0.88
	RF	0.92	0.90	0.91
	XGB	0.92	0.92	0.92
Merlin	LR	0.69	0.70	0.70
	RF	0.77	0.76	0.76
	XGB	0.82	0.77	0.79
TCNAE	LR	0.65	0.68	0.66
	RF	0.74	0.75	0.74
	XGB	0.74	0.70	0.72

Given that the ultimate task of our meta-learning model is to recommend a TSAD algorithm for a given time series instance, we must rank the TSAD algorithms with respect to their predicted likelihood of achieving “good” AUC performance. Thus, four prediction values are sorted for each meta-model. From this ranking, the following metrics are computed:

• Spearman’s rank correlation coefficient: This measure quantifies how well the model’s predicted ranking aligns with the ground-truth order. Spearman’s rank correlation, denoted as ρ, is calculated using the formula:

$\rho =1-\frac{6\Sigma d_i^2}{n\left(n^2-1\right)},$

• where d _i represents the difference between the ranks of the i-th label in the predicted and true rankings, and n is the number of instances evaluated. In our case, one label exists for each TSAD algorithm in 𝒜.

• Kendall rank correlation coefficient: This coefficient measures the degree of agreement between the model’s predicted ranking and the ground-truth rank by computing the number of times they agree and disagree. Kendall rank correlation (τ) is calculated using the following equation:

$\tau =\frac{C-D}{{\displaystyle \frac{1}{2}}n\left(n-1\right)},$

• where C and D are the numbers of concordant and discordant pairs, respectively, and n is the number of instances evaluated.

• Top-k: Let T _k be the set of instances where the predicted best algorithm is within the top-k ranking in the ground-truth. The top-k is calculated as

$\text{Top-}k=\frac{\left|T_k\right|}{n}.$

The Top-k metric is particularly relevant from a practical standpoint, as it reflects decision-making scenarios in which a practitioner may evaluate a small subset of recommended algorithms rather than relying on a single selection. In many real-world applications, testing the top few candidate algorithms is computationally feasible and reduces the risk associated with selecting a single method. Therefore, strong Top-k performance indicates that the proposed framework can effectively narrow down the set of promising algorithms, supporting more robust and effective decision-making in practice.

Table 2 presents the values of the above metrics for the three evaluated meta-models (based on LR, RF, and XGB). The results indicate that the meta-model based on RF outperforms the others, achieving a Spearman correlation of 0.61 and a Kendall τ correlation of 0.52. These correlations suggest a positive relationship between the predicted and observed ranks. Moreover, although the meta-learning models exhibit good values for the Top-1 metric, we observe a substantial increase for all evaluated meta-models from the Top-1 to the Top-2 metrics. This means that when the meta-models fail to pinpoint the top-performing TSAD algorithm, they often position it as the second-best option.

Table 2
Performance comparison of machine learning models Logistic Regression (LR), Random Forest (RF), and XGBoost (XGB) based on Spearman and Kendall correlation coefficients, and Top-1 and Top-2 accuracy metrics.

ML	Spearman	Kendall	Top-1	Top-2
LR	0.53	0.46	0.54	0.84
RF	0.61	0.52	0.64	0.89
XGB	0.57	0.47	0.62	0.92

To further analyze the ranking performance of the meta-models, we examined the difference between the AUC of the best-performing algorithm and the second-best TSAD algorithm for each time series instance. Figure 7 presents a histogram of these differences. As we can observe, most instances (314 ≈ 48%) fall within the first bin (0 to 0.05), suggesting that the performance gap between the best and second-best algorithms is often negligible. This highlights the challenge of the ASP in such cases.

Figure 7
Histogram showing the distribution of the difference between the AUC values obtained by the best-performing and the second-best performing TSAD algorithms in each instance.

Table 3 summarizes the Random Forest meta-model performance metrics for each TSAD algorithm. The column #best indicates the number of time series instances where the respective TSAD algorithm is the best among its peers. The precision metric measures the proportion of instances where the algorithm was correctly predicted as the best (true positives) relative to all predictions where it was identified as the best. The recall metric captures the proportion of instances where the algorithm was correctly predicted as the best (true positives) relative to the total number of times it was actually the best. Finally, the F1-score is the harmonic mean of precision and recall, offering a balanced assessment of the meta-model’s performance.

Table 3
Comparison of #best, Precision, Recall, and F-score metrics for the top-ranked algorithms. Here, #best refers to the number of times each algorithm performed better than the others in terms of AUC.

Algorithm	#best	Precision	Recall	F-score
DeepAnt	256	0.85	0.65	0.74
TCNAE	72	0.49	0.39	0.49
Merlin	65	0.36	0.32	0.34
Matrix Profile	257	0.58	0.72	0.64

The precision metric reveals distinct patterns across the TSAD algorithms. The RF meta-model for DeepAnt demonstrates a high precision of 0.85, indicating that the meta-learning approach accurately identifies DeepAnt as the best algorithm in the majority of recommended cases. In contrast, the RF meta-model yields lower precision values of 0.49 and 0.36 for TCNAE and Merlin, respectively, indicating that it delivers less consistent predictions about the best-performing algorithm when these algorithms are identified as the best. A high recall indicates that the chosen algorithm captures more true positive cases. The RF meta-model presents the best recall metrics for Matrix Profile and DeepAnt (0.72 and 0.65, respectively). Overall, the metrics reported in Table 3 are influenced by those presented in Table 1, indicating that predicting Merlin and TCNAE as the best-performing TSAD algorithm is a more challenging task.

7 CONCLUDING REMARKS

This paper presented a comprehensive framework for meta-learning applied to univariate time series anomaly detection. By systematically extracting and analyzing meta-features from benchmark datasets, we could predict the performance of various anomaly detection algorithms and rank them accordingly. This approach enhances our understanding of which algorithms are best suited for specific types of instances and provides a robust methodology for improving algorithm selection in practical TSAD applications.

Our experimental results demonstrated the effectiveness of using meta-learning to guide the selection of TSAD algorithms, showing that the framework can often identify the most appropriate algorithms for different instances. To validate the effectiveness of our approach, metrics such as Spearman’s rank correlation, Kendall’s Tau, and accuracy in ranking the best algorithms were used.

Furthermore, our work highlights the challenges associated with detecting anomalies for a diverse set of time series benchmark data. These insights are crucial for advancing in the development of more sophisticated anomaly detection models and algorithms that can manage the heterogeneous and frequently unpredictable characteristics of real-world time series data. We also investigated the impact of the input window length on the performance of deep-learning anomaly detection algorithms.

Finally, we note that our approach does not incorporate information about the anomalies themselves, such as their nature, frequency, or patterns within the time series. We believe that incorporating meta-features specifically focused on the anomalies could significantly enhance the predictive power of the meta-learner. By analysing the anomaly characteristics, the meta-learning model could better differentiate between instances, resulting in more accurate rankings of TSAD algorithms. However, in most real-world situations, detailed information about anomalies is often unavailable or incomplete. Future research could focus on overcoming this limitation by developing methods that infer or approximate anomaly characteristics in an unsupervised or semi-supervised manner, further refining the meta-learning framework and enhancing its applicability in real-world contexts where anomaly information is scarce or absent.

Data Availability

The data that support the findings will be available under request.

References

• AHMAD S, LAVIN A, PURDY S & AGHA Z. 2017. Unsupervised real-time anomaly detection for streaming data. Neurocomputing, 262: 134-147.
• AKIBA T, SANO S, YANASE T, OHTA T & KOYAMA M. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
• AUDIBERT J, MICHIARDI P, GUYARD F, MARTI S & ZULUAGA MA. 2022. Do deep neural networks contribute to multivariate time series anomaly detection? Pattern Recognition, 132: 108945.
• BENAVOLI A, CORANI G & MANGILI F. 2016. Should we really use post-hoc tests based on mean-ranks? The Journal of Machine Learning Research, 17(1): 152-161.
• BLÁZQUEZ-GARCÍA A, CONDE A, MORI U & LOZANO JA. 2021. A review on outlier/anomaly detection in time series data. ACM Computing Surveys (CSUR), 54(3): 1-33.
• BRANDES U, DELLING D, GAERTLER M, GORKE R, HOEFER M, NIKOLOSKI Z & WAGNER D. 2007. On modularity clustering. IEEE transactions on knowledge and data engineering, 20(2): 172-188.
• BRAZDIL P, CARRIER CG, SOARES C & VILALTA R. 2008. Metalearning: Applications to data mining. Springer Science & Business Media.
• CHANDOLA V, BANERJEE A & KUMAR V. 2009. Anomaly detection: A survey. ACM Computing Surveys (CSUR) , 41(3): 1-58.
• CORREIA L, GOOS JC, KLEIN P, BÄCK T & KONONOVA AV. 2024. Online model-based anomaly detection in multivariate time series: Taxonomy, survey, research challenges and future directions. Engineering Applications of Artificial Intelligence, 138: 109323.
• ERMSHAUS A, SCHÄFER P & LESER U. 2023. Window Size Selection in Unsupervised Time Series Analytics: A Review and Benchmark. In: International Workshop on Advanced Analytics and Learning on Temporal Data. pp. 83-101. Springer.
• FERNANDES LHDS, LORENA AC & SMITH-MILES K. 2021. Towards understanding clustering problems and algorithms: an instance space analysis. Algorithms, 14(3): 95.
• FRANCISQUINI R, LORENA AC & NASCIMENTO MCV. 2022. Community-based anomaly detection using spectral graph filtering. Applied Soft Computing, 118: 108489.
• HE Y, CHU X & WANG Y. 2020. Neighbor profile: Bagging nearest neighbors for unsupervised time series mining. In: 2020 IEEE 36th International Conference on Data Engineering (ICDE). pp. 373-384. IEEE.
• HUNDMAN K, CONSTANTINOU V, LAPORTE C, COLWELL I & SODERSTROM T. 2018. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. pp. 387-395.
• JOMAA HS, SCHMIDT-THIEME L & GRABOCKA J. 2021. Dataset2vec: Learning dataset meta-features. Data Mining and Knowledge Discovery, 35: 964-985.
• KAHRAMAN A, HOU P, YANG G & YANG Z. 2019. Comparison of the Effect of Regularization Techniques and Lookback Window Length on Deep Learning Models in Short Term Load Forecasting. In: The Purple Mountain Forum on Smart Grid Protection and Control. pp. 655-669. Springer.
• KANDA J, DE CARVALHO A, HRUSCHKA E, SOARES C & BRAZDIL P. 2016. Meta-learning to select the best meta-heuristic for the traveling salesman problem: A comparison of meta-features. Neurocomputing , 205: 393-406.
• KANDANAARACHCHI S, MUÑOZ MA, HYNDMAN RJ & SMITH-MILES K. 2020. On normalization and algorithm selection for unsupervised outlier detection. Data Mining and Knowledge Discovery , 34(2): 309-354.
• KARIMI-MAMAGHAN M, MOHAMMADI M, MEYER P, KARIMI-MAMAGHAN AM & TALBI EG. 2022. Machine learning at the service of meta-heuristics for solving combinatorial optimization problems: A state-of-the-art. European Journal of Operational Research, 296(2): 393-422.
• KEOGH E, LIN J & FU A. 2005. Hot sax: Efficiently finding the most unusual time series subsequence. In: Fifth IEEE International Conference on Data Mining (ICDM’05). pp. 8-pp. Ieee.
• KIM J, KANG H & KANG P. 2023. Time-series anomaly detection with stacked Transformer representations and 1D convolutional network. Engineering Applications of Artificial Intelligence , 120: 105964.
• LAW SM. 2019. STUMPY: A Powerful and Scalable Python Library for Time Series Data Mining. The Journal of Open Source Software, 4(39): 1504.
• LUBBA CH, SETHI SS, KNAUTE P, SCHULTZ SR, FULCHER BD & JONES NS. 2019. catch22: CAnonical Time-series CHaracteristics: Selected through highly comparative time-series analysis. Data Mining and Knowledge Discovery , 33(6): 1821-1852.
• MUNIR M, SIDDIQUI SA, DENGEL A & AHMED S. 2018. DeepAnT: A deep learning approach for unsupervised anomaly detection in time series. Ieee Access, 7: 1991-2005.
• NAKAMURA T, IMAMURA M, MERCER R & KEOGH E. 2020. Merlin: Parameter-free discovery of arbitrary length anomalies in massive time series archives. In: 2020 IEEE international conference on data mining (ICDM). pp. 1190-1195. IEEE.
• PAGE L, BRIN S, MOTWANI R, WINOGRAD T ET AL. 1999. The pagerank citation ranking: Bringing order to the web. Tech. rep.. Stanford infolab.
• PRUDÊNCIO RB & LUDERMIR TB. 2004. Meta-learning approaches to selecting time series models. Neurocomputing , 61: 121-137.
• RICE JR. 1976. The algorithm selection problem. In: Advances in computers, vol. 15. pp. 65-118. Elsevier.
• SMITH-MILES K & LOPES L. 2012. Measuring instance difficulty for combinatorial optimization problems. Computers & Operations Research, 39(5): 875-889. Available at: https://www.sciencedirect.com/science/article/pii/S0305054811001997
  » https://www.sciencedirect.com/science/article/pii/S0305054811001997
• SMITH-MILES K & MUÑOZ MA. 2023. Instance space analysis for algorithm testing: Methodology and software tools. ACM Computing Surveys, 55(12): 1-31.
• SMITH-MILES K & TAN TT. 2012. Measuring algorithm footprints in instance space. In: 2012 IEEE congress on evolutionary computation. pp. 1-8. IEEE.
• TALAGALA TS, HYNDMAN RJ, ATHANASOPOULOS G ET AL. 2018. Meta-learning how to forecast time series. Monash Econometrics and Business Statistics Working Papers, 6(18): 16.
• THILL M, KONEN W, WANG H & BÄCK T. 2021. Temporal convolutional autoencoder for unsupervised anomaly detection in time series. Applied Soft Computing , 112: 107751.
• VANSCHOREN J. 2019. Meta-learning. Automated machine learning: methods, systems, challenges, pp. 35-61.
• WANG X, SMITH-MILES K & HYNDMAN R. 2009. Rule induction for forecasting method selection: Meta-learning the characteristics of univariate time series. Neurocomputing , 72(10-12): 2581-2594.
• WIDODO A & BUDI I. 2013. Model selection using dimensionality reduction of time series characteristics. In: International Symposium on Forecasting, Seoul, South Korea. pp. 57-118.
• WU R & KEOGH E. 2021. Current time series anomaly detection benchmarks are flawed and are creating the illusion of progress. IEEE transactions on knowledge and data engineering , .
• YANKOV D, KEOGH E & REBBAPRAGADA U. 2008. Disk aware discord discovery: Finding unusual time series in terabyte sized datasets. Knowledge and Information Systems, 17: 241-262.
• ZHOU Y, QIN R, XU H, SADIQ S & YU Y. 2018. A data quality control method for seafloor observatories: The application of observed time series data in the East China Sea. Sensors, 18(8): 2628.
• ZIMMERMAN Z, KAMGAR K, SENOBARI NS, CRITES B, FUNNING G, BRISK P & KEOGH E. 2019. Matrix profile XIV: scaling time series motif discovery with GPUs to break a quintillion pairwise comparisons a day and beyond. In: Proceedings of the ACM Symposium on Cloud Computing. pp. 74-86.

• 1
  Available at https://webscope.sandbox.yahoo.com.
• 2
  Available at https://raphaelvallat.com/antropy/build/html/index.html.
• Funding
  The authors received he financial support from the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) (Finance Code 001, Grant: 88887.507037/2020-00), and CNPq (grants 309385/2021-0, 403735/2021-1). This research was conducted using the computational resources of the Center for Mathematical Sciences Applied to Industry (CeMEAI), funded by FAPESP (grant 2013/07375-0).

Appendix A TIME SERIES META-FEATURES

This section provides an overview of the meta-features extracted to characterize the time series used in this study. These meta-features capture the time series’ statistical, structural, and temporal properties, which are essential for predicting the performance of anomaly detection algorithms. Table A1 presents a detailed list of the meta-features, including a brief description and their sources. The meta-features selected through the feature selection process are highlighted in bold.

Table A1
Meta-features used in the analysis, their descriptions, and sources.

Meta-feature	Description	Source
alpha	Alpha parameter from Holt decomposition	Tsfeatures
approximate_entropy	Approximate entropy	Antropy
ARCH.LM	R2 value of an autoregressive model of order specified as lags applied to x 2	TSfeatures
arch_acf	Sum of squares of the first 12 autocorrelations of $x_t^2$	TSfeatures
arch_r2	R2 value of an autoregressive model applied to $x_t^2$	TSfeatures
benford_cor	Correlation from first digit distribution when compared to the Newcomb-Benford’s Law distribution	TSfresh
beta	Beta parameter from Holt decomposition	Tsfeatures
CO_Embed2_Dist_tau_d_expfit_meandiff	Exponential fit to successive distances in 2-D embedding space	Catch22
CO_f1ecac	First $\frac{1}{e}$ crossing of autocorrelation function	Catch22
CO_FirstMin_ac	First minimum of autocorrelation function	Catch22
CO_HistogramAMI_even_2_5	Automutual information, m = 2, τ = 5	Catch22
CO_trev_1_num	Time-reversibility statistics, ⟨(x t+1 − x t ) 3 ⟩ t	Catch22
Crossing_points	Number of times a time series crosses the median line	TSfeatures
curvature	Curvature of a time series calculated based on the coefficients of an orthogonal quadratic regression	TSfeatures
detrended_fluctuation	The estimate α for the Hurst parameter	Antropy
diff1_acf1	First autocorrelation function of the differenced series	TSfeatures
diff1_acf10	Sum of squares of the first 10 autocorrelation functions of the differenced series	TSfeatures
diff2_acf1	First autocorrelation function of the twice-differenced series	TSfeatures
diff2_acf10	Sum of squares of the first 10 autocorrelation functions of the twice-differenced series	TSfeatures
DN_HistogramMode_10	Mode of z-scored distribution (10-bin histogram)	Catch22
DN_HistogramMode_5	Mode of z-scored distribution (5-bin histogram)	Catch22
DN_OutlierInclude_n_001_mdrmd	Time intervals between successive extreme events below the mean	Catch22
DN_OutlierInclude_p_001_mdrmd	Time intervals between successive extreme events above the mean	Catch22
entropy	Shannon entropy of the spectral density of the data	Tsfeatures
e_acf1	First autocorrelation function of the remainder component of the STL decomposition	TSfeatures
e_acf10	Sum of squares of the first 10 autocorrelation functions of the remainder component of the STL decomposition	TSfeatures
e_mean	Mean of absolute value of the remainder component (Mean(|e t |))	TSfeatures
e_var	Variance of absolute value of the remainder component (Var(|e t |))	Tsfeatures
FC_LocalSimple_mean1_tauresrat	Change in correlation length after iterative differencing	Catch22
FC_LocalSimple_mean3_stderr	Mean error from a rolling 3-sample mean forecasting	Catch22
garch_acf	Sum of squares of the first 12 autocorrelations of $z_t^2$	TSfeatures
garch_r2	R2 value of an AR model applied to $z_t^2$	Tsfeatures
higuchi_fd	Higuchi Fractal Dimension	Antropy
hjorth_complexity	Hjorth Complexity	Antropy
hjorth_mobility	Hjorth Mobility	Antropy
hurst	Hurst coefficient	Tsfeatures
IN_AutoMutualInfoStats_40_gaussian_fmmi	First minimum of the automutual information function	Catch22
katz_fd	Katz Fractal Dimension	Antropy
kurtosis	Kurtosis	TSfresh
length	Number of observations in the time series	None
linearity	Linearity of a time series calculated based on the coefficients of an orthogonal quadratic regression	TSfeatures
MD_hrv_classic_pnn40	Proportion of difference magnitudes that are greater than 4% of the standard deviation of the time series	Catch22
nperiods	Number of periods in the time series $\frac{len\left(X\right)}{period}$	None
PD_PeriodicityWang_th0_01	First peak in the autocorrelation function satisfying a set of conditions	Catch22
peak	Strength of peaks	Tsfeatures
petrosian_fd	Petrosian Fractal Dimension	Antropy
qdist1_3	Normalized interquantile range of the remainder component (e t ) of STL	None
qdist3_4	Normalized distance between the third quartile and the extreme value	None
SB_BinaryStats_diff_longstretch0	Longest sequence of successive steps in the time series that decrease	Catch22
SB_BinaryStats_mean_longstretch1	Longest sequence of successive values in the time series that are greater than the mean	Catch22
SB_MotifThree_quantile_hh	Shannon entropy of two successive letters in equiprobable 3-letter symbolization	Catch22
SB_TransitionMatrix_3ac_sumdiagcov	Trace of covariance of transition matrix between symbols in 3-letter alphabet	Catch22
SC_FluctAnal_2_dfa-50_1_2_logi_prop_r1	Proportion of slower timescale fluctuations that scale with DFA (50% sampling)	Catch22
SC_FluctAnal_2_rsrangefit_50_1_logi_prop_r1	Proportion of slower timescale fluctuations that scale with linearly rescaled range fits	Catch22
seasonal_period	Seasonal Period	Tsfeatures
seasonal_strength	$1-\frac{Var\left(e_t\right)}{Var\left(s_t+e_t\right)}$	TSfeatures
seas_acf1	Sum of the first p (period) autocorrelation function of the series	TSfeatures
skewness	Skewness	TSfresh
spectral_entropy	Spectral Entropy	Antropy
spike	“Spikiness” of a time series, computed as the variance of the leave-one-out variances of the remainder component e t	TSfeatures
SP_Summaries_welch_rect_area_5_1	Total power in lowest fifth of frequencies in the Fourier power spectrum	Catch22
SP_Summaries_welch_rect_centroid	Centroid of the Fourier power spectrum	Catch22
svd_entropy	Singular Value Decomposition entropy	Antropy
trend	$1-\frac{Var\left(e_t\right)}{Var\left(f_t+e_t\right)}$	TSfeatures
trough	Strength of trough	TSfeatures
unitroot_kpss	Vector comprising the statistic for the KPSS unit root test with linear trend and lag one	TSfeatures
unitroot_pp	Statistic for the “Z-alpha” version of PP unit root test with constant trend and lag one	TSfeatures
x_acf1	First autocorrelation coefficient	TSfeatures
x_acf10	Sum of squares of the first 10 autocorrelation coefficients	TSfeatures

Appendix B META-FEATURES SELECTION

To perform the meta-feature selection, we employed a method similar to that performed by (Smith-Miles & Muñoz, 2023), to which we calculated the correlation between all the meta-features extracted and created a correlation graph denoted as G(f , w). In this graph, f represents the node set of meta-features, where each node corresponds to a meta-feature of all tested instances (a node is referred to as f _k ∈ ℝ^|𝒫|). The weight of the edge w _k,j linking vertices f _k and f _j is defined as the absolute value of the correlation coefficient |ρ(f _k , f _j )|. If |ρ(f _k , f _j )| ≥ σ, where σ is a threshold hyperparameter, w _k,j assumes the value of the correlation coefficient; otherwise, it is set to zero.

To select the features, we identified clusters using a model that maximizes the modularity measure (Brandes et al., 2007), creating clusters that contain highly correlated meta-features. The selection method involves choosing one meta-feature per cluster, specifically the one with the highest PageRank centrality value (Page et al., 1999). Using this approach, we remove highly correlated meta-features, keeping those that best represent each cluster based on the centrality measure.

Figure B1 shows the graph created using this methodology, with the threshold σ set to 0.7. This configuration created this graph containing 52 vertices and 14 clusters with a modularity score of 0.75, indicating substantial structural organization in the network; 14 meta-features were extracted from the initial 52 highly correlated meta-features, resulting in 36 meta-features in total.

Figure B1
Graph representation of the feature selection process. The coloured areas represent clusters of meta-features. The vertices represents the meta-features, orange vertices are removed meta-features and the green ones are the selected. The blue edges represents positive correlation and red negative.

Appendix C META-LEARNING HYPERPARAMETER OPTIMIZATION

In this section, we describe the process of hyperparameter optimization performed for three machine learning models - Random Forest, XGBoost, and Logistic Regression - using the Optuna framework (Akiba et al., 2019). This optimization aims to enhance the performance of these models by selecting the best hyperparameters, thereby improving their accuracy for the binary classification on the performance of each TSAD algorithm.

To achieve this, we first split the data into a training set (70%) and a test set (30%). The Optuna framework was then employed to optimize the hyperparameters of all meta-learners across all the algorithms, ensuring a systematic and data-driven search for the best configuration.

C.1 Random Forest

Hyperparameters were optimised for the Random Forest algorithm to balance model complexity and generalization. The number of trees (n estimators) was tuned to control the trade-off between accuracy and computational cost, with values ranging from 50 to 300 across different datasets. The depth of the trees (max depth) was optimized to manage model complexity, with deeper trees allowing for more detailed data representation, but with the risk of overfitting if set too high. The min samples split and min samples leaf hyperparameters were optimized to prevent overfitting by controlling the minimum number of samples required to split a node and form a leaf. These values, ranging from 1 to 10 for min samples split and 1 to 10 for min samples leaf, ensured that the trees did not grow too specific, helping the model generalize better on unseen data. By carefully tuning these hyperparameters, the Random Forest models were better equipped to handle the complexity of the data while minimizing overfitting. Table C1 shows the hyperparameters found by Optuna.

Table C1
Random Forest.

Algorithm	n_estimators	max_depth	min_samples_split	min_samples_leaf
DeepAnt	80	17	3	1
TCNAE	50	13	10	2
Matrix Profile	104	14	4	2
Merlin	237	13	5	9

C.2 XGBoost

For XGBoost, several key hyperparameters were optimized to balance model performance and efficiency. The number of boosting rounds, controlled by n estimators, was tuned between 50 and 300 to find the right trade-off between accuracy and computational cost. The max depth parameter, which determines the maximum depth of each tree, was optimized in the range of 2 to 20 to manage model complexity and avoid overfitting. The learning rate (or eta), ranging from 0.01 to 0.3, controlled the step size for each boosting round, balancing the model’s learning speed and precision. Additionally, the colsample bytree parameter, which dictates the fraction of features used for building each tree, was tuned between 0.5 and 1.0 to introduce randomness and prevent overfitting. These hyperparameters helped create a robust model that efficiently captures data patterns while minimizing overfitting. Table C2 shows the hyperparameters found by Optuna.

Table C2
XGBoost.

Algorithm	n_estimators	max_depth	learning_rate	colsample_bytree
DeepAnt	164	13	0.0052	0.2059
TCNAE	65	19	0.0002	0.8496
Matrix Profile	224	12	0.1257	0.2954
Merlin	75	12	0.0279	0.3177

C.3 Logistic Regression.

Logistic Regression, the key hyperparameters optimized were C and penalty. The parameter C, which controls the strength of regularization, was tuned on a logarithmic scale between 10⁻⁶ and 10² . A lower value of C imposes stronger regularization by penalizing large coefficients, which helps prevent overfitting. A higher value reduces the regularization effect, allowing the model to focus more on fitting the training data. The penalty hyperparameter determines the type of regularization applied and was optimized between L1 (Lasso) and L2 (Ridge). L1 regularization encourages sparsity by shrinking some coefficients to exactly zero, while L2 regularization penalizes the sum of squared coefficients, resulting in smaller but non-zero values. Both hyperparameters play a crucial role in avoiding overfitting and improving generalization. Table C3 shows the hyperparameters found by Optuna.

Table C3
Logistic Regression.

Algorithm	C	penalty
DeepAnt	0.2467	l1
TCNAE	0.3813	l2
Matrix Profile	0.3740	l1
Merlin	0.2079	l1

Edited by

• Editor responsible for the review
  Editor-in-Chief: Antônio Augusto Chaves

Publication Dates

• Publication in this collection
  31 July 2026
• Date of issue
  2026

History

• Received
  11 Nov 2025
• Accepted
  23 Feb 2026