Collaborations

Temporal AI: Learning How Systems Change

Time series are a natural entry point into one of AI’s great unsolved problems: the ability to remember observations across long horizons, even when they seem irrelevant at the moment they are perceived. Aionic Labs is building temporal AI to understand this changing world.

By Martin Maritsch and Patrick Langer
About Aionic Labs
Aionic Labs is building temporal foundation models that help people and AI systems understand and act on changes in physical and digital systems. OpenTSLM connects operational signals with language and context, while TimeNet provides the shared data infrastructure to train and evaluate these models.

When a machine fails, an engineer often returns to the sensor history to understand what happened. A small change in vibration that attracted little attention days earlier can become evidence of developing damage. The engineer has to connect it to operating conditions, context, and the failure that followed.

Time series are measurements recorded over time. Most commonly, this refers to sensor data, but can really mean any irregular or regular measurement; like sequence of event logs or images. Time series describe evolving systems, from industrial machinery (vibration, pressure, temperature) over financial markets (stock prices, sales) to the human body (heart rate, blood oxygen, vital signs).  Understanding a measurement requires more than just reading its value at a single moment. Only by combining measurements over time with the surrounding context can we uncover hidden patterns, detect early failures, and make informed decisions.

Existing AI tools fail to address this process. Traditional machine learning often uses separate algorithms or specific models for each sensor configuration. Time series foundation models go a step further by learning shared representations across large datasets, yet they are still limited to fixed tasks and domains, cannot be flexibly instructed in natural language, and remain blind to broader operational context.

Language models offer a way to flexibly formulate tasks, integrate context, ask questions, and discuss findings, but have no interface for sensor data. Even if we convert raw observations into a prompt, the model has no innate concept of when something was recorded, the data is simply flattened into text, stripping out the structural and temporal relationships that give signals their meaning.

At Aionic Labs, we are developing Temporal AI to bridge deep temporal understanding, sensor data integration, and natural language reasoning so we can make sense of how systems evolve over time. With OpenTSLM we created models that bring continuous temporal signals directly into language models. For the next-generation of Temporal AI, we introduced TimeNet which provides a shared data infrastructure to train and evaluate them. Together, they form the foundation for models that can draw on a system’s full history, helping people investigate subtle changes, reason through complexity, and close the gap between signal and decision.

Teaching AI to Understand Signals

OpenTSLM is an open family of time-series language models (TSLMs) that can perceive time series as native modality next to text, enabling joint reasoning over both. A user can provide measurements, describe their context, ask questions or provide instructions about an observation. The model generates an answer in natural language and provides an explanation the user can access. The TSLM is able to perceive raw time series data via a specific gated cross-attention mechanism that allows it to repeatedly query representations of the time series produced by a time-series encoder. Because the cross-attention mechanism and encoder are trained end-to-end, the cross-attention mechanism essentially learns to “ask” the encoder for a suitable representation.

Figure 1. OpenTSLM-Flamingo connects ECG recordings with patient context and text through cross-attention. TimeNet supports shared research data. Illustrative workflow.

The native integration of time series data alongside text into powerful and general LLMs unlocks entirely new workflows for complex systems. Reconsider the machine failure described earlier: rather than treating signal analysis as an isolated task, the engineer can combine vibration recordings with operational context and prompt the model for explanations. OpenTSLM verbalizes relevant findings from the temporal data, helping the engineer navigate the data and decide where to focus their investigation.

Beyond manufacturing, this unlocks many use cases in clinical settings, such as automated Electrocardiogram (ECG) interpretation. An ECG records the heart’s electrical activity, which clinicians interpret alongside patient context. OpenTSLM-Flamingo received these inputs and generated answers with explanations. Five Stanford cardiologists reviewed 84 randomly sampled explanations, with at least two reviewers assessing each example. They assessed ECG feature recognition, clinical reasoning, and use of patient context. The study reported correct or partially correct interpretations in 92.9% of the reviewed cases.

These experiments evaluate models trained for specific tasks and the quality of their generated explanations. Establishing reliability in a deployed system requires further evaluation. We release code, model weights, and datasets so other researchers can inspect the work, reproduce experiments, and develop it further.

Laying the Foundation for Next-Gen Temporal AI

Training models across tasks and domains starts with a data problem. Public datasets use different formats, annotations, and task definitions. Researchers must reconcile these differences before they can bring datasets into the same workflow. Operational data presents a further challenge because recordings and the expertise needed to interpret them often remain within organizations.

TimeNet provides a common format and interface for time-series datasets across domains. Signals retain their annotations and task definitions, so researchers can reuse recordings across tasks without dataset-specific loading code. Dataset versioning makes experiments easier to reproduce. These capabilities let us work toward shared models that learn across tasks and domains.

Access is part of this work. Reusing data requires permission, as well as a format that makes it usable. We are developing open infrastructure alongside trusted data partnerships. TimeNet’s documentation gives researchers a starting point for using the format, contributing datasets, and building on the infrastructure.

Reasoning Over Long-Context Temporal Horizons

Once models can access more data, the next challenge is using longer histories effectively. Longer inputs require more memory and computation, while summarizing them risks losing details whose relevance may only become clear later. To assess the capability of models to understand long-horizon time series data, we previously introduced the TS-Haystack benchmark. It tests whether models can find events in recordings of physiological signals, activity, and household power. We find that models often lose accuracy as recordings grow longer.

Figure 2. TS-Haystack tests event retrieval across longer recordings. The evaluated models often lose accuracy as recordings grow. Source: TS-Haystack, Figure 4.

To overcome the limitations of prior models on long-horizon time series data, our work on TimeRLM adds a new capability enabling models to actively “explore” these recordings. Using reinforcement learning, models learn to “zoom in” on different aspects of a time series, either natively or through additional tool calls and visual inspection. We evaluate TimeRLM on TS-Haystack, AnomalyXL (a new dataset we introduce), and ARFBench. On AnomalyXL-Localize, a synthetic benchmark with deliberately inserted anomalies, we find that TimeRLM outperforms the evaluated baselines on four out of five tasks. On ARFBench, TimeRLM enables models to analyze and reason over long-horizon time series data. By combining this capability with context about the machines and systems being monitored, we can build agents that follow instructions, investigate anomalies, and reason about potential causes using both the recorded signals and their real-world context.

Models that Understand Evolving Systems

Together, long-horizon reasoning and context about the systems being monitored lay the foundation for time series agents that can follow instructions, investigate complex problems, and connect observations across time. But finding relevant evidence is only part of the challenge. Models also need continuity: the ability to retain observations and revisit them when new evidence changes their meaning. They simply cannot fit all data in a context window, but need to remember “when” interesting patterns occurred.

For a machine, this means connecting an earlier change in vibration with a later failure. The challenge is preserving useful details before it becomes clear which observations will matter.

Figure 3. Later evidence changes the interpretation of an earlier vibration signal. Preserving the recording makes that evidence available for review. Illustrative research scenario.

Our next step is to bring capabilities such as forecasting, event detection, and explanation into a shared model. From there, we aim to develop generalist models that can transfer temporal knowledge across domains, sensors, and timescales. Further experiments will establish how well these capabilities transfer and which kinds of data help them develop.

Over longer histories, we envision models that can retain useful details, revisit earlier observations with new context, and revise their interpretations as new evidence emerges.

Ultimately, our goal is to scale human expertise. By combining temporal reasoning with domain-specific context, we aim to build agents that help engineers, clinicians, and researchers investigate complex systems, ask questions about signals, and evaluate answers against the underlying data. These models are intended to support human judgment and decision-making, not replace them. We believe that progress in understanding temporal signals can also advance continuity in AI more broadly.

Sign in to save this post