Time Series and Layers

Time Series play a central role in the Myst Platform. Upon creation, a time series is an empty canvas. You can add one or more Layers to a time series, which contain other Time Series, Sources, Operations, or Models. This allows you to stitch together output data from one or more Nodes, which is stored in the time series database when a Time Series is run. See the Query Time Series Data section to learn how to query data from the time series database.

🚧

Scalar Data

In the Alpha release of the Myst Platform only Time Series with scalar data are allowed. In one of our future releases we plan to add support for N-dimensional data.

You may see Time Series attributes axis_labels, cell_shape, and coordinate_labels in the Client Library. They are part of work towards support for N-dimensional data and are not functional today.

When a Time Series is "run" for a specific range of Natural Times and an As Of Time, it will go through its Layers in order and attempt to retrieve data from its upstream Nodes for the time ranges covered by each Layer. When data for the entire range of natural times is retrieved, the process finishes. Note that depending on the requested time range, only a subset of the Layers may need to be used.

Layers can be configured with:

• Label Indexer: For Layers retrieving data from Nodes that have non-scalar data (e.g. Sources), you must select a label indexer to define which label to retrieve from the upstream Node. The label indexer is defined as follows for each source connector:
  • Yes Energy: Label indexer equals the relevant datatype and object id concatenated with an underscore, along with an optional vintage, concatenated with an at sign. For example, LOAD_FORECAST_10000712973 or LOAD_FORECAST_10000712973@01:09:00.
  • TWC: Label indexer equals the relevant field returned by the source connector, for example, surfaceTemperatureCelsius or cloudCoveragePercent.
  • Time Trends: Label indexer equals the relevant field returned by the source connector, for example, DAY_OF_WEEK or HOUR_OF_DAY.
  • Solar Position: Label indexer equals the relevant field returned by the source connector, for example, elevation or zenith.
• Start Timing: The start timing determines the start time of the range of Natural Times that the Layer covers. If omitted, the Layer will cover the entire history. Note that start times are always inclusive.
• End Timing: The end timing determines the end time of the range of Natural Times that the Layer covers. If omitted, the Layer will cover the entire future. Note that end times are always exclusive.

You can stitch together data from multiple Sources to create one continuous Time Series. For example, you can connect data from three The Weather Company APIs (two of which contain historical data and one contains forecasts) to form a continuous Time Series of temperature.

You can also combine multiple, overlapping Layers in one Time Series. This allows you to automatically fall back to data from a lower priority Layer when a higher priority Layer is missing data – i.e., contains nulls or "masked" nulls. The platform automatically "masks" null values from third party data sources when those values are unavailable but we expect them to be available in the future. That masking indicates to the platform that it should try to fetch those time series again.

🚧

Time series do not fall through on inserted nulls

If you insert data into a time series that contains nulls, those nulls will persist – and the time series will not fall back to other layers of data. We created this exception to make it easy to nullify data that you want your model to ignore when fitting and predicting (e.g., Uri, the winter storm in Texas in 2021).

Once you've added layers, reorder them by hovering over a particular layer and dragging the layer to a new location. Layers ordered higher in the node side-panel will take higher precedence.