Changelog
Back to All

LightGBM and a new metrics view ✨

almost 3 years ago by Ellery Berk

Myst Platform Release (2022-02-21)

Hello Myst Platform users,

We have an action-packed release for you this week! Highlights include a LightGBM model connector, backtesting metrics in the client library, and a metrics display in the web UI. Please update your myst-alpha Python package at your earliest convenience to access these updates (instructions here).

✨ New features:

  • We released a LightGBM model connector! We recommend using LightGBM if you would like to forecast prediction intervals. You can do so by creating one forecast for each bound by selecting quantile as your objective. Additional documentation is available here.
    Note: We do not yet support metrics or visualizations for prediction intervals. If you work with prediction intervals, we would love to hear what metrics and visualizations you’d like generated on the Myst Platform!
  • MAPE, MAE, and RMSE for your backtests are now available in the client library. You can access these metrics by calling backtest_result.metrics.
  • You can now view an auto-generated chart comparing two time series in the web UI for a time period of your choosing. This view also shows MAE, MAPE, or MSE for the as-of offset you select. We hope this quick metrics view is helpful to you! To access this view, click a time series node in your graph, click on the three vertical dots in the top right of the side panel, then click 'View metrics.'
3200

Once you have successfully navigated the above, you should see this view:

3200

⚡️ Enhancements:

  • The Numerical Expressions and Conditional Expression Mask connectors now support mod operation and float-returning functions. Before you could add together two TimeSeries (e.g. a + b). Now you can also call functions on each element of a TimeSeries (e.g. sin(a) + cos(b)). We prioritized this change to enable useful features like periodic representations of time (e.g. sin(day_of_year * 2 * 3.1415 / 365) or cos(day_of_week * 2 * 3.1415 / 7)).
  • ModelRunResults, which previously only returned the outputs of a model run, now also return the inputs of a model run. This means that you can recreate the data used to generate a prediction. This also useful for debugging any missing feature data that may have lead to nans in your predictions (e.g., the XGBoost connector has a predict_on_null_values flag that ensures a model will not make predictions for time stamps with nan feature values) or diagnosing plausible reasons for high prediction error (e.g., weather features used had high error for that hour or set of hours).

✅ Bug fix:

  • We fixed an edge case that led backtests to fail when a fold contained no predictions – for example, in the case of a zero predict offset.

🔧 Known issue:

  • The linear regression connector has a bug that is causing it to fail. We have deactivated this connector while we are implementing a fix.

As always, please do not hesitate to reach out via the chat embedded in the Myst Platform or email us at [email protected] with questions or feedback.

Thank you!
Ellery and the Myst team