---
title: "9. Stability and forecast validation"
output:
  rmarkdown::html_vignette:
    toc: true
vignette: >
  %\VignetteIndexEntry{9. Stability and forecast validation}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE, comment = "#>", message = FALSE, warning = FALSE,
  fig.width = 7, fig.height = 5.5
)
```

```{r libraries}
library(idiographic)
data(srl)
vars <- c("efficacy", "value", "planning", "monitoring", "effort")
has_cograph <- requireNamespace("cograph", quietly = TRUE)
```

Two complementary checks on a person's model: how *stable* the edge estimates
are under resampling, and how well the model *predicts* unseen occasions. Both
operate on a single series, so we select one student up front with `subset()`.

> **Experimental.** Both tools are methodologically grounded — block
> resampling for dependent data and rolling-origin forecast evaluation — but
> unlike the estimators in this package they have no external reference
> implementation to validate against. Their interfaces, defaults, and
> reported statistics may change in a future release.

```{r grace}
grace <- subset(srl, name == "Grace")
```

# Edge stability

`estimate_stability()` resamples consecutive blocks of the series and
re-estimates, quantifying how reproducible each edge is.

```{r stability}
stab <- estimate_stability(grace, vars = vars, estimator = "var",
                           resample = "block", n_resamples = 50, seed = 1)

stab
```

The per-edge stability table is tidy — inspect the most-resampled edges with
`head()`:

```{r stability-table}
head(as.data.frame(stab))
```

The original (point) estimate is stored and plots directly:

```{r plot-stability, eval=has_cograph}
plot(stab, layer = "temporal")
```

# Forecast validation

`validate_forecast()` checks one-step-ahead predictive performance with a
rolling origin. Reusing the same single-person series, the validator splits it
into consecutive blocks of `block_size` occasions.

```{r forecast}
fc <- validate_forecast(grace, vars = vars, estimator = "var",
                        block_size = 10, initial = 8, n_splits = 5,
                        scale = TRUE)

fc
```

Per-variable and overall error metrics come back as a tidy table:

```{r forecast-metrics}
fc$metrics
```

The full one-step prediction record is available with `as.data.frame()`:

```{r forecast-predictions}
head(as.data.frame(fc))
```