Cleaning Presets & Report

The default and strict presets clean data before inference: sorting, de-duplicating, aligning to the grid, filling gaps, handling missing-value codes and so on. Every change is written to the cleaning report as a "decision" with a stable id; nothing is changed silently. You can accept or decline decisions by id, and dry-run for free with /v1/clean.

1. Cleaning presets

Pick a preset with options.cleaning.preset:

Input requirements for raw are in the raw preset.

clean_series: false forbids repairs: if the data needs any, the request is blocked with TIMESTAMP_IRREGULAR or SERIES_DISCONTINUOUS, with the report explaining why.

2. The effective policy effective_policy

Every successful response echoes the policy that actually applied:

"effective_policy": {
"id": "default@2026-13",
"preset": "default",
"version": "2026-13",
"source": "platform"
}

Key and organization defaults are configured by the platform; there is no self-service setting yet. An explicit preset or version in the request always wins.

Policy versions

We recommend omitting version so your requests follow the platform default. The removal date will be announced in advance.

2026-11 keeps every 2026-10 rule and adds:

• Per-gap filling. Each gap gets its own method: PCHIP (shape-preserving, no overshoot) for short gaps, seasonal fills (same phase of nearby cycles) for longer gaps when a seasonal period is accepted, linear otherwise. Each gaps[] entry in the imputed action states its method, tier and reason; series[].seasonality states the period that was tested and whether it was accepted.
• Physical constraints on filled values. Filled values are clipped at 0 for non-negative series (constraint.nonnegative), clipped to a declared range (constraint.bounds), and kept at 0 in phases that are almost always zero, such as solar output at night (constraint.structural_zero). Observed values are never changed; observed values that break a constraint are flagged (negative_in_nonnegative, out_of_bounds). Each constraint is a declinable decision.
• Business days. Daily data with no weekend dates is put on a B grid: weekends are not gaps and forecast stamps skip them (decision grid.business_days, issue business_days_pattern). Decline the decision to keep calendar days.
• More data checks. y text that is not a plain number is reported (thousands_separator, decimal_comma, ambiguous_separator, number_format_mixed, non_numeric_value) and left missing rather than guessed, constant and all-zero series are warned (constant_series, all_zero_series), and every unparseable timestamp is listed in one error.

The seven 2026-11 parameters (impute_method, seasonal_period, nonnegative, bounds, structural_zero, business_days, long_gap), every step each version runs with its thresholds, and the meaning of every decision and issue code in the report are in the Cleaning Policy Reference.

3. The cleaning report cleaning_report/1

The cleaning report appears in three places:

• successful /v1/forecast responses (except under the raw preset);
• /v1/clean responses;
• error.details.cleaning_report when cleaning blocks the request.

Top-level structure:

Under the default 2026-13, health_tier follows the missing-data rule: an entity refused by it is Critical and blocks the request with SERIES_DISCONTINUOUS, an entity accepted with the warning missing_pattern is Warning, and health.factors lists the deductions. The deprecated versions grade the missing share (gaps plus nulls) instead: under 1% is Healthy, 1% to 10% is Warning, and 10% or more is Critical. Outliers are only flagged in issues[]; they are never modified.

4. Decisions decisions[]

Fields of each decision:

Rules at a glance:

value.tokens and value.dash_token only concern text cells (such as uploaded CSV). In a JSON request y is a number or null, so these two normally do not appear.

Before/after examples for each rule follow. All use hourly data for entity meter-001; the decisions are actual cleaning-service output under policy version 2026-12 and the default preset.

time.duplicates

// Input: 10:00 appears twice with different values
{"ds": "2026-09-01T10:00:00Z", "y": 145.7}
{"ds": "2026-09-01T10:00:00Z", "y": 999.0}
// Result: only the last row (999.0) is kept, with a duplicate_conflict warning;
// 999.0 is far from the other values, so an outlier_flagged_not_modified warning
// is added too (the value is not changed)
{"id": "time.duplicates:meter-001", "rule": "time.duplicates", "class": "drop",
"status": "applied", "action": "keep_last",
"before": {"duplicate_rows": 1, "conflicting": 1}, "after": {"duplicate_rows": 0}}

When both rows hold the same value, class is structural, conflicting is 0 and no warning is raised.

time.reorder

// Input: 11:00 comes before 10:00
{"ds": "2026-09-01T11:00:00Z", "y": 146.9}
{"ds": "2026-09-01T10:00:00Z", "y": 145.7}
// Result: stable sort by time
{"id": "time.reorder:meter-001", "rule": "time.reorder", "class": "structural",
"status": "applied", "action": "stable_sort",
"before": {"out_of_order": 1}, "after": {"out_of_order": 0}}

grid.snap

// Input: 10:00:20, 20 seconds after the slot
{"ds": "2026-09-01T10:00:20Z", "y": 145.7}
// Result: snapped to 10:00:00
{"id": "grid.snap:meter-001", "rule": "grid.snap", "class": "structural",
"status": "applied", "action": "nearest_slot",
"before": {"off_slot": 1}, "after": {"off_slot": 0}}

grid.off_grid

// Input: a 10:30 row inside hourly data
{"ds": "2026-09-01T10:00:00Z", "y": 145.7}
{"ds": "2026-09-01T10:30:00Z", "y": 141.0}
{"ds": "2026-09-01T11:00:00Z", "y": 146.9}
// Result: the 10:30 row is dropped, with an off_grid_dropped warning
{"id": "grid.off_grid:meter-001", "rule": "grid.off_grid", "class": "drop",
"status": "applied", "action": "drop",
"before": {"rows": 1}, "after": {"rows": 0}, "evidence": {"ratio": 0.020408}}

evidence.ratio is the share of off-grid rows; above 10% the request is blocked with FREQ_INFERENCE_FAILED.

gap.edge_trim

// Input: the first two rows are null
{"ds": "2026-09-01T00:00:00Z", "y": null}
{"ds": "2026-09-01T01:00:00Z", "y": null}
{"ds": "2026-09-01T02:00:00Z", "y": 146.9}
// Result: edge nulls are removed; the series starts at 02:00
{"id": "gap.edge_trim:meter-001", "rule": "gap.edge_trim", "class": "drop",
"status": "applied", "action": "drop",
"before": {"empty_edge_slots": 2}, "after": {"empty_edge_slots": 0}}

window.context

// Input: 24 points; the model uses at most 16
// Result: only the latest 16 points are kept
{"id": "window.context:meter-001", "rule": "window.context", "class": "drop",
"status": "applied", "action": "keep_latest",
"before": {"points": 24}, "after": {"points": 16}, "evidence": {"max_context": 16}}

The API never produces this decision: a series longer than the model's maximum context is rejected with 422 SERIES_TOO_LONG, and you choose which points to send. The model's maximum context is in Models & Limits.

gap.split

// Input: 128 observations, hours 40 to 51 missing, options.gap_split_min_steps = 8
// Result: the 40 points before the gap are dropped, the 88 after it kept,
// with a segment_split warning
{"id": "gap.split:meter-001", "rule": "gap.split", "class": "drop",
"status": "applied", "action": "keep_latest_segment",
"before": {"points": 128}, "after": {"points": 88}, "reversible": true,
"evidence": {"min_steps": 8, "threshold_source": "option",
            "dropped_observed": 40, "observed": 128}}

• A gap splits the series only when it is at least evidence.min_steps long. Without options.gap_split_min_steps, the threshold is automatic (threshold_source: "auto"): the largest of the fill limit plus one, three seasonal cycles, and 5% of the series span.
• It applies automatically only when at most half of the observations would be dropped; otherwise it is proposed, the gap is filled instead, and a segment_split_skipped warning is raised.
• Use accept to force the split or decline to undo it.

grid.regrid and gap.impute

Missing timestamps produce these two as a pair: insert empty slots, then interpolate.

// Input: 05:00 is followed by 07:00; 06:00 is missing
{"ds": "2026-09-01T05:00:00Z", "y": 138.8}
{"ds": "2026-09-01T07:00:00Z", "y": 138.2}
// Result: 06:00 is inserted and filled by PCHIP (a short gap, see the policy reference)
{"id": "grid.regrid:meter-001", "rule": "grid.regrid", "class": "structural",
"status": "applied", "action": "insert_empty_slots",
"before": {"slots": 11}, "after": {"slots": 12}}
{"id": "gap.impute:meter-001", "rule": "gap.impute", "class": "fabricate",
"status": "applied", "action": "pchip",
"before": {"missing": 1}, "after": {"filled": 1},
"evidence": {"methods": {"pchip": 1}, "gaps": 1, "seasonal_period": 24,
            "seasonality_accepted": false}}

Rows with y: null produce only gap.impute. The filled positions are listed in series[].actions[]. Gaps longer than the fill limit (3 steps) are still filled, with a long_gap_filled warning.

value.sentinel

// Input: one -9999 among 48 points; every other value is between 132 and 148
{"ds": "2026-09-02T06:00:00Z", "y": -9999}
// Result: -9999 is nulled, then filled by gap.impute
{"id": "value.sentinel:meter-001:-9999", "rule": "value.sentinel", "class": "fabricate",
"status": "applied", "action": "null",
"before": {"value": -9999.0, "count": 1}, "after": {"value": null, "count": 1},
"reversible": true,
"evidence": {"source": "builtin", "detached": true, "nearest_other": 132.0,
            "spread": 16.0, "share": 0.020833}}

• Built-in candidates are 9999, -9999, 99999, -99999, -999, 999999, 65535, -32768.
• It applies automatically only when the value sits far from the rest of the data (detached) and makes up at most 5% of points (share); otherwise it is proposed and the value is kept.
• The selector is the value itself, so each code in an entity gets its own id.
• If -9999 is a real value, keep it with decline; the report then raises an outlier_flagged_not_modified warning.

value.tokens and value.dash_token

// Input (text cells): two "N/A", one "-"
// value.tokens: markers are nulled, then filled
{"id": "value.tokens:meter-001", "rule": "value.tokens", "class": "semantic",
"status": "applied", "action": "null",
"before": {"cells": 2}, "after": {"missing": 2}}
// value.dash_token: reading the dash as 0 is only proposed by default
{"id": "value.dash_token:meter-001", "rule": "value.dash_token", "class": "semantic",
"status": "proposed", "action": "zero",
"before": {"cells": 1, "token": "-"}, "after": {"value": 0.0, "count": 1}}

A dash may mean 0 or missing. Until accepted it is treated as a missing value; add value.dash_token:meter-001 to accept to read it as 0.

5. Accepting and declining

List decision ids to accept or decline in the request:

"options": {
"cleaning": {
  "preset": "strict",
  "accept": ["gap.impute:meter-001"],
  "decline": ["value.sentinel:meter-001:-9999"]
}
}

• accept applies proposed decisions: value.sentinel, value.dash_token, gap.split, and any decision the strict preset demoted to a proposal.
• decline undoes reversible decisions (reversible: true): value.sentinel and gap.split, and from 2026-11 on also grid.business_days and constraint.*. Declining gap.masked_tail (2026-12) blocks the request with trailing_gap_too_long instead.
• Do not put the same id in both lists.
• An id that does not exist for this data raises a decision_not_applicable warning; the request proceeds.
• The raw preset cannot be combined with accept / decline.

A strict example

The data below misses one timestamp and contains a -9999. The default preset fills both; strict demotes gap.impute and value.sentinel to proposals and blocks the request:

// 422 SERIES_DISCONTINUOUS, in error.details.cleaning_report:
"decisions": [
{"id": "value.sentinel:meter-001:-9999", "status": "proposed", ...},
{"id": "grid.regrid:meter-001", "status": "applied", ...},
{"id": "gap.impute:meter-001", "status": "proposed", ...}
],
"blocking": [
{"code": "cleaning_required", "unique_id": "meter-001", "reason": "series_discontinuous"}
]

Accepting only gap.impute:meter-001 is still blocked, because value.sentinel remains a proposal. With both ids in accept the request passes and all three decisions are applied. The recommended strict workflow is to get the ids from /v1/clean, then accept them in the forecast request.