Cleaners

A Cleaner is a set of rules that runs over rows of data: the rows a flow has extracted, or a spreadsheet you upload. It standardises formats, converts currencies and units, translates text, computes new columns, looks values up against your own data, flags outliers, collects values from a reviewer, and can trigger actions when a row breaks a rule.

Extraction answers “what does this document say?”. A Cleaner answers “what shape does that have to be in before it can go into our systems?”: one date format, one currency, part numbers matched to your catalogue, a total that adds up.

One execution of a Cleaner over a batch of rows is called a sweep.

Click Create Cleaner on the Cleaners page. Tavnit first asks “Where does the data come from?” and offers four starting points.

For From a flow and From a file, the next step also offers Suggest cleaning rules with AI. Leave it off and the Cleaner is created with just the base fields. Turn it on and, optionally, describe what you want to clean or compute (for example “Normalize supplier names and format the RUC without dashes”), then click Suggest rules.

Start from scratch opens the builder, where you name the Cleaner and work through the same sections the detail page has:

1. 1Base Fields. Choose a Data Source: Manual to define columns from scratch, From Flow to import a flow's output fields, or From File to parse a CSV or Excel header row. At least one field is needed.
2. 2Computed fields. Add any of the field types in the reference below. Optional: a Cleaner with only base fields is a valid pass-through.
3. 3Outputs. An email address and a webhook URL for sweeps the Cleaner runs on its own. Optional.
4. 4Skip Rows. Drop the first N rows or empty rows of uploads. Optional.

To have every run of a flow swept automatically, link the Cleaner to it: tick Link this cleaner to the flow when you build from a flow, or choose the Cleaner in the flow's Cleaner setting. A flow can have only one Cleaner; one Cleaner can serve several flows, listed under Linked Flows on its detail page. Creating and editing Cleaners requires the Admin or Owner role.

Every column in a Cleaner has a type that decides what it does. The detail page groups them in its left rail under Input and Computed, with a count for each. This is the complete set, in the order the app lists them.

Any field can be marked Exclude from output: it stays available to computed columns but does not appear in what the Cleaner delivers. That is how you use an intermediate value (a raw amount, a lookup key) without shipping it downstream.

The formatting and conversion types each have a small set of settings. These are the ones that change the result.

A Conditional Actions field is a rule: when a row matches these conditions, do these things. It is the only field type that changes what happens to the data rather than what a cell contains, and it is how validation failures become something other than a number in a table.

Conditions are grouped, and groups combine with AND or OR, so you can express things like (total > 10,000 OR currency is not USD) AND supplier is not on the approved list. Conditions can read computed columns too, including Summary and Anomaly Check fields. A rule with no conditions at all fires on every row.

Worked example. On an invoice Cleaner, add a Conditional Actions field with two rules. The first matches rows where the line total does not equal quantity × unit price and sends the run to Human in the Loop. The second matches any invoice over your approval threshold and emails the finance lead. Everything else flows straight through to the Bucket without a human touching it.

A Calculated field evaluates an arithmetic expression against the row it is on. Reference other columns by name in braces, and start the formula with an equals sign if you like: both ={Quantity} * {Unit Price} and {Quantity} * {Unit Price} work.

Summary fields aggregate across every row in the sweep instead of computing per row, and write the result on every row. They answer “what is the total of this document?” without you having to sum the rows downstream.

• Only include rows matching: optional condition groups, so you can sum just the rows of one category or status.
• Format result: optional decimals and separators, as on Calculated fields. The value stays numeric.
• Multi-value cells: when a column holds several values per row (a multi-value flow field, such as every deduction line on a payslip), each value counts. Sum adds them all, Count counts each one and Average divides by the number of values.
• Summary results can be used in formulas and in Conditional Actions conditions.

A Human Input field is a column nobody extracts: a reviewer types or picks its value on the review screen. Use it for information that is not in the document, such as the warehouse a delivery goes to or an approval code.

• Input type: Free text, or Menu. A menu takes its options from the distinct values of a Bucket column, optionally filtered by conditions on other Bucket columns or on the current row's values.
• Value is mandatory: when on, approval is blocked until every row has a value. Turn it off to allow blanks.
• Reviewers: pick at least one. They are added to the reviewers of the run.

An Anomaly Check flags rows whose value in one column stands out from the rest of the sweep: a quantity far from the others, or a code from a category almost nothing else uses. It returns true when the value stands out and false otherwise. It is deterministic and uses no AI.

An Anomaly Check only produces a flag. To act on it, add a Conditional Actions rule such as this field equals true → Human in the Loop. The review screen then shows why each value was flagged.

A Date Calculation does date arithmetic on every row, deterministically and without AI. It has two modes.

• Units for a difference: Days, Weeks (whole weeks), Months (complete, like an age), Calendar months (month boundaries crossed, ignoring the day) and Years (complete).
• Today is your organization's date when the data is processed.
• A blank date gives a blank cell. Pair the result with a Conditional field (for example days_to_expiry >= 30 → yes, else → no) to flag rows.

The HS Code field assigns a customs tariff code to each row from its product description. It works down the Panama tariff schedule (HS 2022): section, then chapter, then heading, then national tariff line, rather than pattern-matching a description against a code list.

• Benchmark Fields: the columns the classifier reads, usually the product description, sometimes with material or use alongside it. Leave it unset and the whole row is used.
• Instructions for AI: the cases your catalogue gets wrong, such as how to treat kits, spare parts, or goods that could sit in two chapters.
• Section preselect (recommended): starts the cascade at the 22 HS sections before chapters.
• Output Format: the code only (for example 0101.29.00.00.00) or the code plus its description.
• Rows are classified independently, so one hard row does not affect the others.

After the computed columns, a Cleaner can drop junk rows, choose and order the columns it ships, reshape long data into wide, and send the result to an email address or a webhook.

Pivot is the delivery format, not just a view. Choose the Column to explode, the column the Values from, how to Group rows by (all other columns automatically, or columns you choose), and what to do when a label repeats within a group: sum the values, keep the first value, or keep the first value and flag a warning. Optional Summary columns (Sum, Average, Minimum, Maximum or Count) are computed per row across the exploded columns and appear at the far right. Under Apply to, choose the webhook payload, the email output and downloads; when downloads are on, the sweep page opens on the pivoted view.

A sweep is one execution of a Cleaner over a batch of rows.

Open any sweep from the Cleaner's Sweep History to see exactly what it did. The page updates live while the sweep is processing.

• Status (Processing, Completed, Failed or Cancelled) with the Rows and Source of the sweep.
• Warnings: per-row problems such as a formula that divided by zero or a value that could not be read as a number. Problems stay attached to the row that caused them, so one bad row does not fail the sweep.
• Sweep Information: the Cleaner, when the sweep was created, and the source run and flow when it came from a flow.
• The data table, with a Cleaned / Pivoted / Raw toggle to compare the output with what came in, and Expand for a full-screen view. You can select and copy cells.
• CSV and JSON export of the current view. With a pivot applied to downloads, the export has the pivoted shape.
• Outputs: a log of each delivery (email, webhook, notifications) and whether it succeeded.
• Bucket Changes: every Bucket cell the sweep's Edit Bucket Row actions changed.

A Cleaner can run on its own from your systems, with no flow involved. Copy its ID from the Cleaner ID panel and authenticate with your API key.

Both return HTTP 202 with a sweep_id. When the sweep completes, the Cleaner's own Email Output and Webhook deliver the cleaned rows. See the API page for authentication and the response shape.

Cleaning happens after extraction and before delivery. That order matters: everything downstream (the review screen, the webhook payload, the Bucket rows, an agent's input variables) sees the cleaned output, not the raw extraction.

1. 1The flow extracts rows from the document.
2. 2The linked Cleaner sweeps those rows: formats, conversions, lookups, computed columns.
3. 3Conditional Actions fire: rows are dropped, notifications go out, review may be requested. A Human Input field always requests review.
4. 4If review was triggered, the run pauses and a reviewer sees the cleaned table.
5. 5On completion, results are delivered through the flow's outputs: email, webhook, Bucket, form fill, or a chained agent.

The Pipeline Map shows this for your own workspace, including which flows share a Cleaner.