Document classes and classification

A document class is a type of document you expect in a batch, such as an invoice, a credit note or a delivery note. Each class has its own classification rules, fields, finders and, if you like, its own export. Document classification is how DocView decides which class each document belongs to.

You set up document classes on the Document classes & fields tab of a Batch Setup. In other capture products the same idea is called a document type, form type or Document Definition.

One-touch scanning: the panel at the top of the Document classes & fields tab turns document classes into buttons on the Scan screen, so operators sort documents by type as they feed them. It needs the Manual (operator splits) separation rule switched on. See One-touch scanning.

Add a document class

1. Open the Batch Setup and go to the Document classes & fields tab. Create a draft first if the version is released.
2. Under New document class, type a name and click Add.
3. Click the new class in the list on the left to open it.
4. Check and change its settings (see below), then click Save class.

Drag classes by the grip to change their order.

To delete a class and all its fields, open it and click Delete class.

Class settings

Note: When a class uses another OCR engine, its documents are read again with that engine after classification. See OCR engines and languages.

Classification rules

Classification rules look at the OCR text, barcodes, page size and layout of a document. Each matching rule adds its weight to the class's score.

Add a rule

1. Open the class.
2. Under Classification rules, choose the Type.
3. Type the Pattern.
4. Set the Weight. Higher weights count more; the default is 1.
5. Click Add rule.

To remove a rule, click the × at the end of its row.

Rule types

Page size rules

A pageSize pattern gives the width and the height in millimetres, separated by x. Each side can be:

• <170 (less than 170 mm),
• >180 (more than 180 mm),
• 200-220 (from 200 to 220 mm), or
• 210 (210 mm, give or take 10 mm).

A page turned sideways matches too. For example, <240x<130 matches cheques and 200-220x287-307 matches A4. Page size rules work well together with keyword rules: give the page size a lower weight, so the text still decides between forms of the same size.

Layout rules

layout rules recognise a form by how it looks, that is where the printing sits on the page, not by its text. Use them when a form has no reliable keyword or barcode, or to tell apart the language versions of the same form, for example the English and Welsh versions of one application form.

In the rules table, a layout rule shows a short description in plain words, for example "Looks like the learnt form on 2 page(s) (similarity from 0.5)", or "12 printed boxes found on page 1" for a rule that came from an import.

Learn a layout rule from samples

1. Upload a few sample documents of each document class first, in the zone designer or on the Finders tab. The more classes have samples, the better DocView can tell you whether the rule works.
2. Open the class and, under the classification rules, open Learn the layout from samples (recognise the form by its look).
3. Choose the Pages to compare: First page, or the First 2, 3 or 4 pages.
4. Set Counts from similarity, from 0.1 to 0.95 (default 0.5). Documents that look less alike than this score nothing for the rule.
5. Set the Weight (default 10).
6. Click Learn layout from samples.

DocView uses up to the 20 most recent samples of the class and combines the look of their pages into one rule. The message then tells you how well the rule works, for example:

Layout rule learnt from 2 sample(s), 2 page(s). Its own samples look 99% alike or more; the most similar other type (Application form English) 84%.

If another class looks about as similar as the class's own samples (within 2 points), you see a warning instead: the rule can't reliably tell them apart. Add a keyword, barcode or page-size rule as well, or upload more samples and learn again.

You need a draft version to learn a layout rule, because released versions can't be changed. See Versions and publishing.

Layout rules from imports

When you import a setup from another product, its form recognition can become layout rules too. A layout rule holds one of two things:

• Printed tick boxes of a page, for example { "page": 2, "boxes": [ … ], "min": 0.6, "search": 10 }. The score is the share of the boxes found on that page, within search mm. min is the lowest share that counts.
• Page fingerprints learnt from sample documents, for example { "fingerprints": [ { "page": 1, "columns": 24, "rows": 34, "cells": [ … ] } ], "min": 0.5 }. A fingerprint is the look of a page as a coarse grid. The score is the similarity: below min gives 0, identical pages give 1.

You can delete a layout rule like any other rule, with the × at the end of its row, but you cannot type one in. See Forms, tick boxes and signatures.

How the score is worked out

For each class, DocView adds up the weights of the rules that match. The best class's confidence then takes two things into account:

• how clearly it beats the runner-up, and
• how much of its own rule weight matched.

So a single weak keyword never gives full confidence, and two classes that match equally land at 0.5 or below. If the score of the best class is below its Threshold, the document is marked for review.

Fallback class and single-class setups

A class without rules acts as a fallback. It only wins when no other class matches. If a Batch Setup has a single class and no rules, every document is simply assigned to it. This is how the General documents template works.

Example: invoices and credit notes

The Supplier invoices template sets up two classes like this:

Invoice (threshold 0.5)

Credit note (threshold 0.5)

Credit notes often contain the word "invoice" too, so "credit note" gets a high weight to win clearly.

Tip: Give each class at least one rule that only its own documents match, with a high weight. Use lower weights for words that appear on several document types.

Classification rule list: the first rule that matches decides

When you want full control of the order, choose Rule list under How the document class is decided (on the Classification tab of the workflow's OCR, separation & classification step). You build an ordered list of rules. DocView tries them from the top, and the first rule that matches gives the document class. When no rule matches, the document goes to Classification for an operator.

For the last two, At least is the score needed, from 0 to 1. Leave it empty to use each class's confidence threshold. A rule can be switched off without removing it, and you can give it a name. The name shows in the explanation of the result (for example "rule 2 (Invoice barcode): barcode INV-1043").

A script sees text (all the OCR text), pages, lines, words, barcodes (each with value and type), pageCount and documentTypes (each with key and name). It can also use the find(pattern) helper that script finders use. It returns the class key or name, or nothing to try the next rule:

if (barcodes.some(b => b.value.startsWith('CN-'))) return 'credit-note';
if (/credit\s+note/i.test(text) && pageCount <= 2) return 'credit-note';
// nothing returned: the next rule is tried

A script that fails is skipped, and the reason shows in the explanation. A typical list combines the methods: a barcode first, then a script for special cases, then The trained model at 0.9, then The document types' rules. The rule list is saved with Save workflow and travels with the Batch Setup when you export it.

What happens to uncertain documents

A document goes to manual classification when:

• no class matched at all, or
• the best class scored below its Threshold.

If your workflow has a Classification step, operators classify these documents there, before extraction. Operators can also change a document's class in Index / Validate, if the step allows Change document class.

Machine-learning classification

Rules work well when documents have clear titles or barcodes. When they do not, you can train a text classifier from sample documents: choose Samples & training at the top of this tab, drop a folder with a sub-folder of samples per document type, and train. Each type shows how many samples it has. Once the version uses a model, it classifies instead of (or together with) the rules. See Samples and classifier training.

Classification and separation

Classification can also split documents. The Classification change separation rule classifies each page and starts a new document where the class changes. See Image cleanup and separation.

Fields

Below the rules you find the class's Fields: the index values to extract, check and export. Click Add field to add one. See Fields and extraction.

Test your classes

Use the Test tab to run a sample file through separation and classification. For each document it shows the score of every class and Why, that is, which rules matched. See Testing a Batch Setup.