About this site 한국어

Unofficial explanatory translation of the Korean AI-Ready Data (AIRD) draft standard. The Korean text prevails.

Preparing data › Procedure

Measuring quality (stage 2)

Type · ProcedureReading time · about 4 minData providers

ContentsContents
  1. What happens in this stage
  2. Seven dimensions
  3. Scores versus tiers
  4. Diagnostic Maturity
  5. Order for reading the diagnostic report
Stage 2 workflow

In stage 2, quality measurement, the data provider prepares the material needed for measurement, and the diagnostic tool measures the indicators and produces the diagnostic report.

GoalHave the material needed for measurement ready, and check the scores and Diagnostic Maturity (DM) in the diagnostic report produced by the diagnostic tool.

What happens in this stage

The diagnostic tool computes indicator scores and Diagnostic Maturity even without an operating guideline. How many indicators are measured depends on the data type and the material the data provider has. For the tabular (STRUCT) example file, the file alone allows 6 of the 13 required indicators to be measured. With the stage 1 manifest filled in, 7 of the 13 are measured. The remaining indicators need external material (for example, code tables or a reference time). Results by step are in Walkthrough with example data.

The quality tier is judged with the threshold profile of an operating guideline.

The goal of stage 2 is to measure the quality of the data and leave the evidence for it.

Data provider Diagnostic tool Guideline publisher
Prepares measurement inputs (code tables · reference time · value ranges) · checks the diagnostic report · fixes values that can be fixed · records limitations that cannot be fixed Measures indicators · calculates scores · judges Diagnostic Maturity · produces the diagnostic report Issues and publishes the operating guideline (including the threshold profile)

The release version of the diagnostic tool is in development. The release plan is in Development status and upcoming releases.

The data provider does not apply the measurement formulas to calculate scores. The data provider prepares the material needed for measurement. The data provider checks the diagnostic report for low-scoring indicators and reasons for not measuring. The data provider records fixable values and unfixable limitations separately.

In stage 2, the data provider fills in missing values or corrects wrong values. Data values can therefore change in stage 2.

Work Output
Measuring quality in the seven dimensions Diagnostic report
Fixing the values of low-scoring indicators Cleaned version
Recording the fixes Processing history
Recording limitations that cannot be fixed Limitation record
Recording measurement results and limitations in the manifest Manifest understanding layer

Seven dimensions Required

The standard measures quality in seven dimensions: Completeness (D1) · Consistency (D2) · Accuracy (D3) · Timeliness (D4) · Validity (D5) · Uniqueness (D6) · Machine readability (D7). [Part 2 5.2]

Seven dimensions of quality

Check machine readability (D7) first. If the file cannot be opened, the other six dimensions cannot be measured. [Part 2 7.3]

The definitions, measurement formulas and preconditions of the 16 indicators are in Quality indicators. How to fix low scores is in Fixing defects (stage 2).

Scores versus tiers

Scores and the quality tier are different values. The diagnostic tool calculates a score for each indicator and records a score for each of the seven dimensions in the diagnostic report.

The quality tier is judged with the threshold profile of an operating guideline. The same scores can yield different tiers under different operating guidelines. A diagnostic report with no operating guideline applied records the quality tier as “not judged: no operating guideline applied.”

Tier judgment starts from the minimum dimension score (the lowest of the seven dimension scores). If one dimension scores low, that dimension determines the candidate tier even when the other dimensions score high. Averages are not used for tier judgment.

The example threshold profile in Part 2 Appendix I has the status EXAMPLE. It is used only for provisional calculations and tool development, not for tier judgment.

Results measured without an operating guideline are still valid baseline data. The data provider keeps the scores and the reasons for not measuring. When an operating guideline is applied, indicators whose measurement method changes with the guideline’s values (for example, the dummy-value patterns of D5-03) are measured again. Measuring again means issuing a new diagnostic report.

The operating guideline provided with the standard will be published when the standard is established (expected December 2026). Organizations and institutions can build their own operating guideline and judge tiers. How to build an operating guideline is in Building an operating guideline, and the steps of tier judgment are in Judgment procedure. Tier judgment is not a procedure the data provider performs.

Diagnostic Maturity Required

The diagnostic report records Diagnostic Maturity separately from the scores. Diagnostic Maturity is how far the required indicators have been measured. The diagnostic tool applies the decision table from the top and judges Diagnostic Maturity by the first row that is met. [Part 2 Annex B.5 Table B.4]

Apply from the top — stop at the first row that is met1Diagnosis status is SCHEMA_ONLY or FILE_BROKENNot judgedYesNo2All applicable required formal indicators APPLIEDDM-2YesNo3All applicable MMIs APPLIED, coverage ≥ lower boundDM-1YesNo4All applicable MMIs APPLIEDDM-0YesNo5All other casesDM-0 not metWithout an operating guideline, D5-03 is measured provisionally with example patterns — ends at row 4 (DM-0)
Diagnostic Maturity decision table. The conditions of each row and the lower bound for DM-1 are in Judgment procedure.

The Diagnostic Maturity reached without an operating guideline differs by data type. Tabular (STRUCT) and time series (TSERIES) data reach DM-0 (provisional). The diagnostic tool provisionally measures D5-03 (Statistical plausibility), a minimum measurable indicator (MMI), with the example patterns in Part 2 Appendix I and records the evaluation status as PROVISIONAL. Text (TEXT) data reaches DM-2 when all five required formal indicators are measured. Instruction–response pairs (PT) and images (IMAGE) are at DM-0. Measuring D3-02 (Label accuracy) for these two types requires expert review results. DM-1 is not judged. The lower bound of evaluation coverage for DM-1 is set by the operating guideline. [Part 2 5.1 · Annex A.4.1 · Appendix I]

The table by type is in Reading the diagnostic report (stage 2), and the row-by-row conditions of the decision table are in Judgment procedure.

If a file is empty (0 records) or damaged and cannot be opened, the diagnostic tool records Diagnostic Maturity and the tier as “not judged.”

The five indicator applicability values and reasons for not measuring

The diagnostic report records an applicability value for each indicator. What the data provider does depends on this value. [Part 2 Annex B.2]

Applicability valueMeaningData provider’s action
APPLIEDThe indicator was measured. A score is recordedNone
PARTIALLY_APPLIEDOnly targets with reference material were measured. No score is recorded (null)Obtain reference material for the remaining targets
PRECONDITION_UNMETNot measured because reference material needed for measurement (for example, a code table) is missingObtain the reference material and measure again
NOT_APPLICABLEThe indicator does not apply to that data type or conditionNone
OPTIONAL_SKIPPEDAn optional indicator was not measuredNone

NOT_APPLICABLE is not a deficiency. There is nothing to fix.

If even one indicator is PARTIALLY_APPLIED, Diagnostic Maturity cannot be DM-2. If Diagnostic Maturity is not DM-2, the quality tier is “not judged: below DM-2” however high the scores are. Scores and Diagnostic Maturity do not substitute for each other. [Part 2 8.1]

The full list of status values and reasons for “not judged” is in Status and judgment values.

Order for reading the diagnostic report

Order for reading the diagnostic report

The six values in the diagnostic report and how to read each one are in Reading the diagnostic report (stage 2).

Completion criteria

  • A list of the material needed for measurement is ready (whichever of the data file, stage 1 manifest, code tables, reference time and value ranges apply to your data)
  • If you have received a diagnostic report, Diagnostic Maturity and the reasons for not measuring are checked

What you need Producing a diagnostic report requires the diagnostic tool. The release version of the diagnostic tool is in development (Development status and upcoming releases).

Reading the diagnostic report (stage 2)

Last updated · 2026-10-07Report an error