Check what was not flagged too.
AI can raise a false alarm or miss a real defect. Either way, the comparison must concern the same object and reference state. Any subsequent repair or change needs to be documented before concluding that AI made an error.
Illustrative example: a report mentions no defect in an area it claims to cover. The expert finds damage there and documents its presence at the relevant time. This establishes an omission. If the original state cannot be reconstructed, the point remains unresolved.
Make the coverage of the check explicit.
The report identifies the areas, objects or points examined, the access available and the limits encountered. It separates verified facts from checks that could not be completed. A one-off audit does not establish that the entire AI product will work without errors in every situation.
Completeness and error severity are among the Hu-score’s axes. A determinant omission remains explicit in the conclusion; the number of correct answers does not erase it.
Source and scope
NIST AI RMF 1.0 discusses testing and evaluation in the context of use, including documented limitations. Huxpert uses this as a reference for a scoped comparison; NIST does not prescribe our grading scale. NIST AI RMF 1.0.
Related reading
Have an AI-generated result checked
Identify the output, actual object and checks required before requesting a quote.