AI evaluation services glossary
18 terms buyers meet when evaluating AI evaluation companies.
- Capability evaluation
- A structured test of what a model can do on defined tasks, scored against agreed criteria.
- Safety evaluation
- A test of whether a model produces harmful, deceptive, or policy-breaking behavior.
- Red teaming
- Deliberate adversarial testing by people or automated attackers to find unsafe, insecure, or unwanted model behavior.
- Pre-deployment evaluation
- An evaluation of a model before public release, often by a third party, used to inform launch decisions.
- Benchmark
- A fixed, reusable evaluation set used to compare models over time.
- Private (held-out) test set
- Evaluation items kept out of public view so models cannot be trained on them.
- Contamination
- When test items or close copies appear in a model’s training data, which inflates scores.
- Rubric
- Written grading criteria that tell evaluators how to score an output.
- Domain-expert grading
- Scoring of model outputs by people with verified professional or academic expertise in the subject.
- Side-by-side (pairwise) evaluation
- Graders compare two model outputs for the same input and pick the better one.
- LLM-as-a-judge
- Using a model to grade another model’s outputs, usually calibrated against human grades.
- Prompt injection
- An attack that hides instructions in content a model reads, such as a web page or document, to change its behavior.
- Jailbreak
- An input crafted to get a model to ignore its safety rules.
- Scheming
- A model covertly pursuing goals that differ from its developer’s intent, for example by deceiving evaluators.
- System card
- A document published with a model release that describes its evaluations, risks, and safeguards.
- SOC 2 Type II
- An independent auditor’s report on how a company’s security controls operated over a period.
- ISO/IEC 27001
- An international certification for an information security management system.
- FedRAMP
- The US government program that authorizes cloud services for federal use, at Low, Moderate, or High impact levels.