Objectives

Objectives consist of a human-readable Intent and ground truth examples directly. An objective serves both the purposes of

• Communication: Expressing the intended business purpose of the evaluator

• Coordination: Serving as a battery of measures

Intent Structure

Root Signals uses a standardized intent structure that bridges human-readable descriptions with machine-understandable syntax. This universal format ensures semantic consistency across all evaluators while maintaining clarity for human interpretation.

Standard Format

Property [of Object types [with respect to Reference objects]] [in Context] [for Goal] [with weights level1 for a₁, b₁, c₁ [; level2 for a₂, b₂, c₂]]

Components

• Property: The single quality being measured (e.g., Relevance, Safety, Coherence)

• Object types: The text artifacts being evaluated (response, content, answer, JSON)

• Reference objects: What the evaluation compares against (request, prompt, ground truth)

• Context: Specific situational constraints (child-audience, professional, RAG evaluation)

• Goal: The desired outcome (keeping responses on-topic, age-appropriate consumption)

• Weight levels: Criteria importance (high, avoiding, detecting)

Examples

Relevance Evaluator:

Relevance of response with respect to request for keeping responses on-topic and informative with weights high for accuracy, completeness, adherence to prompt, logical consistency

Safety for Children:

Safety of content in child-audience context for age-appropriate consumption with weights avoiding for explicit language, violent content, adult themes

Conciseness:

Conciseness of response for efficient communication with weights high for brevity, directness ; avoiding for redundancy

JSON Representation

The structured format also translates to machine-readable JSON:

{
  "property": "Relevance",
  "object": "response",
  "respect_to": "request",
  "goal": "keeping responses on-topic and informative",
  "weights": {
    "high": ["accuracy", "completeness", "adherence to prompt", "logical consistency"]
  }
}

This standardized approach ensures that every objective intent is both semantically precise and universally interpretable across different contexts and implementations.