Scoring Run Event

POST

Sent when a batch of output scoring settles, however it was triggered.

Authorizations

• bearerAuth
• oauth2

Request Body^required

Media typeapplication/json

ScoringRunWebhookEvent

Delivered when a scoring run settles.

Also raised for the automatic scoring that follows a generation, in which case data carries the inference_id of the generation whose outputs were judged.

object

id

required

Id

Unique event id, repeated in the webhook-id header.

string

type

required

WebhookEventType

Scoring_run.completed or scoring_run.failed.

string

Allowed values: inference.completed inference.failed inference.cancelled training.succeeded training.failed scoring_run.completed scoring_run.failed workflow_run.succeeded workflow_run.failed workflow_run.cancelled

created_at

required

Created At

When the event was raised (RFC 3339).

string

data

required

GetScoringRunOutput

Identical to the body of GET on the scoring run.

object

scoring_run_id

required

Scoring Run Id

Unique identifier for this scoring run.

string format: uuid

status

required

ScoringRunStatus

Current status: pending, complete, or failed.

string

Allowed values: pending complete failed

inference_id

Inference Id

The generation whose outputs this run judged, when it was started automatically.

string format: uuid

nullable

poll_interval_seconds

Poll Interval Seconds

Suggested polling interval in seconds. Present while pending.

integer

nullable

requested_pair_count

required

Requested Pair Count

How many (file, rule) pairs the run set out to judge.

integer

completed_pair_count

required

Completed Pair Count

How many produced a verdict. Lower than requested means some pairs went unjudged.

integer

file_ids

File Ids

The files the run was asked to judge.

Array<string>

default:

rule_ids

Rule Ids

The rules the run judged against.

Array<string>

default:

scores

Scores

The verdicts recorded. Present once the run is complete.

Array<object>

nullable

ScoringRunFileScore

object

file_id

required

File Id

The file this verdict is about.

string format: uuid

rule_id

required

Rule Id

The scoring rule that produced it.

string format: uuid

rule_name

required

Rule Name

The rule’s name as it stood when the verdict was recorded.

string

score

required

Score

Score in [0, 100] against that rule.

integer

rationale

Rationale

The judge’s reasoning, in its own words.

string

""

finished_at

Finished At

When the run settled. Present once terminal.

string format: date-time

nullable

error

Error

Error message. Present only on failed.

string

nullable

error_code

Error Code

Machine-readable error code. Present only on failed.

string

nullable

webhook

Any of:

WebhookStatusView

WebhookStatusView

What a job’s GET reports about the callback attached to it.

The URL is echoed so a caller can tell which of its endpoints a run was aimed at; the rest is how the delivery went, which is the only place a failed delivery is visible.

object

url

required

Url

The callback URL this run was submitted with.

string

status

required

WebhookDeliveryStatus

Pending until a delivery settles, then delivered or failed.

string

Allowed values: pending delivered failed

attempts

required

Attempts

How many delivery attempts have been made.

integer

last_attempt_at

Last Attempt At

When the last attempt was made.

string format: date-time

nullable

last_response_status

Last Response Status

HTTP status the receiver returned on the last attempt, if it responded.

integer

nullable

last_error

Last Error

Why the last attempt did not succeed. Null once delivered.

string

nullable

null

null

metadata

Metadata

Whatever you sent as webhook.metadata.

object

key

additional properties

string

Responses

200

Successful Response

Media typeapplication/json

Example^generated

example

422

Validation Error

Media typeapplication/json

HTTPValidationError

object

detail

Detail

Array<object>

ValidationError

object

loc

required

Location

Array

msg

required

Message

string

type

required

Error Type

string

input

Input

ctx

Context

object

Example^generated

{
  "detail": [
    {
      "loc": [
        "example"
      ],
      "msg": "example",
      "type": "example",
      "input": "example",
      "ctx": {}
    }
  ]
}