Human Review

These endpoints allow you to programmatically analyze human review results.

GEThttps://api.autoblocks.ai/human-review/jobs

Get a list of human review jobs

This endpoint returns a list of all of your human review jobs. It is useful for getting the id of a job so that you can use other endpoints that require a job's id.

Code samples for getting a list of jobs

GET

https://api.autoblocks.ai/human-review/jobs

curl 'https://api.autoblocks.ai/human-review/jobs' \
  --header "Authorization: Bearer $AUTOBLOCKS_API_KEY"

{
  "jobs": [
    {
      "id": "cljzx8zhd00077w3sqredd0fz",
      "name": "My First View",
      "reviewer": {
        "id": "cljzx8zhd00077w3sqredd0fz",
        "email": "test@test.com"
      }
    }
  ]
}

GEThttps://api.autoblocks.ai/human-review/jobs/<jobId>/test-cases

Get a list of test cases for a job

This endpoint returns a list of all test cases for a given job. It is useful for getting the id of a test case so that you can use other endpoints that require a test case's id.

Code samples for getting a list of test cases

GET

https://api.autoblocks.ai/human-review/jobs/<jobId>/test-cases

curl 'https://api.autoblocks.ai/human-review/jobs/<jobId>/test-cases' \
  --header "Authorization: Bearer $AUTOBLOCKS_API_KEY"

{
  "id": "cljzx8zhd00077w3sqredd0fz",
  "name": "My First Job",
  "reviewer": {
    "id": "cljzx8zhd00077w3sqredd0fz",
    "email": "test@test.com"
  },
  "testCases": [
    {
      "id": "cljzx8zhd00077w3sqredd0fz",
      "status": "Submitted"
    },
    {
      "id": "cljzx8zhd00077w3sqredd0fz",
      "status": "Pending"
    }
  ]
}

GEThttps://api.autoblocks.ai/human-review/jobs/<jobId>/test-cases/<testCaseId>

Get the result of a test case

This endpoint returns a list of all test cases for a given job.

Code samples for getting a list of test cases

GET

https://api.autoblocks.ai/human-review/jobs/<jobId>/test-cases/<testCaseId>

curl 'https://api.autoblocks.ai/human-review/jobs/<jobId>/test-cases' \
  --header "Authorization: Bearer $AUTOBLOCKS_API_KEY"

{
  "id": "cljzx8zhd00077w3sqredd0fz",
  "reviewer": {
    "id": "cljzx8zhd00077w3sqredd0fz",
    "email": "test@test.com"
  },
  "status": "Submitted",
  "grades": [
    {
      "name": "Accuracy",
      "grade": 0.85
    },
    {
      "name": "Clarity",
      "grade": 0.9
    }
  ],
  "automatedEvaluations": [
    {
      "id": "context-evaluation-id",
      "originalScore": 0.75,
      "overrideScore": 0.8,
      "overrideReason": "Adjusted for context"
    }
  ],
  "inputFields": [
    {
      "id": "cljzx8zhd00077w3sqredd0fz",
      "name": "prompt",
      "value": "Explain quantum computing",
      "contentType": "TEXT"
    }
  ],
  "outputFields": [
    {
      "id": "cljzx8zhd00077w3sqredd0fz",
      "name": "response",
      "value": "Quantum computing is a type of computation that harnesses the unique properties of quantum mechanics...",
      "contentType": "TEXT"
    }
  ],
  "fieldComments": [
    {
      "fieldId": "cljzx8zhd00077w3sqredd0fz",
      "startIdx": 0,
      "endIdx": 20,
      "value": "Good introduction",
      "inRelationToGradeName": "Clarity"
    }
  ],
  "inputComments": [
    {
      "value": "Clear and concise prompt",
      "inRelationToGradeName": "Clarity"
    }
  ],
  "outputComments": [
    {
      "value": "Explanation could be more detailed",
      "inRelationToAutomatedEvaluationId": "context-evaluation-id"
    }
  ]
}

GEThttps://api.autoblocks.ai/human-review/jobs/<jobId>/pairs

Get a list of test case result pairs

This endpoint returns a list of all test case result pairs for a pairwise comparison job.

Code samples for getting a list of test case result pairs

GET

https://api.autoblocks.ai/human-review/jobs/<jobId>/pairs

curl 'https://api.autoblocks.ai/human-review/jobs/<jobId>/pairs' \
  --header "Authorization: Bearer $AUTOBLOCKS_API_KEY"

{
  "pairs": [
    {
      "id": "cljzx8zhd00077w3sqredd0fz"
    }
  ]
}

GEThttps://api.autoblocks.ai/human-review/jobs/<jobId>/pairs/<pairId>

Get a specific test case result pair

This endpoint returns the details of a specific test case result pair.

Code samples for getting a specific test case result pair

GET

https://api.autoblocks.ai/human-review/jobs/<jobId>/pairs/<pairId>

curl 'https://api.autoblocks.ai/human-review/jobs/<jobId>/pairs/<pairId>' \
  --header "Authorization: Bearer $AUTOBLOCKS_API_KEY"

{
  "pair": {
    "id": "cljzx8zhd00077w3sqredd0fz",
    "hash": "abc123def456",
    "chosenOutputId": "cljzx8zhd00077w3sqredd0fz",
    "testCaseResults": [
      {
        "id": "cljzx8zhd00077w3sqredd0fz",
        "runId": "cljzxfcxr00077w1yvis84a72",
        "hash": "abc123def456",
        "datasetItemId": "cljzx8zhd00077w3sqredd0fz",
        "durationMs": 1500,
        "events": [],
        "body": {},
        "output": {}
      }
    ]
  }
}