People Score 93.1% on Scale’s Humanity’s Sixth Sense; the Best Model Manages 53.6%

Scale AI introduced Humanity’s Sixth Sense (HSS) on October 7, 2026, a benchmark built with Elorian. It measures the intuitive visual reasoning people do almost instantly: who holds authority in a room, whether a car will fit between two parked ones, or why a runner suddenly slows down. On Scale’s leaderboard, human participants score 93.1%. The strongest model, GPT-6-astra at maximum reasoning effort, reaches 53.6%, and the median model scores 30.9%. The tasks, reference answers and grading rubrics are on Hugging Face under the MIT license.

Scale argues that today’s visual benchmarks mostly test either expert academic analysis or low-level perception, which leaves out the everyday inference that people do without thinking. “For AI agents deployed in homes, vehicles, and workplaces, this skill is not a luxury, but a core necessity,” the announcement says.

522 tasks, each passed by three reviewers

The dataset contains 522 open-ended tasks: 288 built on still images and 234 on video clips, totaling 17.6 hours of footage. Each task pairs a scene with a human-written question, a reference answer and a rubric of checkable criteria, with one to seven criteria per task and 723 in total. The tasks are grouped into four domains and eleven subdomains:

• Physical and spatial logic (176 tasks): occluded objects and hidden forces, whether something will fit, what a scene looks like from a viewpoint the camera never had, and whether a person or object can reach a spot.
• Temporal and causal dynamics (134): what already happened, what is physically driving an event, and what happens next.
• Social understanding (110): beliefs, knowledge gaps and emotions, plus social roles, unwritten norms and power dynamics.
• Abstract and contextual inference (102): consequences of a change the scene does not show, and patterns, including faces seen in objects.

Authoring rules were strict. A task had to require inference beyond what is visible. If a high-resolution crop made the answer obvious, the task was rejected. It could not require specialist knowledge, and its answer had to be one people agreed on unanimously. Each task went through three independent review rounds. Of 3,466 authored tasks, 522 survived, a 15.1% acceptance rate.

Models answer in free form, with no multiple-choice options to eliminate. An LLM judge scores each answer against the rubric, and a task counts as solved only if every criterion is met. Scale says it re-graded five models using judges from three vendors. Rankings were identical under all three, and agreement exceeded 95%.

Where the models break down

Scale’s announcement says it evaluated 25 multimodal models from eight vendors alongside 20 human participants. The dataset card lists 24 evaluated models. Several findings stand out:

• The image actually matters. With the image or video removed, GPT-6-astra falls to 6.6%, so the tasks cannot be solved from language patterns alone.
• More thinking does not close the gap. Models spend an average of 4,046 reasoning tokens per task on questions people answer at a glance. Raising effort helps overall but hurts in some areas. GPT-6-astra drops 14 points on retrodiction when moving from high to xhigh effort.
• Most failures are perception, not logic. Across 8,573 failures, 94% trace to perception or latent inference and only 5% to faulty logic. The two biggest causes are missing the decisive visual cue (21%) and misidentifying an object, person or role (20%).
• Social scenes are hardest. Social understanding is the lowest-scoring domain for 21 of the 25 models, averaging 24.4% against 34.1% for the other three. Video is harder than images for 23 of 25 models, by 7.3 points on average.

In one example, models missed that a runner slowed down because a bus had stopped to wait for her.

Agent tooling narrows the gap only partly

Scale also tested models inside agentic coding harnesses, Claude Code and Codex, where they can crop, zoom, search the web and re-sample media. The best of those setups reached 59.3% on a 388-task subset. A closer look fixed some missed cues and misidentified objects, Scale says, but not errors of depth perception. Mistakes from reading a 2D overlap as 3D alignment went from 102 to 101.

A test set built for agents that see

HSS joins a growing set of open evaluations aimed at the abilities agents need once they leave the text box. Scale’s conclusion is that the gap holds “even with more test-time compute or agentic tooling,” which points to perception and inference, not reasoning chains, as the bottleneck. Because the dataset ships with its rubrics and a test split only, outside labs can reproduce the scoring. Teams building agents for robots, vehicles or homes now have a public measure of the gap.