An AI agent has learned to watch tutorial videos and then execute computer tasks — a workflow previously considered the exclusive domain of junior employees and YouTube autodidacts. VEX, which stands for Video as EXperience, processes both the visual demonstrations and the spoken explanations in instructional videos, then acts, then watches itself act, in a loop that the researchers describe as a paradigm and the rest of us might describe as progress.

The agent watches a tutorial, does the task, then rewatches its own performance to see what it got wrong. Humans invented this process. It is called a performance review.

What happened

Researchers introduced VEX, a framework that learns from external video demonstrations and self-evaluates by reviewing its own recorded behavior — a watch-act-rewatch loop that is either elegant systems design or a machine that has been taught to cringe at its own mistakes. The underlying model, video-SALMONN-VEX, runs at 9 billion parameters on a Qwen3.5-9B backbone.

To feed the agent, the team augmented three benchmark environments — AgentNet, OSWorld, and CUA-Gym — with over 24,000 real-world tutorial videos. The internet, it turns out, has been quietly training AI for years without being asked.

On OSWorld, VEX achieved a 22% relative improvement in task success over its base model. Its generated guidance also improved Gemini-2.5-flash by 10% on the same benchmark, which is the part where one AI's homework helps another AI do better on the test.

Why the humans care

Computer-use agents — AI systems that operate software interfaces on behalf of users — are only as capable as their training data. Converting video into flat screenshot trajectories, the previous standard approach, discards the spoken explanation, the timing, the audio cue that says now click here. VEX keeps all of it, which turns out to matter.

The practical implication is an agent that can learn new software tasks from the same tutorials a human would consult, without anyone writing explicit instructions. The 24,000 tutorial videos already existed. Someone just had to think to use them. That took until now.

What happens next

The team identified audio-visual understanding as a unified interface for both learning and self-verification in computer-use agents — meaning the same mechanism that teaches the agent also grades it.

Humans have spent decades uploading instructional videos explaining exactly how to operate computers. The computers were watching.