What Is ThinkAct?

ThinkAct is introduced in arXiv 2507.16815 by Chi-Pin Huang, Yueh-Hua Wu, Min-Hung Chen, Yu-Chiang Frank Wang and Fu-En Yang (NVIDIA, National Taiwan University); the project page labels it NeurIPS 2025. The abstract says end-to-end VLA models map inputs to actions without explicit reasoning, which hinders multi-step planning. Read: the arXiv page, project page and repository README.

Third-party research; Firsthand is not affiliated and claims no use, training or compatibility with it. Firsthand captures first-person human video, not robot data, fast, on demand and to a written spec: 21 keypoints per hand with a visibility flag, wrist trajectories and action segments, plus one task instruction per episode.

The paper calls ThinkAct a dual-system framework. A reasoning multimodal LLM reads the observation and instruction and produces a plan; a separate action model turns it into executable actions. The two can run asynchronously: slow thinking, fast control. Each latent plan covers N interactions, set to 15 for SimplerEnv and 75 for LIBERO.

The reasoning model is fine-tuned with GRPO. The reward combines a goal term (predicted start and end gripper points against a detector’s), a trajectory term (predicted 2D gripper path against the demonstrated one, by dynamic time warping) and a format score. The paper calls this action-aligned visual feedback; the reasoning is then abstracted into a compact visual plan latent.

A Q-Former with 32 queries projects the plan latent into a DiT-based diffusion-policy action model of 432M parameters, pre-trained on Open X-Embodiment. The reasoning model stays frozen while the state encoder, projector and action model are updated by imitation learning on annotated demonstrations.

The reasoning model starts from Qwen2.5-VL 7B. Trajectories come from the fractal and bridge subsets of Open X-Embodiment and, for human manipulation, Something-Something V2 with a hand detector. Question-answer data comes from RoboVQA, EgoPlan-IT, Video-R1-CoT, RoboFail and LLaVA-Video-178K.

The paper evaluates manipulation on SimplerEnv and LIBERO and reasoning on EgoPlan-Bench2, RoboVQA and OpenEQA. It reports a best overall LIBERO success rate of 84.4%, few-shot adaptation from 10 demonstrations per task, a replanning example after a dropped box, and about 17% longer execution than OpenVLA on LIBERO. These are the authors’ claims, not verified here.

The site entries describe ECoT as writing a text plan, sub-task, motion and grounded features, CoT-VLA as predicting a future subgoal image, and MolmoAct as predicting depth tokens and a 2D path trace. ThinkAct trains its reasoner with reinforcement learning on visual rewards and passes a compressed latent, not text, to a separate action model.

Licence, checkpoints and a code release are Not stated in the sources read. The thinkact-vla repository under the project-page author’s account holds a project-page template README; no model code was seen. The paper names inherited hallucination in visual and spatial reasoning as a limitation. Whether first-person human data would suit ThinkAct is Not stated.

ThinkAct’s named inputs are robot trajectories, human video with hand trajectories, and question-answer sets. Firsthand captures human first-person video, not robot data, and states one task instruction per episode, so reasoning traces and question-answer pairs are Not stated; plan them separately. Write tasks, environments and signals into a spec and confirm on a Spec Pilot. Median time to first validated batch: 48 hours where coverage is deep, three to four weeks where thin. Formats: RLDS, LeRobot v2.1, WebDataset, HDF5; mapping to ThinkAct’s pipeline is the buyer’s.

What is ThinkAct?

A dual-system VLA reasoning framework: a multimodal LLM reasons, trained with reinforcement learning on action-aligned visual rewards, and a separate action model executes from a compressed visual plan latent (arXiv 2507.16815).

What is reinforced visual latent planning?

Reinforcement learning with goal and trajectory rewards on predicted 2D gripper paths, then compressing the reasoning into a plan latent that conditions the action model.

What does ThinkAct claim on benchmarks?

A best overall LIBERO success rate of 84.4%, per the paper. This is the authors’ claim, not verified here.

Is ThinkAct open source?

Not stated in the sources read. The thinkact-vla repository holds a project-page template README; no model code or licence was seen.

Does Firsthand use ThinkAct, and can I request a custom dataset?

Firsthand is not affiliated and claims no use, training or compatibility. You can request a custom dataset, confirmed first on a Spec Pilot.