What Is Dita?

Dita is introduced in Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy (arXiv 2503.19757), labelled ICCV 2025 on its GitHub repository, with an official project page and GitHub repository. The authors observe that vision-language-action models trained on diverse robot datasets generalize with limited in-domain data, but rely on compact action heads to predict discretized or continuous actions, which constrains adaptability to heterogeneous action spaces. Dita instead uses Transformer layers to denoise continuous action sequences directly, through a unified multimodal diffusion process, and scales the denoiser together with the Transformer. The paper presents it as a clean, lightweight, open-source baseline for generalist robot policy learning. Every technical statement on this page comes from the paper, the project page and the GitHub repository, every result belongs to the authors, and this page reproduces no scores. Firsthand is not affiliated with the Dita authors and makes no claim that Dita or any other named model was trained or evaluated on Firsthand data.

Per the paper, earlier generalist policies such as Octo and pi0 denoise each action with a small network conditioned on one embedding from a causal Transformer. Dita instead concatenates the language tokens, image features, timestep embeddings and the noised action into one token sequence and feeds it to a causal Transformer, so denoising is conditioned directly on image tokens from historical observations. The authors say this lets the model capture action deltas, the subtle changes between observations, and that the diffusion objective is applied in the action-chunk space by a large Transformer rather than by a diffusion action head. In training, Gaussian noise is added to the action at a random timestep, the network predicts that noise, and the loss is mean squared error. Inference starts from noise and iterates denoising steps. The authors test the idea against a baseline with a three-layer MLP diffusion head on the same Transformer. The paper adds that further inputs, such as wrist-camera images, robot state or tactile feedback, could be added to the Transformer, but its default configuration takes only language and third-person images.

The language instruction is tokenized by a frozen CLIP model. Images are resized to 224 by 224 and encoded by DINOv2, whose parameters are fine-tuned along with the policy, and a Q-Former with FiLM conditioning, depth 4, reduces the image features to a length of 32 based on the instruction. The action is a 7D end-effector vector, three translation dimensions, three rotation dimensions and one gripper dimension, padded with zeros to match the token dimension. The core is a LLaMA2-style causal Transformer trained from scratch, with 12 self-attention blocks and a hidden size of 768. The paper puts the total at 334M parameters, 221M of them trainable.

Pretraining uses the DDPM objective. The main text gives 1,000 timesteps for pretraining and DDIM with 20 timesteps for zero-shot evaluation, while the appendix describes a DDPM scheduler with 100 timesteps. How the two statements relate is Not stated in the sources read. The model takes 2-frame image observations and predicts 16 action chunks, trained with AdamW for 100,000 steps at batch size 8192 across 32 NVIDIA A100 GPUs. For real-robot adaptation the paper fine-tunes with 10 samples per task, using LoRA, while the README says fine-tuning without LoRA gives clearly better performance, including robustness to variances and longer-horizon tasks.

The paper selects the Open X-Embodiment datasets for pretraining, following the dataset selection and weight assignment of Octo and OpenVLA, with actions normalized and filtered similarly to Open X-Embodiment. The number of trajectories, datasets or hours is Not stated in the sources read. The authors evaluate on four simulation benchmarks: SimplerEnv, evaluated zero-shot, plus LIBERO, CALVIN and ManiSkill2, which are fine-tuned on their own datasets. For ManiSkill2 the paper builds a camera-view generalization benchmark from a pool of 300K random cameras.

The real-robot setup is a Franka Emika Panda arm with a Robotiq 2F-85 gripper and one RealSense D435i camera about 1.5 m away in a third-person view, running at a control frequency of 3 Hz. Task families are pick and place, pour, stack, pick and rotation, pull and push, and long-horizon tasks, each fine-tuned from 10 demonstrations. The README links checkpoints on Google Drive, including one pretrained on Droid. Dataset licences are Not stated in the sources read. The GitHub page lists the Apache License 2.0 while the README badge reads MIT, so read the licence file before a commercial program relies on the code. Whether human egocentric video was used in pretraining is Not stated in the sources read.

The first comparison table sets them side by side. The Dita paper notes that existing diffusion manipulation policies predominantly rely on U-Net or shallow cross-attention networks designed for single tasks, and that RDT uses cross-attention Transformers or DiT decoders for bimanual manipulation. Its own proposal is a DiT with in-context conditioning over historical observations. Per this site’s entries, Diffusion Policy is the original conditional denoising formulation, RDT-1B conditions on language, up to three RGB views and proprioception to predict the next 64 actions, and CogACT places a diffusion transformer action module behind a 7B vision-language model’s cognition feature.

Dita has no pretrained vision-language model: it trains a LLaMA2-style Transformer from scratch on top of CLIP and DINOv2 encoders, and it uses third-person images only by default. Which design is better is Not stated in the sources read, and this page ranks none. The authors report their own comparisons against other generalist policies, which this page does not reproduce.

Dita’s inputs are a language instruction and third-person camera images, and its output is a robot end-effector action. Firsthand captures first-person human video, not robot, teleoperation or simulation data, so it does not deliver the end-effector actions Dita is trained to denoise. Whether human egocentric data would help a policy like Dita is Not stated in the sources read, and this page draws no conclusion about it.

What the site does state is custom egocentric collection, fast and on demand, to a written spec: a head-mounted camera plus two wrist cameras on one clock, 3D hand pose at 21 joints per hand with a per-joint visibility flag, wrist trajectories, action segments, calibrated stereo depth and 6-DoF IMUs at 200 Hz. The only language data is one task instruction per episode. Confirm the spec on a Spec Pilot, a small fast first batch. The site states a median of 48 hours from signed spec to first validated batch where coverage is deep, and three to four weeks where contributors must be recruited first. Billing is per validated hour, and delivery is in RLDS, LeRobot, WebDataset or HDF5 under a buyer-owned commercial license.

What is Dita?

A diffusion transformer policy for generalist robot learning. It takes a language instruction and third-person camera images and denoises a chunk of continuous end-effector actions with in-context conditioning, per arXiv 2503.19757.

What does in-context action denoising mean in Dita?

Per the paper, the language tokens, image features, timestep embeddings and noised action are concatenated into one sequence for a causal Transformer, so denoising conditions directly on image tokens from historical observations instead of on one fused embedding.

How many parameters does Dita have?

The paper appendix gives 334M parameters in total, with 221M trainable.

What data was Dita pretrained on?

The paper selects the Open X-Embodiment datasets. Trajectory, dataset and hour counts are Not stated in the sources read, and the sources read do not say whether human egocentric video was used.

Is Dita open source, and what is the licence?

The paper calls it open-source and the code is on GitHub. The GitHub page lists the Apache License 2.0 while the README badge reads MIT, so read the licence file before a commercial program relies on it.

Did Dita use Firsthand data, and can I request a custom dataset?

No. Firsthand is not affiliated with the Dita authors and makes no claim that Dita or any model was trained on Firsthand data. You can request a custom egocentric dataset: write the tasks, fields and format into a spec, confirm it on a Spec Pilot, and billing is per validated hour. Firsthand captures first-person human video, not robot, teleoperation or simulation data.