Latest / Embodied AI 101 / From Tokens to Thoughts: Continuous Latent Reasoning in Large Models and Robot Control

Transcript

From Tokens to Thoughts: Continuous Latent Reasoning in Large Models and Robot Control · Embodied AI 101

• From Tokens to Thoughts: Continuous Latent Reasoning in Large Models and Robot Control. Large AI models – whether pure language models (LLMs), vision-language models (VLMs), or embodied agents (sometimes called VLA for Vision-Language-Action) – traditionally perform multi-step reasoning by emitting discrete tokens. A language model answering a math problem will spell out each arithmetic step as a word or symbol; a robot with a “Vision+Language” controller might generate a sequence of textual commands before executing a motor primitive. But recent work is challenging this paradigm. Instead of chaining words, researchers are experimenting with continuous latent representations – internal vectors and codes – as the medium of thought. The motivating insight is that human language tokens are often a cumbersome proxy: many tokens serve only coherence, and committing to explicit words too early can waste information or force a wrong path. By reasoning in an abstract latent space, a model can carry richer information, explore multiple ideas in parallel, and interface more flexibly with its environment (for example, controlling a real robot). This shift appears across the board. In LLMs, new methods propose doing chain-of-thought in the model’s hidden state rather than as text. In vision-language systems, models are being designed to inject or reason over latent “visual tokens” instead of stepping through words alone. And in robotics, teams are replacing symbolic language plans with learned latent action codes that bridge high-level goals and low-level motions. In each case, the latent representation can encode complex substeps, multiple alternatives, or perceptual context that discrete tokens would struggle to capture. We are seeing a new research wave aiming to use continuous latent spaces as the substrate for reasoning. This article surveys these trends – with an emphasis on work that could inform embodied AI – and explains the motivations and methods behind them. Why Go Latent? The Limitations of Tokens. To understand why latent reasoning might help, it’s worth reflecting on how discrete chains-of-thought work today, and where they fall short. In a typical LLM solving a math or logic puzzle, the model is prompted (or fine-tuned) to elaborate its reasoning step by step in natural language. This is the well-known “chain-of-thought” (CoT) paradigm. Each intermediate result or substep is explicitly written as a token sequence, which the model then uses to generate the next part. While CoT can dramatically improve accuracy on hard problems, it also has costs:. Token Inefficiency: A lot of tokens get spent on linguistic glue (“if”, “let’s say”, “therefore”, etc.) and explanations that mainly serve readability. The internal mathematical or logical relations only occupy part of the text. For example, solving “(7 − 2)*(5 + 1) = ?” with chain-of-thought requires multiple words for a simple calculation, or explaining the distributive property with many tokens, none of which add new semantic information beyond what the model could carry in a vector. Premature Commitment: When an LLM decodes its hidden state into a specific word, it effectively commits to that idea. Suppose the true solution involves branching reasoning (e.g. an “either/or” strategy). Once a token is chosen, backtracking is hard. This can lead to tunnel vision: the model generates a single explanation tree even if several were plausible, potentially missing the correct one. Learning Overhead: Fine-tuning or training a model to generate good explanations can devour the model’s capacity. The weights have to encode not only the problem logic but also how to express it in fluent prose. Moreover, some downstream tasks (like controlling a robot) might suffer if the model has overfit to language fluency rather than task-specific signals. Because of these concerns, researchers ask: what if the model could “think” in its hidden representations, rather than in spoken tokens? A few key ideas have emerged:. Continuous Latent Ideas: Instead of generating a word, the model keeps its computation vector (the hidden state) in place and uses it directly as the next input. Thus, the chain-of-thought occurs in the continuous activation space, not in symbol space. This can carry more precise information (you’re not rounding or approximating by a word token) and can in principle hold a superposition of multiple ideas. Latent Multiple-Path Search: A continuous state can implicitly encode multiple partial solutions. For example, one recent work observed that its hidden state sometimes contained information about several possible next steps (like nodes in a search tree), allowing a kind of breadth-first exploration. In contrast, discrete CoT immediately chooses one token path. Bridging Domains: Hidden vectors can naturally interface with non-linguistic systems. For robotics, a latent code can plug straight into a policy network to generate an action, without translating through English. For multimodal tasks, a latent visual feature can be manipulated without writing down every pixel description. These ideas sound abstract, but several new papers have begun to validate them on real tasks. For example, Shibo Hao et al. (Meta AI) propose COCONUT (Chain of Continuous Thought), a method that simply keeps the LLM’s hidden state as the “reasoning trace” instead of decoding it into words. In COCONUT, at each step the model uses its last hidden representation as input for the next step, skipping the token-generation step altogether. This means the model reasons by updating its latent state directly. In practice they found this improved reasoning on tasks that require complex planning: COCONUT sometimes outperformed standard CoT on logical puzzles, despite using far fewer tokens. Moreover, the hidden-state approach could represent multiple tentative conclusions at once; the authors visualized cases where the model was essentially doing a breadth-first search in latent space, rather than prematurely choosing one path as CoT would. In short, COCONUT shows that letting the LLM “think” in its own continuous space can yield more powerful reasoning than forcing it to talk every step out loud. Another thread is adding auxiliary latent tokens that guide reasoning. A recent paper on multimodal LLMs (still under review) introduces the idea of generating “vision-aligned latent tokens” just before each reasoning step. In essence, the model creates a vector representing visual context or a latent hint, then conditions its next chain-of-thought on that vector. This way, the LLM has a distilled summary of relevant perceptual data feeding into its logic. We can think of this as a variant of “latent reasoning” where images or other modalities inject continuous cues. Early reports (including on latent visual chain-of-thought methods) suggest this reduces verbosity and improves accuracy on vision-involved tasks, though much of this is very new work. The key point is that even within an LLM that consumes text and images, reasoning can happen partly in a latent embedding space instead of a pure text dump. The high-level takeaway is that latent reasoning in LLMs is an emerging principle. Instead of every sub-step being a sentence, the model can manipulate vectors behind the scenes. This allows it to compress information, branch out, and integrate other neural cues. It’s perhaps a more “brain-like” approach, where internal representations prepare an answer before the speaker articulates it. Vision-Language Latent Thinking. The shift to latent reasoning is not confined to text-only models. In vision-language models (VLMs) – systems that jointly process images and text – a similar insight is gaining traction. Traditional VLMs like CLIP or BLIP essentially map images to visual embeddings, then operate in the discrete token space of the language model. But there are now visions of doing more of the reasoning directly in a combined latent space. One flavor of this is using latent representations within chain-of-thought-like processes for visual reasoning. For example, when a model is asked a question about an image, instead of generating a textual explanation step by step, it might update an internal visual “scene” representation at each reasoning step. Conceptually, the model could refine its latent understanding of the image as it thinks, rather than repeatedly referring to raw pixels or image patches. This is still an open area. Some early proposals suggest generating latent visual tokens – vectors encoding relevant image features – at each step and infusing them into the reasoning. This is akin to the vision-aligned latent tokens idea mentioned above. While concrete results are still emerging, the intuition is that a VLM can be much more efficient if it reasons over its own continuous visual features rather than trying to string together a purely textual justification. Another line of work is interweaving visual and textual latents in a unified language model. For instance, imagine extending COCONUT’s idea: an LLM that processes an image might keep a joint hidden state that absorbs both text and vision cues, and update that hidden state continuously, instead of dealing with image tokens symbolically. Some prototypes of this have been floated (e.g. “multimodal latent CoT” frameworks), and they aim to let the model generate both text and non-text tokens in a seamless latent flow. The precise methods vary, but they share the goal of richer multi-step reasoning by leveraging the high-capacity latent space that combines language and vision. To give a concrete example, consider a robot with a camera that’s asked to perform a task like “move the red cube to the green bin.” A traditional approach might have the model describe the scene in words, then spit out “pick up red cube” as a text command, then later “place cube in green bin.” In a latent-space approach, the model could instead update a neural representation of the scene as it thinks (“the cube is here, the bin is there”), internally generate a latent action plan (e.g. a vector that encodes the whole operation), and only at the last moment produce the actual motor commands. Some recent research has even implemented pipelines where the AI never outputs an English command; it just works with its own latent codes to perform the task. This often makes the planning smoother and allows the model to “preview” the outcome in its latent space. All this points toward vision-language-action (VLA) models using latent spaces as well. In fact, the leading edge of VLA research is precisely exploring how continuous action representations (latent or code-based) can unify perception, language, and control. The lists of recent papers under “Vision-Language-Action” surveys include many hits on this idea (for example, Omnijarvis, LAPA, QueST, UniVLA, etc.; see below). These works show that as we push toward generalist robot agents, latent representations are emerging as the lingua franca for reasoning and action, rather than natural language alone. Bridging Language and Action with Latent Codes. One of the most tangible areas where latent thinking matters is robotics. Embodied agents must translate high-level instructions into low-level controls, and for years people have explored language as the bridge: ask a chatbot for steps and have the robot attempt each. But natural language can be an awkward interface for fine motor control or complex maneuvers. Recent work takes the bold step of inserting a learned latent layer as the interface layer between “thinking like a human” and “acting like a machine.”. Take the LCB (Latent Codes as Bridges) approach. In a recent paper, a team recognized that tying language plans directly to policies has limitations: not every action sequence is expressible or optimal in words, and adapting the whole system end-to-end can corrupt the language model’s knowledge. So they introduce a small latent vector (a learned code) that sits between the high-level planner and low-level policy. Concretely, an LLM like GPT-4V is used to reason about the task, but instead of outputting a textual command, it outputs a latent goal vector. This latent vector is then fed into a policy network that generates the actual robot actions. Crucially, this latent code is trainable, so during fine-tuning the model learns what it should output to make the policy act correctly. The result is that the LLM can communicate complex goals implicitly; it’s not constrained to use words. At the same time, finetuning on new tasks can focus on mapping latent codes to actions without overwriting the LLM’s language knowledge (the LLM’s token embeddings remain intact). In experiments, the LCB model outperformed systems that used pure language commands, even when comparing against a strong vision-LM like GPT-4V. On tasks requiring multi-step reasoning (e.g. solving a sequence of pick-and-place goals), LCB achieved higher success. In other words, giving the LLM a little continuous “elbow room” via latent codes lets it specify goals flexibly and ultimately led to better performance. Another system taking a similar tack is OmniJARVIS (NeurIPS 2024). This is a vision-language-action model trained in the open-world game Minecraft. Instead of relying on separate modules for text planning and policy, OmniJARVIS learns a unified tokenization of everything the agent does. The key trick: they train an encoder that discretizes entire behavior trajectories into a finite set of “behavior tokens.” These tokens (think latent codes representing chunks of action) are then added to the language model’s vocabulary. At training time, the model sees sequences that include instructions, observations, internal thoughts, rewards, and those learned behavior tokens. It is trained to autoregressively predict the entire sequence. Because the behavior tokens carry semantics about the agent’s actions, OmniJARVIS can use them to plan and act. In effect, it reasons in a mixed space of words and action codes. After training, the model is capable of both “thinking” (generating chain-of-thought text) and “acting” (outputting behavior tokens that a decoder turns into game controls). Impressively, OmniJARVIS handled a wide variety of tasks in Minecraft – from completing quests to answering questions about the game – all using this joint language-action latent framework. An important takeaway is that even when action space is complex, we can teach a model a discrete latent vocabulary of actions and let it plan using those tokens. Beyond these examples, several other recent lines of work show the power of latent actions:. QueST (NeurIPS 2024) learns to represent short sequences of robot controls (like 1–2 seconds of arm movement) with discretized skill tokens. This model is essentially a latent-variable behavior generator: each skill token indexes a small neural motion pattern. By clustering actions into such tokens via a latent codebook, QueST trains on multi-task robot data and learns a vocabulary of general-purpose action chunks. At inference time, a high-level policy outputs a sequence of these skill-tokens, which are then decoded into joint trajectories. The result is a system that generalizes better to new tasks, because it builds on shared latent primitives instead of raw actions. LAPA (ICLR 2025) pushes pre-training of vision-language-action models without any human-labeled actions. LAPA uses a VQ-VAE to learn a discrete latent action space from off-the-shelf videos. Concretely, it takes internet videos of humans interacting (with no robot data or labels) and learns to encode the frame-to-frame changes into a finite codebook of “latent actions.” Then it trains a large VLA model to predict these latent actions from egocentric observations and instructions. After this, a small amount of real robot data is used to map those latent codes to actual robot joint commands. This pipeline – which never needed explicit labeled actions except at the very end – significantly outperformed previous methods. On real manipulation tasks, LAPA beat state-of-the-art VLA models that had been trained with extensive labeled robot trajectories. In particular, it did better at generalizing to novel objects and instructions. The implication is clear: a rich latent action space learned from diverse videos can actually surpass even the best fully-supervised robot policies. It taps into broad visual patterns of motion that discrete language or imitation data missed. UniVLA (RSS 2025) takes latents even further across embodiments. The problem: most robot learning approaches train on just one robot or simulator, so the policies don’t transfer if the hardware changes. UniVLA addresses this by creating a universal latent action space that ties together many contexts. It learns task-centric latent actions from a huge mix of internet videos (covering human kitchens, workshops, YouTube clips, etc.) using hidden features from a vision model (DINO) as the latent space. Then, given a new robot or environment, the model decodes these latents into that robot’s actions. In tests, this lets one model control very different robots (e.g. pushing tasks, navigation) far more effectively than baselines. UniVLA achieved state-of-the-art results on several benchmarks with far less data, all by leaning on that shared latent space. Notably, as more diverse (even human-hand) data are added, performance increases, showing the latent-focused approach scales elegantly. Together, these examples sketch a new paradigm: robots (and agents) can internally reason in a latent action space. This space is often learned (via VQ-VAEs, clustering, self-supervision) and then interfaced with high-level planners. The high-level model (LLM or VLA transformer) produces latent codes rather than words. Those codes are rich: they can capture temporally extended skills, encode multi-step strategies, or align with visual goals. And because they are continuous or high-dimensional, the model need not fumble for the “right word” for each nuance. At the end of the pipeline, those latent codes are decoded (often by a small neural network) into actual motor commands. Crucially, this decouples the reasoning from the actuation: the planner never has to relearn the joint control from scratch, it just learns to hit the right latent embedding. As evidence mounts, latent intervening representations are showing clear benefits. In controlled experiments, models with latent bridges often solve tasks that language-only systems fail. They also tend to require less fine-tuning data to start solving new tasks – presumably because the latent abstraction generalizes better than raw language. Put another way, the latent codes act like an information bottleneck and an information bank: they force the planner to commit to only what’s necessary for action, and simultaneously allow storing rich structure outside human language. Continuous Latents in Differential Game Landscapes. Why does this matter for robotics and embodied AI? At a high level, we want our systems to be grounded in the real world but still leverage the power of modern AI. Language is great for human-AI interaction, but it may not always be the most natural internal representation. The move to latent spaces suggests we can have both worlds: natural language to express goals (“put the bottle on the table”), and powerful latent codes to implement them in continuous space. Moreover, thinking in continuous spaces often smooths out reasoning. Discrete tokens can produce a very rugged search space: changing a single word can flip the plan. Latent spaces can encode continuity (small changes in the vector cause small changes in action), which could make planning and learning easier. For example, interpolation between latent codes can smoothly morph one skill into another – an effect hard to get if the model only sees distinct English instructions for each skill. Importantly, many robotics tasks are compositional and hierarchical, and latent layers fit naturally into that hierarchy. An LLM can decide in latent space which sub-policy to run next, or blend sub-policies together via latent superposition. Meanwhile, low-level controllers learn to decode those latents into smooth motions. This mirrors how humans might plan (with action primitives in mind) rather than parsing instructions word by word. Finally, these advances reflect a broader trend in AI: world models and active perception often inhabit latent spaces. If a robot has a world model in latent form (like imagining future states in code, not raw images), bridging to action via latents is very natural. We see related work on video diffusion as predictive world models, on high-level latent planners, and on using VAEs for state aggregation – all part of the same latent toolkit. The 100+ papers collected in the “Latent Space in LLM/VLM/VLA” compendium point out this convergence. Across language, vision, and robotics, latent representations are becoming the lingua franca of complex reasoning. Outlook: Latent Learning and Beyond. The idea of latent-space reasoning and acting is still young. Many questions remain: How best to structure these latent spaces? Should they be discrete codes or fully continuous vectors? How do we ensure they stay interpretable enough to fine-tune effectively? Some of the new work (like the Vocabulary-Space Superposition approach) even suggests tying latent reasoning to the model’s own token space for safety and interpretability. Others are exploring more exotic supervisions (e.g., incorporating feedback or even human intent prompts into the latent encoder training). For practitioners, the message is that these methods are worth experimenting with. If you have a complex multi-step task, consider whether forcing an LLM to speak every step is really necessary. Could you insert a hidden latent state instead? Many recent code libraries (like VQ-VAEs or latent diffusion tools) make it feasible to implement your own latent skill discovery. Likewise, if you work on multi-modal models, think about whether a few latent tokens summarizing vision could speed up your inference. And in robotics, look at these bridges as a way to decouple your high-level planner from the unpredictable details of control. That decoupling often translates into robustness against domain shift and more sample-efficient learning. In summary, the shift to continuous latent spaces is one of the most exciting undercurrents in modern AI. It promises to remove bottlenecks in reasoning and to unify vision, language, and action in a common framework. For embodied AI researchers, this means we may soon have agents that literally “imagine” their actions in a rich internal space before moving a finger. The hundreds of papers listed in recent surveys (as of late 2024/early 2025) collectively point to latent thinking as a high-return research area. As we build the next generation of robots and agents, giving them a lush continuous canvas for thought might just make them greatly smarter – and more capable of turning instructions into intelligent behavior. Citations: Recent work includes COCONUT: Chain of Continuous Thought (Hao et al., ICLR 2025), which reasons entirely in hidden-state space; OmniJARVIS (Wang et al., NeurIPS 2024), which tokenizes behaviors into discrete latent actions within a VLA model; QueST (Mete et al., NeurIPS 2024), introducing “skill tokens” for robotic motions; LAPA (Ye et al., ICLR 2025), which pretrains VLA models using latent actions learned from unlabeled videos; LCB: Latent Codes as Bridges (Shentu et al., arXiv 2024), which uses a learned continuous code between language planning and policies; and UniVLA (Bu et al., RSS 2025), which leverages task-centric latent action models across different robots. Each of these shows how continuous latents can outperform purely language-based interfaces on reasoning and control tasks. Our discussion has drawn on these and related works to highlight the emerging theme of latent-space reasoning for embodied AI.