Latest / Embodied AI 101 / Tactile VLA Avocado Sorting: Do Continuous Features Beat Labels?
Transcript
- Tactile VLA Avocado Sorting: Do Continuous Features Beat Labels?. Continuous Versus Discrete Tactile Representations for Vision-Language-Action Avocado Sorting • Ran Yang et al. • Proceedings of the AAAI Symposium Series • 2026. Cold Open. Imagine two avocados sitting in front of a robot. The instruction is simple: put the riper one in the collection bin. The arm knows where both objects are. It can execute the grasps and reach the container. The interesting decision happens between those movements: what should the robot believe after making contact?. Now imagine two interfaces between the tactile sensor and the policy. One sends a category: “firm.” The other sends a graded description of the contact response. Which interface gives the robot the information it needs to choose its next action?. That is the research question suggested by Yang and colleagues’ paper. Source-access note: I could not retrieve the paper through its DOI or locate an accessible full-text copy. Consequently, I cannot verify its architecture, training data, experimental results, or ablations. What follows is a research-grounded analysis of its question using accessible primary literature—not a completed review of the paper’s experiments. The distinction matters: the results below belong to explicitly named related studies. The Idea. Start by separating three decisions that often get bundled together: how touch is measured, how its evidence is represented, and how the robot produces actions. A continuous tactile input does not imply a continuous-action policy, and a categorical tactile input does not imply discrete robot movements. CLAW, for example, feeds discrete directives into a policy that generates continuous actions. Tactile-VLA instead integrates tactile measurements into the policy’s input and predicts both position and force targets. These are different choices at different interfaces. For a hypothetical avocado sorter, a discrete representation might be a predicted ripeness category. A continuous representation might be an estimated firmness value, a probability distribution over ripeness categories, or a learned feature vector summarizing contact. Those alternatives preserve different kinds of evidence. They also ask different things of the downstream policy. That vocabulary is a conceptual map, not a verified description of Yang’s implementation. In particular, “discrete” could mean a semantic label or a learned codebook. Those are substantially different experimental treatments. The surrounding literature provides several concrete architectural precedents. CLAW demonstrates a semantic interface. A fine-tuned CLIP model monitors a scale’s digital display and produces directives indicating whether collection should continue or stop. A π₀ policy receives those directives together with camera observations and executes the manipulation. The important architectural move is separating condition evaluation from movement generation: the perceptual component handles the threshold decision, while the VLA handles execution. Tactile-VLA demonstrates direct policy integration. It encodes a history of tactile measurements with a small multilayer perceptron, combines the resulting token with visual and language inputs, and trains a force-aware action expert through imitation learning with flow matching. Its outputs specify desired positions and contact forces, which a hybrid position–force controller executes. Here, touch participates in generating the physical interaction itself, including how firmly the robot acts. UniTouch addresses another part of the problem: constructing useful tactile representations before choosing a downstream controller. It aligns touch embeddings with pretrained image embeddings already associated with other modalities. Sensor-specific tokens support learning across heterogeneous tactile sensors. Its contribution is a shared representational space connecting touch with vision and language, rather than a particular avocado-sorting controller. Together, these examples reveal the architectural question behind the paper’s title. At what point should a system turn a physical observation into a decision? A semantic interface makes that decision upstream. A learned tactile embedding allows downstream computation to use a richer intermediate state. A measured property, such as estimated hardness, occupies a useful middle ground. My criterion for a meaningful avocado study would therefore be straightforward: demonstrate how the tactile interface changes the robot’s decisions while making the rest of the system understandable. The interesting contribution would be evidence about where interpretation should happen, not simply another demonstration that a robot can squeeze fruit. The Result. The verified experimental anchors come from related work. They establish why this question is worth asking. The closest avocado example is Octopi, presented in 2024. Its authors mounted GelSight sensors on a Franka Emika Panda and collected tactile observations from avocados. The model combined tactile representations with explicit descriptions of physical properties, then selected which avocado in a pair was riper. The reported pairwise ripeness accuracy was 63%. The robot could receive the resulting selection through ROS and place the chosen fruit in the ripe bin. That experiment identifies a concrete role for touch: supplying evidence for object selection. The measured outcome is a ripeness judgment between two candidates. It connects a physical observation to a semantic decision that can direct subsequent manipulation. Octopi’s broader experiments also found that intermediate property descriptions improved several physical-understanding and reasoning tasks. A complementary result comes from TAP-VLA, introduced in June 2026. It extracts shear fields from GelSight observations and draws spatially grounded vectors into the camera images already consumed by the VLA. The policy therefore receives tactile information through visual annotations. On a real Franka Panda, the authors reported 78% task success across experiments involving object mass, center of mass, and insertion alignment, outperforming their vision-only and alternative tactile-fusion baselines. The implementation used π₀.₅ and teleoperated demonstrations. Its baselines included feeding raw tactile images as additional views and introducing a separate tactile encoder. The comparison makes the observation interface an explicit experimental variable: the same sensing hardware can support different routes into the policy. TactEx supplies another relevant example. It performs controlled pressing with a GelSight-Mini sensor, estimates hardness from a tactile image sequence using a ResNet–LSTM model, and combines those estimates with object identity and spatial context to generate language-grounded explanations. This architecture gives a numerical physical estimate a clearly defined role between sensing and interpretation. My synthesis is that these studies establish three useful destinations for tactile evidence: selecting an object, adjusting manipulation, and estimating a physical property. An avocado-sorting comparison becomes especially informative when it identifies which of those decisions benefits from its representation. The numerical result for Yang and colleagues’ continuous-versus-discrete comparison remains unverified here. Neither Octopi’s accuracy nor TAP-VLA’s success rate should be attributed to that paper. The Biggest Weakness. The strongest threat to a broad “continuous is better” thesis is an unequal-information comparison. This is the central methodological objection to check—not a defect I can confirm in Yang’s paper without reading its methods. Consider a thought experiment. One policy receives an entire tactile trajectory, including the evolution of contact and the magnitude of deformation. Another receives one predicted category. If the first policy wins, several explanations remain possible. It may benefit from temporal evidence, uncertainty, a physical quantity relevant to grasping, or information that the category was never designed to express. The experimental difference is then larger than continuity versus discreteness. The clean question is whether the representation’s form helps when the underlying evidence is controlled. I would want the comparison to begin with the same contact observations and make the compression step explicit. A continuous embedding could be compared with quantized versions of that embedding at different resolutions. A hard semantic label could be compared with its class-probability vector to test the value of retaining uncertainty. Those comparisons answer different questions, and that is precisely their value. They distinguish the effect of quantization from the effect of committing early to a particular interpretation. There is a useful limiting case. Suppose the task is only to assign a fruit to a known ripeness bin, and an upstream classifier supplies the correct bin label. That label already contains the required routing decision. A richer representation becomes valuable when additional evidence changes what the robot should do—for example, whether it should gather another observation or adjust its handling. A victory over a single hard label would establish the value of retaining information beyond that label. Establishing a general advantage for continuous tactile representations requires isolating representation form from information content. That is the one comparison I would examine first in the full paper. Verdict. The verified literature makes this question relevant to two groups: researchers adapting pretrained VLAs to physical feedback, and manipulation engineers deciding where perception should end and control should begin. For VLA researchers, the interesting object of study is the interface. There are concrete precedents for semantic directives, tactile tokens, numerical property estimates, and visualized tactile measurements. My takeaway from that variety is that “add touch” is not a sufficiently precise architectural objective. Specify which decision the additional observation is intended to improve. For a food-handling engineer, I would frame the design around a sequence of decisions: selecting the fruit, choosing how to make contact, interpreting the measurement, and executing placement. That decomposition gives every component a job. It also creates an intelligible story about why a particular tactile representation belongs at a particular stage. There is a practical starting point in the surrounding work. The official Octopi repository includes training and evaluation code, configuration examples, and instructions for using its model weights. It documents tactile-encoder training, feature alignment, and downstream language-model training. These artifacts provide a concrete reference for constructing a tactile-language baseline and tracing the transformation from sensor observations to property descriptions. My recommended research emphasis is the decision pathway: follow one observation from contact, through representation, into the action it changes. That makes the contribution understandable to someone who might actually integrate the system. The one-line takeaway is: choose a tactile representation around the decision the robot must make, and measure its value at that decision. That is the substantive position supported by this research context. A paper-specific verdict on Yang and colleagues’ contribution requires their experimental evidence. Outro. An avocado sorter is a compelling test because it connects sensing, interpretation, and action in one short interaction. The question worth carrying forward is where that interaction turns a measurement into a commitment. For this research request, I located the paper’s bibliographic listing and examined relevant primary studies and an official implementation. Those sources explain the technical context, but they do not establish the focal paper’s methods or findings. Please upload the PDF or provide an accessible manuscript copy so the requested paper-specific analysis can be completed without inventing results.