Revised on 2026-10-09: Section 3 is organized by policy architecture, spatial representation, action interfaces, task progress, agent orchestration, prediction, and training mechanisms. This revision expands Section 3.8 with RL mechanisms, reward design, PPO / GRPO, and same-backbone gain evidence, with key conclusions linked to the original papers. Detailed readings are available in VLN Papers: Instruction Following and VLN Papers: Goal and Agentic Navigation. See Section 9.4 for coverage.

Reading guide

This survey addresses three questions: what tasks VLN solves, how models navigate, and how to establish that an improvement is effective. Follow the sequence of task settings → state and action interfaces → training signals → closed-loop evaluation. Consult the dataset catalog and simulator details once your experimental objective is clear.

Keep four distinctions in mind: reaching the destination and following the route instructions are different abilities; continuous environments and full robot dynamics are different levels of simulation; caching computational state and remembering spatial experience serve different purposes; and inference speed must be evaluated within the complete control loop.

1. Introduction

Vision-Language Navigation (VLN) studies how an agent uses natural language to localize itself, choose a route, and execute actions from a changing stream of egocentric observations. It is a partially observable, long-horizon problem: the agent must remember where it has been, track instruction progress, repeatedly align local observations with the overall goal, and recover from deviations, occlusions, or collisions.

In 2018, R2R established the classic setting of natural-language instructions, scanned real indoor environments, and generalization to unseen scenes. Research has since expanded from discrete panoramic navigation to continuous control, multilingual instructions, goal search, dialogue, dynamic crowds, aerial navigation, and real robots. Since 2024, VLM/VLA foundation models have further shifted the focus toward organizing general vision-language knowledge, spatial memory, high-level reasoning, and low-level control into embodied systems that run in real time.

As of 2026, the evidence is better described as a convergence of end-to-end models and modular systems. Unified models provide scalable general capabilities, while structured maps, fast-slow hierarchies, skill invocation, and safety control constrain long-horizon behavior. This survey uses that perspective to revisit VLN’s conceptual boundaries, methodological development, benchmarks, and open problems.

2. VLN fundamentals

VLN requires sustained language grounding, localization, historical memory, action selection, and stopping decisions under partial observability. Visual input and instruction progress change with every movement, making navigation a closed-loop process rather than a single visual question-answering step.

2.1 Task definition

Given a natural-language instruction $I$, an agent receives observation $o_t$ at time $t$ and predicts action $a_t$ using its historical state $h_t$:

flowchart LR
    I["Natural-language instruction"] --> G["Language-scene alignment"]
    O["Current visual observation"] --> G
    H["Trajectory history and spatial memory"] --> G
    G --> D["Next-step decision"]
    D --> S{"Should the agent stop?"}
    S -->|No| A["Action or local goal"]
    A --> E["Environment changes"]
    E --> O
    E --> H
    S -->|Yes| Z["End trajectory and evaluate success"]

    style I fill:#e7f5ff,stroke:#1971c2,stroke-width:2px
    style O fill:#d3f9d8,stroke:#2f9e44,stroke-width:2px
    style H fill:#fff4e6,stroke:#e67700,stroke-width:2px
    style G fill:#e5dbff,stroke:#5f3dc4,stroke-width:2px
    style D fill:#ffe3e3,stroke:#c92a2a,stroke-width:2px
    style A fill:#ffe8cc,stroke:#d9480f,stroke-width:2px
    style Z fill:#c5f6fa,stroke:#0c8599,stroke-width:2px

In discrete environments, the output is usually a neighboring viewpoint or STOP. In continuous environments, it may be a forward/turn action, a 2D waypoint, a trajectory segment, or low-level controls. These interfaces change perceptual coverage, control difficulty, and success conditions. Sharing R2R instructions alone does not make results directly comparable.

2.2 Reading a VLN benchmark

2.2.1 VLN, ObjectNav and VLA

ObjectNav can supply semantic exploration modules for VLN, and VLA can provide a policy backbone. Their results nevertheless cannot automatically be included in standard VLN leaderboards. Fair comparison requires matching task inputs, sensors, action interfaces, data splits, and additional training data.

2.3 A modern VLN system

Early models often described VLN as a visual encoder, a language encoder, and an action classifier. A more useful systems view in 2026 is: perception supplies semantic and geometric evidence; the state layer maintains instruction progress and spatial memory; planning selects subgoals; execution converts them into safe actions; and actual observations continuously correct errors.

flowchart TB
    subgraph input["Inputs and observations"]
        I["Natural-language instruction"]
        V["Egocentric visual stream"]
        R["Depth, odometry, robot state"]
    end

    subgraph cognition["Semantic and spatial state"]
        P["Vision-language perception"]
        M["Topological graph, BEV, memory cache"]
        T["Instruction progress and failure state"]
    end

    subgraph decision["Planning and decisions"]
        Q["Subgoal decomposition and candidate generation"]
        W["Candidate evaluation and replanning"]
    end

    subgraph control["Actions and control"]
        L["Local waypoint or trajectory policy"]
        C["Controller and safety constraints"]
        A["Robot actions"]
    end

    I --> P
    V --> P
    R --> M
    P --> M
    P --> T
    M --> Q
    T --> Q
    Q --> W
    W --> L
    L --> C
    C --> A
    A -.->|New observation| V
    A -.->|Pose update| R
    C -.->|Collision or immobilization| T

    style P fill:#d3f9d8,stroke:#2f9e44,stroke-width:2px
    style M fill:#fff4e6,stroke:#e67700,stroke-width:2px
    style T fill:#fff4e6,stroke:#e67700,stroke-width:2px
    style Q fill:#e5dbff,stroke:#5f3dc4,stroke-width:2px
    style W fill:#ffe3e3,stroke:#c92a2a,stroke-width:2px
    style L fill:#ffe8cc,stroke:#d9480f,stroke-width:2px
    style A fill:#c5f6fa,stroke:#0c8599,stroke-width:2px

This diagram does not require every method to implement each module explicitly. End-to-end models can fold several boxes into a unified network; dual-system, map-based, and agent-based approaches expose some interfaces explicitly. Assess architectures through their information flow and training objectives, not just the names used in a paper.

2.4 Core challenges

2.5 Toward navigation foundation models

%%{init: {"themeVariables": {"fontSize": "20px"}, "flowchart": {"nodeSpacing": 24, "rankSpacing": 28}}}%%
flowchart TB
    A["2018–2019<br/>Sequence modeling<br/>Seq2Seq · Speaker-Follower"]
    B["2020–2021<br/>Cross-modal pretraining<br/>PREVALENT · VLN-BERT · HAMT"]
    C["2021–2023<br/>Graph planning and long histories<br/>DUET · ETPNav · ScaleVLN"]
    D["2023–2024<br/>VLMs and video policies<br/>NaVid · NavGPT-2"]
    E["2025<br/>Fast-slow systems and data scaling<br/>StreamVLN · DualVLN<br/>NavFoM · OmniNav"]
    F["2026<br/>Agents and unified navigation under exploration<br/>AgentVLN · Qwen-RobotNav<br/>NavWAM"]

    A --> B --> C --> D --> E --> F

    style A fill:#f8f9fa,stroke:#868e96,stroke-width:2px
    style B fill:#e7f5ff,stroke:#1971c2,stroke-width:2px
    style C fill:#fff4e6,stroke:#e67700,stroke-width:2px
    style D fill:#e5dbff,stroke:#5f3dc4,stroke-width:2px
    style E fill:#ffe8cc,stroke:#d9480f,stroke-width:2px
    style F fill:#c5f6fa,stroke:#0c8599,stroke-width:2px

This progression involves several problems that keep converging: pretraining supplies semantic priors, spatial representation and memory support long-term state, hierarchical control coordinates planning and execution, and progress tracking and active perception help check the instruction continuously. Agent orchestration, corrective training, and future prediction can also be combined into these systems, improving capability invocation, error recovery, and action decisions respectively.

2.6 A map of research questions

2.7 Where recent research converges

Recent work converges on unified policies and hierarchical control, and also on geometric representations, historical memory, spatial action interfaces, progress tracking, and active perception. Agent orchestration, predictive learning, and closed-loop training can act on many architectures. Section 3 therefore organizes methods by research question, and one paper may illustrate several mechanisms. This organization is this survey’s analytical framework, not a mutually exclusive classification, and it does not imply that these ideas originated in 2026. Section 9.3 discusses the limits of the evidence.

3. Methods and key mechanisms

Understanding a VLN method requires separate answers to five questions: how the policy is organized, how history is represented, how decisions become actions, how instruction progress is tracked, and what feedback the model learns from. This section discusses unified policies, hierarchical control, spatial memory, task orchestration, and future prediction along these questions, and treats action interfaces, active perception, and training mechanisms on their own. One method can adopt several of these designs at once; the subsections are not mutually exclusive paper categories and do not imply a performance ranking under matched conditions.

The diagram below shows how these responsibilities relate along the information flow. Spatial memory, progress tracking, and future prediction can be implemented inside one unified network or as separate modules; agent orchestration is an optional upper layer.

flowchart TB
    I["Instruction and current observation"] --> S["Spatial representation and historical memory"]
    I --> P["Instruction progress and evidence judgment"]
    S --> D["Policy decision: unified policy or hierarchical planning"]
    P --> D
    D --> A["Action interface: primitive actions, pixels, waypoints, trajectories"]
    A --> C["Local execution and control"]
    C --> E["Environment and new observation"]
    E -.->|Feedback and correction| I
    P -.->|More evidence needed| O["Active observation"]
    O --> E
    G["Optional agent: task orchestration and tool calls"] -.-> D
    E -.->|Execution results| G
    W["Optional future prediction"] -.->|Assists runtime decisions| D
    T["Training: demonstrations, data scaling, correction, RL, predictive supervision"] -.->|Learns policy and representations| D

    style S fill:#fff4e6,stroke:#e67700,stroke-width:2px
    style P fill:#e7f5ff,stroke:#1971c2,stroke-width:2px
    style D fill:#e5dbff,stroke:#5f3dc4,stroke-width:2px
    style A fill:#ffe3e3,stroke:#c92a2a,stroke-width:2px
    style G fill:#ffe8cc,stroke:#d9480f,stroke-width:2px
    style W fill:#f3d9fa,stroke:#862e9c,stroke-width:2px
    style E fill:#c5f6fa,stroke:#0c8599,stroke-width:2px

3.1 How methods evolved

These focuses overlap in time: graph structures can still serve VLM policies, unified models can use spatial memory, and reinforcement learning can optimize different action interfaces. Gains in modern methods also often come with larger backbones, more trajectories, or extra sensors, so attribute mechanism changes separately from added resources when reading a paper.

3.2 Policy architecture

Architecture first decides what each layer is responsible for. A unified policy combines language, observations, and history in one backbone and predicts the next action or trajectory; a hierarchical system passes subgoals explicitly and lets another layer execute them locally. The number of models, whether they are trained jointly, whether a map is maintained, and whether high- and low-frequency roles are separated are different properties. A unified policy may still need a trajectory tracker, and a hierarchical network may be optimized jointly; “end-to-end” should state which interface the supervision or gradient reaches.

3.2.1 Unified policies

NaVid and StreamVLN use video history and language to generate actions; NavFoM, Qwen-RobotNav, and related work further explore multitask shared policies. For this class of methods, the key questions are how inputs and outputs are unified and whether the training data support transfer, not whether the model avoids every external module.

3.2.2 Hierarchical control

A fast-slow system is one implementation of hierarchical control: the slow system interprets the instruction at low frequency and produces subgoals, while the fast system keeps generating local actions from the latest observations. The lower layer can be a learned policy or a geometric planner; check whether it executes in closed loop, when it requests a new goal, and how upper-layer latency and errors propagate.

flowchart LR
    I["Instruction and history"] --> S2["High level: progress judgment and subgoals"]
    S2 --> G["Pixel goals, waypoints, sub-instructions"]
    O["Current observation and robot state"] --> S1["Low level: policy or geometric planner"]
    G --> S1
    S1 --> T["Local actions or trajectories"]
    T --> C["Tracking control and robot execution"]
    C -.->|New observation| O
    C -.->|Done, blocked, off route| S2

    style S2 fill:#e5dbff,stroke:#5f3dc4,stroke-width:2px
    style G fill:#fff4e6,stroke:#e67700,stroke-width:2px
    style S1 fill:#d3f9d8,stroke:#2f9e44,stroke-width:2px
    style C fill:#c5f6fa,stroke:#0c8599,stroke-width:2px

DualVLN, OmniNav, ABot-N1, and Robostral Navigate offer different examples of dividing planning and execution. Route2Step separates semantic progress from action generation, a reminder to distinguish “layered responsibilities” from “layered control frequencies”: having two modules alone does not make a fast-slow system.

Mechanisms with the same name need to be read for what they actually do. StreamVLN’s SlowFast mainly describes the update rate of context and memory; DualVLN’s fast-slow division sits between goal reasoning and trajectory execution. The former should be evaluated on history retention and computational efficiency; the latter also on control frequency, trajectory smoothness, and asynchronous goal updates.

3.3 Spatial representation and memory

Spatial representation answers “how are objects and traversable regions in the environment related”, and historical memory answers “which past information should influence the current decision”. The two can be combined in a map, or shared among geometric features, hidden states, keyframes, or a retrieval store. Extracting 3D geometric features from RGB alone does not amount to building a global map that a planner can query.

Geometric fusion and history selection can be optimized together. AdaGeoVLN feeds features from different depths of a geometry model into the policy and keeps historical KV states according to instruction relevance, geometric confidence, and transition novelty. The example shows that compression is not only about reducing tokens: what is kept also changes the spatial reasoning available at the next step.

Computation reuse and task memory should be distinguished by purpose, not only by whether a KV cache is used. VLN-Cache mainly reuses cross-frame computation to reduce cost; JanusVLN, AdaGeoVLN, and similar methods also use historical neural states as semantic or geometric memory. The former should report speedup and accuracy loss; the latter also needs tests of long-range landmark retention, backtracking, and error correction.

Maps and memory can both be wrong. Evaluations should state how information is written, updated, reconciled when conflicting, and forgotten; when a memory method from goal navigation is used as a reference, the task inputs and cross-episode information rights must also be fixed.

3.4 Actions and spatial grounding

The action interface decides what the high-level model must predict and what work remains for the executor. “Turn left”, “walk to the door in the image”, “reach a 3D waypoint”, and “a trackable trajectory segment” differ in precision, time horizon, and geometric requirements. They cannot be distinguished merely by whether the output is text or numbers.

Visual grounding affects both reasoning and execution. GroundingVLN binds visual evidence in its reasoning to image locations, then outputs pixel goals aligned with task progress for a geometric planner to execute. A measurable intermediate interface makes it possible to check separately “what was recognized”, “where the target is”, and “where the agent actually went”, and it also grounds execution-aware rewards.

Tokenization is not the same as primitive discrete control. LightNav-0 uses a unified token interface for pointing and quantized trajectories; its action tokens decode into trajectories and should not be conflated with an action class such as “turn left once”. Conversely, a pixel goal need not be paired with a neural continuous action head: a geometric planner can also turn it into motion.

When comparing interfaces, fix the available RGB, depth, pose, and map, and report closed-loop results for the complete execution chain. A model that outputs discrete symbols can still be evaluated in continuous environments, and a model that outputs continuous coordinates does not automatically satisfy the robot’s dynamic constraints.

3.5 Progress, active perception and recovery

Long-horizon navigation requires continual judgment of “which part of the instruction is done and what evidence is still missing for the next step”. Choosing the wrong active sub-instruction and failing to execute the right one are different errors; a single corrective action may not repair the internal progress state.

Route2Step separates semantic progress from action generation through a step-level interface, so wrong progress can receive state supervision. The lesson is that a single deviation may first require correcting “what am I doing” and then “how should I move”, rather than leaving every problem to action prediction.

SeekVLN goes further into cases where the current view is insufficient for judging progress: the policy can either navigate directly or observe more, then continue with the added evidence. Its training compares the subsequent progress of looking again versus navigating directly from the same state. The extra observation cost must be reported separately; the gain from adding views on demand cannot all be attributed to stronger reasoning, and the mode switch alone does not make the method tool orchestration or fast-slow control.

For example, the instruction is “pass the sofa, turn left, and stop before the second door”. When the side door is not visible, an additional observation may help most; when the junction has already been missed, the agent must back up and correct its progress; when it keeps moving after arriving, the stopping condition needs analysis. Count these three error types separately, then check whether recovery actually preserves the instruction’s constraints along the route.

Recovery is a capability, and more than one mechanism can provide it. CorrectNav generates corrective data from the model’s own deviations and continues training, which is a training mechanism; agent systems can also recover at runtime using tool feedback. Both should be verified with actual trajectories: the presence of “reflection” text does not establish effective recovery or imply weight updates during deployment.

3.6 Agents and tool execution

This section limits agent orchestration to an upper layer that selects and calls perception, memory, planning, or navigation capabilities based on task state, and reads execution results to update later decisions. It can call a unified navigation policy or connect to a hierarchical control system. Evaluation should separate “what the tool itself can do” from “what the upper layer gains by deciding when to call it”.

HarnessVLN puts perception, memory, and execution tools behind a shared interface and checks evidence, geometric feasibility, and subgoal consistency before dispatch. The example shows that agent orchestration and spatial memory often coexist; filing a paper under only one of them loses the other half of its design.

3.6.1 Evaluating backbone vs. agent

Qwen-RobotNav is a navigation model that can be used in agent systems. Evaluation should distinguish the underlying navigation results from system-level results with orchestration. The former reflect the model and its input configuration; the latter also include task decomposition, mode switching, and runtime feedback. The full system’s gains cannot all be attributed to either the backbone or the planner.

3.6.2 An inspectable agent loop

This survey’s analytical framework is: task state → select subtask and tools → execute → read results → update task state. Each transition should be inspectable in logs: was a tool failure recognized, did the strategy change, were ineffective actions repeated, and when did execution terminate? The diagram below illustrates system responsibilities and does not require every paper to implement identically named modules.

sequenceDiagram
    participant Planner as Task planner
    participant Memory as State and memory
    participant Navigator as Navigation policy
    participant Env as Environment and actuators
    Planner->>Memory: Query completed subtasks and explored regions
    Memory-->>Planner: Return state and supporting observations
    Planner->>Navigator: Issue current subgoal and constraints
    Navigator->>Env: Execute action or local trajectory
    Env-->>Navigator: New observation, pose, or execution feedback
    Navigator-->>Planner: Subgoal progress and failure evidence
    alt Evidence supports subgoal completion
        Planner->>Memory: Record completion and key observations
    else Incomplete or execution blocked
        Planner->>Memory: Record attempt and failure conditions
        Planner->>Navigator: Adjust subgoal or retry strategy
    end

For example, “go through the corridor to the kitchen and find a cup” requires at least two checks: reaching the kitchen and confirming the cup. Detecting a cup does not establish the room condition; logs must connect actual observations, tool calls, and subsequent trajectories to locate failures in task decomposition, spatial memory, or execution.

3.6.3 Orchestration gains and costs

Compare a fixed execution pipeline with adaptive orchestration under the same underlying skills, models, and budgets, and report task success, recovery rate, tool calls, latency, and human interventions separately. If more retries yield higher success, present the cost alongside it. Whether the navigation model was trained should also be recorded separately; an agent is not the same as a training-free system.

3.7 Predictive learning and world models

Prediction mechanisms should first be distinguished by what is predicted, whether prediction happens in training or at execution time, and how the prediction affects actions. Future images, latent features, and values can all provide supervision or decision evidence, but auxiliary prediction losses, runtime imagination for planning, and joint world-action generation differ in cost and evidence requirements.

PROSPECT predicts next-step semantic and spatial latent features, and the prediction branch is removed at inference. It is therefore predictive representation learning, and cannot be cited as unrolling candidate futures or searching actions at runtime. Its “no extra prediction overhead” applies only to the removed branch; the full system still includes perception fusion and streaming-policy computation.

flowchart LR
    subgraph train["Predictive supervision during training"]
        H["Observations and history"] --> Z["Policy representation"]
        Z --> A["Action prediction"]
        Z --> F["Future feature prediction"]
        T["Teacher features from later real observations"] -.->|Supervision| F
    end
    subgraph run["Prediction in runtime decisions"]
        O["Current observation and goal"] --> W["Prediction and decision module"]
        W --> P["Action or trajectory"]
        P --> R["Environment execution"]
        R -.->|Correction from real observations| O
    end

    style Z fill:#e5dbff,stroke:#5f3dc4,stroke-width:2px
    style F fill:#f3d9fa,stroke:#862e9c,stroke-width:2px
    style W fill:#f3d9fa,stroke:#862e9c,stroke-width:2px
    style R fill:#c5f6fa,stroke:#0c8599,stroke-width:2px

The two sides illustrate different uses that can be adopted separately or combined. The prediction and decision module on the right may use candidate evaluation or joint generation; not every method performs explicit tree search.

A name cannot replace checking the direction of prediction. Dual-Anchoring’s landmark auxiliary branch reconstructs historical evidence, mainly to constrain memory; even though it uses the term world model, it should not be classified directly as prospective planning. Semantic cues generated by visual imagination are also not automatically equivalent to action-conditioned environment dynamics.

Task boundaries and closed-loop gains still need separate confirmation. WorldVLN targets aerial language navigation, and NavWAM evaluates image-goal navigation. The latter and NWM can serve as references for prediction-control mechanisms, but cannot be merged directly into route-instruction VLN leaderboards. Prediction quality should be interpreted together with SR / SPL, collision rate, long-horizon error, and inference cost; realistic generated frames alone do not demonstrate better navigation.

3.8 Data scaling and closed-loop training

Architectures and mechanisms decide how a system uses information; training paradigms decide which feedback it learns from. The two should be described separately: unified policies, hierarchical control, and memory-equipped models can all use demonstrations, data augmentation, or reinforcement learning, and the same model may first be supervised fine-tuned, then collect corrective samples or undergo RL post-training.

3.8.1 From demonstrations to closed-loop RL

Corrective data must distinguish semantic-state errors from execution errors. CorrectNav builds perception and action correction data from deviated trajectories; Route2Step provides state supervision for wrong progress and uses direct action supervision for the corresponding recovery cases. What matters is which states the model learns on and what supervision it receives, which the final architecture name alone cannot explain.

Behavior cloning / SFT learns “what to do next in states the expert passed through”. At test time, a single under-turned corner changes the following views, and the policy may enter states the demonstrations never covered. DAgger lets the policy visit its own states and then queries the expert for corrective actions, easing this distribution shift; closed-loop RL instead optimizes the return of the whole task from the consequences of autonomous execution. The two can be combined, and interactive sampling alone does not determine whether training is imitation learning or reinforcement learning.

Two objectives show the difference in feedback sources:

Here $I$ is the instruction, $h_t$ is the observation and action history up to now, $a_t^*$ is the expert action, $\tau$ is the executed trajectory, $r_t$ is the reward, and $\gamma$ is the discount factor. The former raises the probability of expert actions; the latter raises the expected return of autonomous trajectories. When a task admits several valid routes, a reward can accept different paths that still follow the instruction; correspondingly, a reward that omits requirements along the route may encourage wrong shortcuts.

What RL is worth optimizing in VLN is the consequence of key decisions. Whether to look again or turn back after going off route, when to STOP near the goal, and whether to move temporarily away from the goal in order to turn correctly are hard to measure adequately with average action accuracy. Long-horizon credit assignment must link final success or failure to earlier decisions; process rewards, value estimates, and segmented returns provide different signals. When perception, depth scale, or the actuator itself is faulty, first confirm the system bottleneck; adding RL is not a universal fix.

This line of research predates large-model post-training: RCM rewards instruction-trajectory matching with a cross-modal matching critic, and EnvDrop combines environmental feature dropout, back-translation, behavior cloning, and Advantage Actor-Critic. The large-model era expanded policy scale and action-generation interfaces, but the contributions of data augmentation, memory design, and RL should still be analyzed separately.

3.8.2 Reward granularity

“Uses GRPO” describes only part of the optimizer and does not tell whether the model completes whole navigation episodes autonomously in the environment. The table below distinguishes representative methods by what is scored during training; this is a summary of the original mechanisms, and one method may combine several rewards.

Local answer rewards mainly ask “does this action segment match the expert”, execution-aware rewards ask “is this target traversable and where does execution end up”, and full closed-loop returns ask “after repeated observation and action, is the task completed”. All three may improve final SR, but they differ in training cost, coverage of error states, and the abilities they can demonstrate. In particular, under action optimization with a fixed history, later inputs do not change naturally with the currently sampled action, so it cannot be taken directly as evidence of long-horizon exploration.

Action granularity also affects learning: MacroAction-VLN performs closed-loop RL over topological frontier macro-actions, reducing the number of high-level decisions; SeekVLN assigns rewards according to the subsequent progress of additional observation relative to navigating directly. Geometric rewards do not require continuous coordinate outputs; what matters is that the relationship between the scored object, actual execution, and new observations is explicit.

3.8.3 Reward design

If the instruction is “go through the kitchen first, then follow the corridor into the bedroom”, a shortcut that bypasses the kitchen may earn higher endpoint success and shortest-path efficiency while violating the requirements along the route. Reaching the goal, following the instruction faithfully, and moving safely and efficiently are related but different optimization targets. A reward should first make the task requirements explicit, then choose metrics.

The original nDTW study measures route fidelity through ordered path matching, and SDTW adds a success constraint; see Section 8.3 for definitions. For example, ETP-R1’s R2R reward combines success@1.5m, SPL, and distance to goal, while for RxR it uses nDTW, SDTW, gSPL computed with the reference route length, and distance to goal. gSPL adjusts the efficiency reference, while consistency along the route still needs route terms as constraints; the 1.5m in the reward also cannot replace the success thresholds of other evaluation protocols. Original Eq.5

The following only shows terms a return can include; it is not any paper’s fixed recipe or a validated general set of weights:

Here $\tau^*$ is the reference route, $N_{\mathrm{collision}}$ is the number of collisions, and $T$ here denotes the number of execution steps; the weights control the relative influence of different goals, and dense signals such as distance progress can be issued per step separately. Long-horizon tasks should calibrate costs by accumulated totals over the whole episode, so that hundreds of step penalties do not overwhelm the success reward. Before adopting a reward, compare the term scores of expert, shortcut, detour, collision, and early-stop trajectories to confirm that their ordering matches the task requirements.

Keep the training information available to the reward separate from the information visible to the policy. Simulation can score with goal ground truth, navigation meshes, and reference routes, but the deployed actor should read only the observations, instructions, and memory the protocol allows. If a critic uses privileged state to aid training, state that separately. Otherwise, gains obtained from true goal coordinates cannot serve as evidence of navigating from vision and language alone.

3.8.4 PPO and GRPO

Policy gradients update the policy by raising the probability of high-advantage actions. Here the advantage means “how much better the subsequent return of this choice is than the baseline for the current conditions”; a core difference between PPO and GRPO is how that baseline is obtained.

PPO: a value estimate helps assign step-level credit. Common actor-critic implementations train a value function $V(I,h_t)$, combine it with returns to estimate each step’s advantage $A_t$, and then limit how far the probability ratio between the new and old policies can move. Its clipped objective is:

$\pi_{\mathrm{old}}$ is the behavior policy that sampled this batch of trajectories, and $\epsilon$ is the clipping range. The value baseline helps judge the contribution of early junction, backtracking, and stopping decisions, but also introduces value-estimation error. The critic can be a small value head on a shared representation rather than a copy of a same-sized model; PPO can also use terminal rewards and does not require dense process rewards. Original PPO paper

GRPO: multiple outcomes under the same condition form the baseline. Instead of training a separate critic, it samples a group of outputs for the same input and compares their rewards. If the scoring unit is a full navigation trajectory, the group should fix the instruction, start position, heading, and environment conditions. The common group-normalized advantage is:

Here $G$ is the group size and $\varepsilon$ is a small constant that prevents a zero denominator, distinct from PPO’s clipping range. The advantage is then applied to the actions or policy-generated tokens of the corresponding trajectory, with clipped updates and the corresponding reference constraint. This expression is the common form; NavGRPO uses Dr.GRPO, so its normalization and length aggregation cannot be copied wholesale. GRPO source: DeepSeekMath, NavGRPO algorithm description

Dropping the critic does not make total training cheaper. If the success reward is only 0 / 1, all-failure or all-success groups may carry no relative advantage; dense scoring, curricula, and hard-task supervision can change the signal, but each adds design cost. GRPO’s full-trajectory return also cannot automatically pinpoint the one turn that caused failure. A KL term to the reference policy and auxiliary BC / IL can preserve existing abilities; they are optional constraints and should be reported separately from the main task reward.

3.8.5 Closed-loop training in practice

For an existing navigation model, an experimental route that makes the source of gains easy to locate is: SFT / DAgger initialization → autonomous sampling → return and advantage computation → policy update → validation in unseen scenes. The diagram below illustrates the training flow, usable with PPO or full-trajectory GRPO; the specific scheme should still be chosen by action interface and sampling budget.

flowchart TB
    B["SFT or DAgger initialization"] --> S["Fix training tasks and behavior-policy version"]
    S --> P["Read instruction, current observation, and history"]
    P --> A["Sample action and record behavior probability"]
    A --> E["Execute in environment and return new observation"]
    E -->|Task continues| P
    E -->|Ended or truncated| R["Keep successes and failures, compute returns"]
    R --> V["PPO value advantage or GRPO group advantage"]
    V --> U["Rebuild sampled history and update policy"]
    U --> T["Independent validation: arrival, route, recovery, and cost"]
    T -.->|Next training round| S

    style B fill:#d3f9d8,stroke:#2f9e44,stroke-width:2px
    style E fill:#c5f6fa,stroke:#0c8599,stroke-width:2px
    style R fill:#fff4e6,stroke:#e67700,stroke-width:2px
    style V fill:#f3d9fa,stroke:#862e9c,stroke-width:2px
    style U fill:#e5dbff,stroke:#5f3dc4,stroke-width:2px
    style T fill:#e7f5ff,stroke:#1971c2,stroke-width:2px

Three details in implementing this flow can easily change conclusions:

1. Account for autonomous behavior and expert corrections separately. A transition after an expert takeover cannot be paired with the probability of the action the model originally proposed and had rejected. Corrective samples can go through an IL channel, while autonomous RL should keep failures, wrong STOPs, collisions, and timeouts rather than filtering for successful trajectories only; task termination and sampling truncation also need to be distinguished.
2. Sampled probabilities and recomputed probabilities must use the same history. For streaming models such as StreamVLN, store the observations, selected history frames, memory updates, and action tokens so the new and old policies are scored under the same conditions. After a parameter update, a KV cache produced by the old parameters cannot be treated as the exact history of the new policy; first check that probability replay agrees while the weights are unchanged.
3. Align generated action chunks with the steps actually executed. STOP or a collision may truncate an action chunk, and how the unexecuted suffix participates in optimization should be explicit. Long chunks reduce model calls, while short chunks give faster feedback from new observations; after changing chunk length or re-observation frequency, add a supervised baseline with the same interface to tell whether gains come from RL or from more decision opportunities.

These are implementation checks derived from the closed-loop objective and the policy probability ratio; they do not require using any particular project’s trainer. For model selection, PPO can serve as a step-level comparison with a value head and trajectory-level GRPO as a group-sampling comparison, with reward, action interface, and environment-interaction budget fixed. With only expert trajectories and no interaction, first confirm the rewards, transitions, and failure coverage that offline RL needs; trajectory preference optimization must keep the visual history each branch actually produced.

3.8.6 Attributing RL gains

Before-and-after comparisons within the same paper, backbone, and protocol come closer to answering “what does post-training add” than final scores across papers. All numbers below are reported by the authors and not independently reproduced; SR / SPL are percentages, and gains are absolute percentage points. Read each row only within its own protocol; rows that include joint training or specific reward components do not equal the gain from swapping the optimizer alone.

These results support that reward-driven post-training can improve VLN, but the gains depend on the feedback level, initialization, and task protocol. ActiveVLN’s single-turn / multi-turn comparison shows that the way interaction is organized deserves separate study; ETP-R1’s DAgger → RL gain reminds readers not to credit RL with the whole system’s improvement over older models. In the same JOP-VLN table, pure GRPO gives SR / SPL 67.5 / 62.8 versus 68.1 / 60.7 for pure IL, so success rate and efficiency did not improve together, and joint corrective supervision was more effective in that experiment. ActiveVLN ablation, ETP-R1 comparison, JOP-VLN Table II

Reproductions should also check reward definitions: ActiveVLN v1’s Eq.8 is written as a success indicator multiplied by endpoint distance, which contradicts in direction the textual explanation that closer is better, so it should not be copied directly into an implementation. NavGRPO’s PPO comparison uses step-level progress / heading rewards while the full GRPO method uses a different trajectory reward, so that comparison does not show that GRPO generally beats PPO; GroundingVLN’s GEAR gain should likewise be understood within its interface of visual grounding, pixel goals, and geometric execution. ActiveVLN Eq.8, NavGRPO §4.4, GroundingVLN §3.5 / Table 3

Beyond SR / SPL, evaluation should check nDTW / SDTW, wrong STOPs, off-route recovery rate, collisions, and task duration; evaluation in unseen scenes should be kept separate from adaptation protocols that allow test-time exploration. A rising reward with falling SR may indicate a problem with reward ordering, task difficulty, or the execution interface, so the training curve alone is not enough. Model size, data, sensors, action space, training interaction, and test-time inference budget should each be fixed or reported, and recovery ability needs controlled off-route experiments.

The companion article’s component adoption matrix can help check whether data, cameras, control interfaces, RL, and corrective data appear together. However, the matrix covers only entries on particular benchmarks under particular filters, and adoption rates do not equal causal contributions; model size, observation range, and extra training data still need item-by-item checks.

Organize ablations at three levels: first fix data and backbone to compare mechanisms, then fix the architecture to compare data and supervision, and finally compare complete systems under the same hardware and inference budget. Also state “zero-shot transfer after pretraining” and “no navigation-specific training” separately, so the two meanings of zero-shot are not mixed.

3.9 Choosing methods

3.9.1 Describing methods on several axes

The table below shows how to describe combinations starting from a paper’s mechanisms. “Evidence to verify” is a comparison proposed by this survey; it does not mean a common controlled experiment exists, and it is not an overall model ranking.

3.9.2 Choosing by failure type

The following are research recommendations based on system responsibilities. First confirm the bottleneck from failed trajectories, then run budget-constrained comparisons.

3.9.3 Four checks for new papers

1. Matched protocols: are sensors, action spaces, data splits, and success thresholds the same? Are on-demand observation, extra maps, and goal ground truth stated separately?
2. Matched resources: do gains come from the mechanism, or from a larger backbone, more data, or extra geometric priors? Are training samples, environment interaction, and inference budgets comparable?
3. Closed-loop benefits: do progress, memory, or prediction actually improve navigation, with corresponding ablations and failed trajectories?
4. Transparent deployment costs: are latency, GPU memory, control frequency, and real-robot tests reported for the complete execution chain?

The companion VLN Papers: Instruction Following maintains results by benchmark for R2R-CE, RxR-CE, and discrete R2R / REVERIE, separating trained, training-free, and validation-subset evaluations; the goal and agentic navigation collection maintains goal-navigation results. This survey explains mechanisms and evidence; for implementations, paper versions, and results under matched settings, continue with the companion readings.

4. VLN task types

Similar task names do not make results comparable. Building on the five dimensions in Section 2.2—goal specification, observations, actions, environment model, and interaction protocol—this section classifies common tasks by reasoning complexity, interaction, and action/physical realism, with representative benchmarks. Embodiment deserves particular attention: virtual agents without dynamics, wheeled robots, quadrupeds, humanoids, and drones face very different collision, fall, immobilization, and 6-DoF control challenges. Results under the same task name should not be compared directly across embodiments.

4.1 By reasoning complexity

1. Instruction-Following VLN

The agent navigates from a starting point to a destination using a supplied natural-language instruction. Open-ended goal search is not necessarily required, but landmark disambiguation, sequential reasoning, and long-term memory may still be necessary. The emphasis is on grounding language constraints along the route in the executed trajectory.

Representative datasets: Room-to-Room (R2R), Room-for-Room (R4R).

2. Reasoning-Oriented VLN

These tasks introduce target-object search or semantic constraints during navigation. The agent must reason about the correspondence between language instructions and environmental semantics to navigate and localize the target.

Representative datasets: REVERIE, SOON.

3. Long-Horizon VLN

These tasks emphasize long routes and composed instructions, requiring long-term planning, memory, and error recovery. They are important settings for evaluating sustained decision-making.

Representative datasets: LHPR-VLN and long-route settings in R4R / RxR.

4.2 By interaction

1. Non-interactive VLN

After receiving an initial instruction, the agent completes navigation independently without further interaction with the user. This is the most common VLN evaluation setting.

2. Interactive and dialog-based VLN

These tasks allow multiple interaction turns during navigation, using questions or feedback to refine the goal. They more closely resemble real human-robot collaboration.

Representative dataset: CVDN (distinguish supplied dialogue history from online questioning protocols).

4.3 By action and physical realism

1. Discrete panoramic navigation

The agent moves on a predefined graph of traversable viewpoints, typically with 360° panoramic features at each node. This deemphasizes local control and collisions, making it suitable for language-landmark alignment, global planning, and history modeling.

Representative benchmarks: R2R, R4R, RxR, REVERIE.

2. Continuous-environment navigation

The agent moves over a navigable surface through forward/turn actions or local waypoints, rather than teleporting along a manually constructed graph. Traversability and stopping control matter; field-of-view restrictions, localization noise, and dynamics depend on the specific configuration.

Representative benchmarks: R2R-CE, RxR-CE; other transferred settings must identify their implementation.

3. Physically embodied navigation

Robot dynamics, morphology, and control frequency impose constraints, including falls, slip, immobilization, and different camera mount positions. These tasks are closer to deployment but harder to align with classic VLN results.

Representative benchmarks: VLN-PE, VLNVerse, and real-robot evaluations.

Comparison principle: report R2R, R2R-CE, RxR-CE, ObjectNav, and real-robot tests in separate tables. Even when all use SR / SPL, their inputs, action spaces, and success conditions may differ.

5. VLN applications

Applications determine sensors, action spaces, map priors, and safety constraints, and therefore the appropriate datasets and simulators. The table compares three settings before examining each in detail.

5.1 Indoor environments

Indoor VLN focuses on homes, offices, and public buildings. Multiple rooms and extensive furniture provide the landmarks in instructions such as “go through the kitchen and stop beside the sofa,” placing strong demands on room-level topology, object-level grounding, and precise stopping. Indoor data and benchmarks are also the most mature: most datasets in Section 6 use real scans such as Matterport3D and HM3D, or synthetic scenes such as ProcTHOR and GRScenes.

Applications: domestic service robots, indoor logistics, hospital and shopping-center guidance, and quadruped / humanoid inspection.

5.2 Outdoor environments

Outdoor VLN faces larger spatial scales and greater uncertainty. Similar-looking landmarks may be hundreds of meters apart; lighting, weather, and traffic change continuously; GPS and map availability vary. Touchdown (Chen et al., CVPR 2019) studies language navigation and spatial localization in street scenes. StreetLearn provides a street-view learning environment and should not itself be equated with a route-instruction dataset. map2seq addresses map-assisted navigation instructions. This branch is increasingly connected to language-conditioned autonomous-driving planning and semantic navigation for outdoor service robots.

Applications: campus and urban delivery, guidance for visually impaired people and mobility assistance, and language-conditioned autonomous-driving planning.

5.3 Aerial environments

Aerial VLN targets flying platforms such as multirotor drones. Compared with ground navigation, it requires controlling both horizontal position and altitude; viewpoints change continuously between downward and oblique views, and landmark appearance varies greatly with height. Flight constraints—no-fly zones, minimum altitude, and obstacle avoidance—must be addressed explicitly at the action layer. AerialVLN, CityNav, and OpenFly in Section 6.6 represent simulated cities, real aerial imagery, and large-scale multiengine data, respectively.

Applications: drone inspection and mapping, aerial search and rescue, urban low-altitude logistics, and language-directed aerial photography.

6. Major VLN datasets

Start benchmark selection with the capability to be tested, then consider scale. Scene assets, task data, and training augmentation data are different resources: Matterport3D supplies environments, R2R supplies instructions and paths, and ScaleVLN supplies synthetic training examples. A shared environment does not imply a shared action interface or evaluation protocol.

6.1 Dataset overview

The following table includes only scales with a clear counting unit. Years indicate first public release; conference years are noted separately. Paths, instructions, dialogues, and episodes are not interchangeable, and continuous-environment conversions cannot simply inherit the discrete version’s counts.

6.2 Instruction-following and continuous

6.2.1 R2R (Room-to-Room)

R2R pairs natural language with reference paths between start and goal viewpoints on navigation graphs in scanned Matterport3D environments. It supports landmark alignment and cross-scene generalization. Traversable graph edges already simplify local obstacle avoidance, so R2R success does not establish robot control ability. See the original paper and official simulator.

Understanding a sample: the instruction describes landmarks and turns; the reference path is a sequence of discrete viewpoints. Starting from a viewpoint and heading, the agent selects reachable neighbors and eventually stops. A path is not a frame-by-frame video, and different instructions for one path are not independent examples from different buildings.

Value and limitations: R2R is a common starting point for language alignment and planning. When interpreting Val-Unseen results, check whether training visual features or additional data involve test environments. Real deployment additionally requires validation under narrow fields of view, localization errors, and low-level execution.

6.2.2 R4R (Room-for-Room)

R4R concatenates R2R paths and instructions, exposing the difference between following the described route and reaching the endpoint directly. It supports research on route constraints and long-path following; use CLS / nDTW alongside SR. The original paper also introduced CLS.

Why longer reference routes matter: consider “pass through the dining room, then loop back to the living room,” where the endpoint is close to the start. Going straight to the living room may achieve endpoint success without following intermediate constraints. R4R makes this tension explicit: the shortest arrival path need not be the language-specified route.

Record endpoint success, coverage, and visit order together. Longer routes amplify accumulated errors, but a larger history window alone does not establish task-state reasoning. Inspect whether the model tracks completed route segments and landmarks still to be visited.

6.2.3 RxR (Room-across-Room)

RxR provides multilingual instructions and spatiotemporal annotations associated with observations. Specify language subsets, Guide / Follower usage, and test splits. Evaluation on English alone does not demonstrate multilingual ability. See the official data and field documentation.

Annotation process: a Guide describes a route, and a Follower follows the description. Their trajectories can differ, supporting both reference-path imitation and analysis of ambiguity and human following errors. Word timings and camera poses provide finer vision-language alignment cues. See the official annotation documentation.

6.2.4 VLN-CE

VLN-CE moves instruction navigation into Habitat’s navigable space, removing movement constraints imposed by graph edges. Environment positions are continuous; policy actions may still form a finite set, such as forward and turn. Execution becomes harder, but legged dynamics are not automatically included. See the original paper.

The original baseline repository recommends R2R_VLNCE_v1-3 and pins its Habitat version. Different sensors, waypoint controllers, or modified simulators require protocol checks. An identical dataset name alone does not establish comparability.

Data and code: official VLN-CE repository · R2R-CE data download and directory layout · data format and version history. The download entry provides the R2R_VLNCE_v1-3 base package and the preprocessed package.

From selecting the right node to reaching the location: a valid neighbor in a discrete graph generally permits direct transfer. In continuous space, even a correctly selected doorway can lead to deviation, immobilization, or overshooting the stopping point. Record waypoint predictors, controllers, and navigation policies separately rather than attributing whole-system gains to the language model.

{
  "episode_id": 1,
  "trajectory_id": 4,
  "scene_id": "mp3d/SCENE/SCENE.glb",
  "instruction": {"instruction_text": "Walk forward to the doorway."},
  "start_position": [0.0, 0.0, 0.0],
  "start_rotation": [0.0, 0.0, 0.0, 1.0],
  "goals": [{"position": [0.0, 0.0, -4.0], "radius": 3.0}],
  "reference_path": [[0.0, 0.0, 0.0], [0.0, 0.0, -4.0]]
}

6.2.5 RxR-CE

RxR’s continuous-environment adaptation is released with VLN-CE. The RxR-Habitat challenge explicitly constrains RGB-D observations, action step lengths, and turn angles. Panoramic-observation results cannot directly be treated as results under that challenge protocol. See the official required task configurations.

Data and code: official VLN-CE repository · RxR-CE data download and directory layout. RxR-CE shares this repository with R2R-CE; this entry provides the RxR_VLNCE_v0 data package and instructions for downloading text features.

Separate multilingual effects from execution errors: a language-specific drop can reflect semantic understanding or action errors on longer routes. Group results by language, route length, and endpoint error, using the same cameras and controllers. If machine translation or extra text features are used, clarify whether the model understands the original language directly or navigates through a translation system.

6.2.6 VLN-PE

VLN-PE supports humanoid, quadruped, and wheeled robots, studying embodiment gaps caused by camera viewpoints, lighting, collisions, and falls. Physical realism here means simulation that better reflects robot constraints, not that all data come from real robots. Report results by morphology and controller. See paper v2.

New variables introduced by embodiment: the same semantic subgoal may not be executable by every robot. Camera height changes visible landmarks, body dimensions change doorway clearance, and motion controllers affect tracking and stability. Distinguish understanding the destination but failing to reach it from misunderstanding the goal.

Retain three log layers: high-level goals and waypoints, actual low-level trajectories, and collisions or abnormal poses. Cross-robot comparisons should report controllers and failure categories alongside SR. Simultaneously changing camera, controller, and policy does not isolate morphological generalization.

6.2.7 ScaleVLN

ScaleVLN expands training coverage rather than introducing a new task to rank alongside R2R. It shows that more environments and synthetic supervision can substantially change existing models’ performance. Architecture comparisons must disclose such additional data. See the original paper.

What data scaling changes: more scenes broaden visual and geometric coverage; more synthetic instructions add supervised examples. Discuss these effects separately. Synthetic language may repeat templates, and path sampling may favor easy regions, so more samples do not automatically mean richer tasks.

With the same model, compare original data, additional scenes, additional instructions, and full augmentation. Also report training steps or compute budgets to distinguish coverage, language variation, and extra optimization as sources of improvement.

6.2.8 InternData-N1

InternData-N1 unifies VLN-CE, VLN-PE, and VLN-N1 subsets in LeRobot format. A shared storage format helps training but does not erase differences in observations, actions, or success criteria. Record subsets, filtering, and sampling ratios. See the official data card.

Shared storage still requires aligned semantics: videos, states, and actions may share storage interfaces while differing in action scales, camera settings, and robot states. Specify action normalization, task conditioning, and how sampling prevents large subsets from overwhelming smaller ones.

vln_ce/
├── raw_data/                 # Original task annotations
└── traj_data/
    └── <scene-dataset>/<scene>/
        ├── data/chunk-000/   # Episode Parquet data
        ├── meta/            # Metadata: info, tasks, episodes, etc.
        └── videos/          # Observations by camera and modality

6.2.9 VLNVerse

VLNVerse brings multiple tasks, embodied simulation, and evaluation into a shared framework. It supports cross-task evaluation, but a common platform does not justify collapsing all tasks into one SR. Report each task and embodiment setting rather than only an overall mean. See the original paper.

Reading a unified benchmark: common sensors and actuators may support tasks whose success means arrival, target confirmation, or multistage completion. Report a task × embodiment result matrix, indicating shared weights versus task-specific fine-tuning. An aggregate mean alone can conceal failure on particular tasks.

6.3 Goal-oriented and long-horizon

6.3.1 REVERIE

REVERIE requires navigation from a high-level description and identification of a target object. Reaching a viewpoint where the target is visible differs from correct object grounding. Separate navigation metrics from remote-grounding metrics. See the original paper.

The original paper reports 10,567 panoramic viewpoints in 90 buildings, 4,140 target objects, and 21,702 crowdsourced instructions averaging 18 words. Instructions generally describe the target region and object, rather than the route step by step. Navigation success requires stopping at a viewpoint that can observe the target (objects within 3 m are considered visible). The original appendix defines grounding success by predicted-box / ground-truth-box IoU ≥ 0.5. Later work commonly selects from supplied object candidates and reports RGS (Remote Grounding Success, the proportion selecting the correct target) and path-length-weighted RGSPL. Check both the grounding criterion and candidate source when comparing results.

Two stages, two error types: for “go to the upstairs bedroom and find the lamp by the window,” the agent must first reach a location where the target can be observed, then select the described instance. Reaching the right room but selecting the wrong lamp is a grounding error; never reaching an observable region calls for search and navigation diagnosis.

Object detectors, candidate boxes, and object features are therefore part of the experimental setting. If one policy receives better object candidates, differences in final grounding success cannot all be attributed to planning.

6.3.2 REVERIE-CE

REVERIE-CE refers to a family of continuous-environment adaptations. This survey has not identified a unified official protocol with fixed versions and scale maintained by the original authors, comparable to VLN-CE, so it gives no release year or sample count. Cite the specific paper, conversion scripts, filtering, and object-success criterion; do not reuse discrete REVERIE counts.

Continuous conversion requires at least mapping discrete targets into continuous scenes, specifying locations from which targets can be observed, and defining how object predictions are submitted after stopping. Report how many original samples were filtered and why. An unchecked conversion cannot be assumed to preserve difficulty or target distributions.

6.3.3 SOON

SOON uses descriptions of the target and surrounding scene to search and localize from different starting points. It shifts from route following toward scene-conditioned exploration, supporting semantic search and graph planning. Do not conflate descriptions, targets, and start-point combinations in its FAO data into one sample count. See the original paper.

Difference from route following: route instructions explain how to get there; scene descriptions emphasize where and what to search for. Without stepwise route supervision, the system must choose search regions, decide when to abandon a hypothesis, and update its direction from new observations.

Separately count failure to find the target region, failure to recognize a visible target, and failure to stop after recognition. This distinguishes the value of semantic priors and reveals how exploration budgets are spent.

6.3.4 LHPR-VLN

LHPR-VLN focuses on planning consistency across sequential subtasks. Alongside overall completion, ISR, CSR, and CGT analyze subtask success and the effects of preceding failures. Unlike R4R, multistage execution requires task-state tracking; total path length alone cannot characterize difficulty. See paper v3.

Long-horizon failures propagate: stopping at the wrong location changes the next stage’s starting state. Resetting and testing each subtask independently hides this accumulation. Retain both independent-subtask and continuous-execution evaluations, recording the first failed stage, completed progress, and recovery cost.

Validate memory around task state: does the model remember completed goals, revisit ruled-out regions, or incorrectly mark failed subtasks complete? Such evidence says more about long-horizon ability than a longer context window alone.

6.4 Dialog and social navigation

6.4.1 CVDN

CVDN collects human Navigator–Oracle dialogues. Navigation from Dialog History continues navigation using an existing dialogue history. Using dialogue history does not imply permission to ask a human for help online; active-questioning evaluation needs a separately specified interaction protocol. See the original paper.

History understanding and active interaction are different problems: a historical reply such as “turn left at that doorway” must be aligned with the traveled path. Asking in real time “the red door or the white door?” additionally requires deciding when to ask, how to use the answer, and how to account for interaction costs. These settings provide different information and need separate results.

For active help-seeking, report question counts, useful-answer rates, and navigation improvements after each interaction. For NDH alone, analyze history length, reference resolution, and completion of the next route segment.

6.4.2 TEACh

TEACh includes dialogue, navigation, and object interaction, making it a broader embodied-task dataset. Derived tasks such as EDH have distinct inputs and completion conditions and should be discussed separately from pure navigation. Paper v3 also explains revised test-set evaluation targets.

Arrival alone is insufficient: “put a clean cup on the table” requires object localization, state assessment, and post-manipulation conditions. Reaching a cup does not complete the task.

EDH (Execution from Dialog History) continues execution from a supplied interaction history, whereas TfD (Trajectory from Dialog) infers and executes a task trajectory from dialogue. Their initial information differs. Identify the evaluation task before comparing task success and goal-condition completion. See the TEACh paper.

6.4.3 HA-VLN 2.0

HA-VLN 2.0 includes dynamic interactions with multiple people and personal-space constraints. Evaluate arrival and social compliance together, fixing crowd behavior and run seeds. One successful video does not establish reliability across repeated encounters. See paper v5.

Success in dynamic environments is conditional: waiting, detouring, and passing close to people may reach the same goal with different social consequences. SPL alone can penalize reasonable detours; interpret it alongside collisions, personal-space violations, and time limits, with waiting costs assessed through timing measures.

Use fixed or paired seeds and repeat tests across crowd-interaction scenarios. Report social violations in successful episodes, and whether failures end through timeout, collision, or getting lost. Do not show only the smoothest trajectory.

6.5 Demand-driven and commonsense

6.5.1 DDN

DDN infers objects that satisfy user demands, such as mapping “I need to clean” to a suitable functional target. The original work studies this in AI2-THOR / ProcTHOR as demand-conditioned goal navigation; it does not require stepwise route following. See the original paper.

A demand may admit multiple targets: several objects may satisfy the same demand, but acceptable answers depend on task annotations and the scene. “I want to sit down and rest” might suggest a chair or sofa; language commonsense alone cannot make every candidate count as success.

Diagnose demand inference and spatial search separately: is the proposed goal acceptable, available in the environment, and subsequently found with a correct stop? If demands are first rewritten as fixed object categories, disclose the additional model or supervision used.

6.6 Aerial and specialized

6.6.1 AerialVLN

AerialVLN uses UE4 / AirSim urban scenes, adding altitude and spatial-relation reasoning. Table 1 in the original paper lists the standard task as 4 DoF; it should not be described broadly as full 6-DoF flight control. See the original paper.

Altitude changes language references: “go around the rooftop and then descend” combines horizontal paths and vertical relations; landmark scales in downward views also change with height. Specify controllable degrees of freedom, camera orientation, and flight step sizes, and check dependence on building appearances specific to training cities.

Record horizontal error, altitude error, and stopping position separately. Proximity in a 2D map projection alone cannot establish 3D navigation success.

6.6.2 CityNav

CityNav uses real urban geographic environments and language-paired human demonstrations to study aerial navigation with visual and geographic information. Real-city data do not automatically mean autonomous flights by real drones. State whether geographic information and maps are available. See paper v3.

Maps are additional input: geographic coordinates, maps, and prior locations may substantially change search. Compare vision-language inputs, added geographic information, and added maps, and report city splits. Cross-city evaluation better tests transfer of spatial semantics, but asset and data overlap must still be controlled.

6.6.3 OpenFly

OpenFly combines multiple rendering engines, automated data-collection tools, and an aerial-navigation benchmark. Its contribution includes data production and environment coverage. Cross-engine generalization still requires separation of training and testing engines, scenes, and assets. See paper v7.

Cross-engine transfer changes more than appearance: rendering, coordinates, depth definitions, and action execution can differ. Align these interfaces first, then distinguish same-scene/different-renderer, same-engine/different-scene, and cross-engine/cross-scene experiments. After generating many trajectories, sample-check that language describes actually visible landmarks and the executed routes.

6.7 Data conventions for reproduction

Record original versions, scene lists, episode and unique-instruction counts, language subsets, Guide/Follower types, and filtering rules. State whether extra training data touch test buildings or derived assets. Unknown overlap in private pretraining data should be marked “not disclosed,” not assumed absent.

Dataset fields and directory details are expandable under their respective benchmarks. The following cross-dataset experiment-record template supplements them. Keep official formats, this survey’s illustrations, and actual runtime configurations separate. Use download and installation commands from the official repository for the selected version.

dataset:
  name: null
  revision: null
  split: null
  episode_count_after_filtering: null
  scene_list_hash: null
  languages: []
  extra_training_sources: []
environment:
  simulator_commit: null
  scene_assets_version: null
  robot_and_controller: null
  camera_modalities_and_fov: null
  action_space_and_units: null
  collision_and_sliding: null
evaluation:
  evaluator_commit: null
  distance_definition: null
  success_threshold_and_stop_rule: null
  max_steps_or_time: null
  trajectory_sampling: null
  seeds: []

7. Major VLN simulators

Simulators determine how observations are generated, actions executed, and success judged. Compare rendering, collisions, dynamics, task implementations, and assets separately. Photorealism or a rich physics engine alone cannot establish that a VLN protocol better reflects real deployment.

7.1 Matterport3D Simulator

Built around scanned panoramas and navigation graphs, Matterport3D Simulator supports reproduction of discrete tasks such as R2R, R4R, and RxR. Legal graph connections supply traversability constraints and therefore cannot establish real-robot obstacle avoidance. Follow the official repository and relevant data licenses for installation and image assets.

Mechanism and strengths: policies observe and move on a graph of scanned viewpoints, making visual inputs reproducible and classic benchmarks reusable. Precomputed visual features let research focus on language alignment, history modeling, and graph search.

Limitations and selection: transfers between viewpoints hide control along intermediate segments. High-level planning success does not establish passage through narrow doors or collision handling. Use it for a discrete-planning baseline; validate action-error questions separately in continuous environments.

7.2 Habitat

Habitat-Sim provides the simulation backend; Habitat-Lab organizes tasks, sensors, actions, and evaluation. Classic VLN-CE uses their navigation configuration. Interaction, physics, and human-robot coexistence features added later are not automatically present in older benchmarks. See Habitat-Sim and Habitat-Lab.

Match the paper’s specified version first. Engine upgrades can change collision and sliding behavior. Establish baseline results before reporting experiments with an upgraded engine as a separate configuration.

Mechanism and strengths: sensor observations, navigable space, and task evaluators form an experimental loop for RGB-D inputs, local waypoints, and continuous instruction execution. Pin assets, navigation meshes, and task code together.

Limitations and selection: a default navigation agent is not a robot with full dynamics. Camera height, field of view, turn angles, step length, and sliding affect results. Match the paper configuration before attributing differences to the model. Treat added physics or interaction as new experimental conditions.

7.3 Isaac Sim / Isaac Lab

Isaac Sim provides robot simulation and sensors; Isaac Lab adds robot-learning workflows. They suit explicit study of morphology, contact, and control, but require VLN data and tasks to be implemented or integrated separately. Official Windows / Linux installation paths are available; Python, drivers, and Isaac Sim versions must be compatible. See the official installation documentation.

Mechanism and strengths: robot bodies, joints, contacts, sensors, and controllers jointly determine execution, supporting study of how navigation goals meet real motion constraints. High-level navigation and low-level motion control can be analyzed separately.

Limitations and selection: asset import, collision geometry, inertia, and control interfaces affect results; high-quality rendering adds resource costs. Verify an observation-action loop for one robot in one scene before increasing parallel environments. Physics-step throughput is not multimodal-system throughput.

7.4 MuJoCo / MJX

MuJoCo supports contact dynamics and control research; MJX provides batched physics computation on accelerators. They can host a VLN system’s motion-execution layer, but do not automatically supply language instructions, indoor assets, or navigation evaluators. Measure physics throughput separately from training throughput including image rendering and VLM inference. See the MuJoCo / MJX documentation.

Mechanism and strengths: explicit motion-state and contact models support local-controller, trajectory-tracking, and stability evaluation. In hierarchical VLN, high-level goals or velocities can be passed to the motion layer, with actual arrival errors fed back upward.

Limitations and selection: a full VLN experiment additionally needs assets, language tasks, camera observations, and evaluators. MJX batching gains depend on models and hardware. Measure physics, rendering, and policy-inference times separately.

7.5 AI2-THOR

AI2-THOR emphasizes interactive objects and changing environment states, supporting combined navigation and manipulation. Its official README lists macOS / Ubuntu requirements. Unity’s cross-platform support does not establish native Windows support for the Python simulation stack. Follow the target version’s build and rendering-backend requirements. See the official repository.

Mechanism and strengths: interactive objects and states connect reaching an object with satisfying manipulation conditions in one environment. This supports study of navigation, object search, and task execution together.

Limitations and selection: visibility, reachability, and interactability are different states. Proximity may not satisfy interaction prerequisites. For TEACh and similar tasks, record navigation and manipulation failures, and fix object states and task initialization. Endpoint distance alone does not evaluate the whole task.

7.6 Gibson / iGibson

Gibson, iGibson, and OmniGibson are related but distinct: using scanned scene assets differs from using an interactive physics platform. For household tasks and object states, establish the exact platform, asset version, and behavior definitions. OmniGibson’s entry point is now the BEHAVIOR-1K project; see the separate iGibson repository.

Mechanism and strengths: interactive environments incorporate furniture, objects, and robot contact, supporting combined navigation and household manipulation. Available capabilities depend on whether Gibson, iGibson, or OmniGibson is used.

Limitations and selection: assets, dependencies, and task definitions are not interchangeable across these projects. Establish the paper’s platform, then verify object states and success conditions. Scene transfers require renewed checks of collision and interaction semantics.

7.7 AirSim

AirSim simulates drones and vehicles and underlies some aerial VLN projects. Pin Unreal / AirSim and project-scene versions for reproduction. Moving to a community fork requires revalidating controls and evaluation behavior; it cannot be assumed to preserve the environment. See the original repository.

Mechanism and strengths: flight state, cameras, and actions support aerial route execution, altitude changes, and city-scale navigation. In VLN, language data and success evaluators are typically supplied by the research project built above it.

Limitations and selection: pose or waypoint calls differ in difficulty from flight under low-level control constraints. Document teleportation, speed limits, collision termination, and timeouts, and pin scene maps and engine versions.

7.8 InternUtopia

InternUtopia organizes scenes, robots, and embodied tasks on Isaac Sim. At the time of review, its prerequisites listed Ubuntu 20.04 / 22.04 and Isaac Sim 4.5.0. Isaac Sim’s cross-platform support therefore does not establish Windows support for the entire InternUtopia stack. See the official prerequisites, accessed 2026-09-26.

Mechanism and strengths: a common environment framework integrates scenes, robots, and tasks, supporting comparison of high-level capabilities across embodiment settings. The selected task configuration and robot controller still determine each experiment.

Limitations and selection: framework support for a task family does not mean every robot-scene combination has been evaluated. Run the official minimal example to confirm reset, observations, execution, and success judgments before integrating your navigation policy.

7.9 Simulator comparison

7.10 Choosing a platform

7.10.1 By host OS

Check support for the complete research codebase before the underlying engine. Windows users can choose natively supported Isaac Sim / Isaac Lab; other projects should follow official Linux-host, container, or validated WSL configurations. Running Linux programs under WSL does not guarantee that EGL, Vulkan, CUDA, window systems, and simulation assets work without further setup.

7.10.2 By research task

Use the original task environment for reproduction, add morphology and contacts when studying dynamics, and add interactive objects when studying interaction. Tie each additional simulation capability to a research question and establish baselines for new variables. Avoid ranking platforms with fast/slow labels across unmatched hardware and scenes.

8. Metrics and evaluation

Navigation has at least four evaluation dimensions: goal completion, path cost, instruction fidelity, and execution reliability. Classic VLN has established metrics for the first three; the fourth needs robot- and scenario-specific definitions. No single score replaces all four.

Optimizing one dimension can conceal deterioration in another. SR alone may reward extensive trial-and-error exploration; SPL alone may reward shortcuts that ignore instructed detours; proximity at any point can hide poor stopping; contact-free simulation hides collisions, falls, and immobilization. Report metrics in groups that cover these dimensions.

flowchart TB
    Q["Four questions for one navigation episode"]

    subgraph D1["Goal completion"]
        SR["SR: endpoint success"]
        NE["NE: endpoint error"]
        OSR["OSR: ever entered the success region"]
    end

    subgraph D2["Path cost"]
        SPL["SPL: path-length-weighted success"]
        TL["TL: actual trajectory length"]
        ST["Steps or elapsed time"]
    end

    subgraph D3["Instruction fidelity"]
        CLS["CLS: coverage and length matching"]
        NDTW["nDTW: order-aware alignment"]
        SDTW["SDTW: success-weighted alignment"]
    end

    subgraph D4["Execution reliability"]
        CR["Collisions with environment or pedestrians"]
        FS["Falls and immobilization, e.g. VLN-PE FR / StR"]
        HI["Human interventions and end-to-end latency"]
    end

    Q --> D1
    Q --> D2
    Q --> D3
    Q --> D4

    style Q fill:#e7f5ff,stroke:#1971c2,stroke-width:2px
    style D1 fill:#f1f8ff,stroke:#339af0,stroke-width:2px
    style D2 fill:#f0fbf2,stroke:#40c057,stroke-width:2px
    style D3 fill:#fff8f0,stroke:#ff922b,stroke-width:2px
    style D4 fill:#fff5f5,stroke:#fa5252,stroke-width:2px

Definitions in the first three groups are relatively consistent across benchmarks. For the fourth, counting units, thresholds, and episode termination depend on the benchmark; see Section 8.4.

8.1 Protocols and metric reading

Record task and split, sensors and field of view, depth/pose/map access, action interface, stop and timeout rules, distance function, success threshold, extra training data, and inference budget. R2R graphs, Habitat navigable surfaces, and aerial scenes are different distance spaces.

8.1.1 Trajectories and metric responses

These are illustrations for understanding metrics, not results from a particular paper. Assume an instruction requires visiting landmarks before reaching the endpoint:

8.1.2 Choosing metrics by setting

Route following needs both endpoint and path measures; goal search needs search success and target verification; dialogue tasks must specify whether active questions are allowed; physical robots need execution-failure measures. Use each task’s official success definition rather than applying a universal 3 m endpoint threshold.

The diagram helps choose metric groups. Multiple branches may apply: referring-expression navigation in a physical environment needs both target recognition and execution-safety evaluation.

flowchart TB
    Q["Which capability must the task validate?"]
    Q --> G["Reach a specified location"]
    Q --> R["Follow the required route"]
    Q --> O["Identify or manipulate a target"]
    Q --> E["Physical and social execution"]
    G --> GM["SR, NE, OSR<br/>SPL for travel efficiency"]
    R --> RM["CLS for soft coverage<br/>nDTW / SDTW for ordered alignment"]
    O --> OM["Navigation + target grounding<br/>or task postconditions"]
    E --> EM["Arrival + collisions / falls / violations<br/>Time and control latency"]
    classDef metric fill:#e8f1fb,stroke:#426a91,color:#18354f;
    class GM,RM,OM,EM metric;

Retain per-episode results after selecting metrics. Overall means can hide substantial degradation on long routes, particular languages or robots, or specific failure conditions.

8.1.3 Core metric overview

8.1.4 Diagnosing metric gaps

These are hypotheses to test. Multiple mechanisms can produce the same discrepancy; it does not justify immediately concluding that a model lacks reasoning or merely memorizes.

8.2 Goal completion metrics

8.2.1 Success Rate (SR)

Let $S_i\in{0,1}$ be the official success indicator for episode $i$, over $N$ evaluation trajectories:

Success may require endpoint distance, an explicit stop, target visibility, or object recognition. Preserve these conditions in reporting. Whether timeout-truncated episodes count as success also depends on the evaluator.

8.2.2 Navigation Error (NE)

Let $x_i^{\mathrm{end}}$ be the terminal position, $g_i$ the goal, and $d$ the benchmark distance function:

Classic graph navigation generally uses shortest graph distance; continuous navigation commonly uses geodesic distance over navigable surfaces. Neither should universally be rewritten as Euclidean distance. If multiple goal positions are valid, explain how the goal set is handled. See the official RxR metric documentation.

8.2.3 Oracle Success Rate (OSR)

For tasks with a goal-distance threshold $\tau$, OSR checks whether the trajectory ever entered the success region:

Follow the official implementation for strict versus non-strict thresholds. OSR cannot replace an actual stopping policy: accidentally passing the goal can raise it. When SR and OSR share distance and sampling conventions, their gap can help diagnose stopping-related problems.

Teaching example: passing a goal differs from stopping correctly. Assume success depends only on distance to the goal at stopping, with a 3 m threshold and the same distance definition throughout.

flowchart LR
    A["Start"] --> B["1 m from goal during the route<br/>Inside success region"]
    B --> C["Keeps moving; ends 5 m from goal<br/>Stops here"]
    B -.-> D["Oracle success = 1"]
    C -.-> E["Success = 0"]
    classDef pass fill:#e4f2e8,stroke:#548565,color:#21452e;
    classDef fail fill:#fbe9e5,stroke:#b66b58,color:#612f25;
    class B,D pass;
    class C,E fail;

This gap motivates inspection of stopping, target recognition, and backtracking. High OSR alone does not prove accurate localization: the trajectory may pass the goal accidentally, or the model may never recognize that it has seen it.

8.3 Path efficiency and fidelity

8.3.1 SPL

Let $l_i$ be the shortest feasible start-to-goal path length and $p_i$ the actual traveled length:

Compute the score per trajectory and then average; it is not SR multiplied by mean efficiency over all trajectories. Follow the benchmark implementation for degenerate cases such as zero lengths. See the original SPL evaluation recommendations.

For two trajectories, if one succeeds with efficiency 0.5 and the other fails, SR is 0.5 and SPL is 0.25. SPL measures travel efficiency; extra generated tokens, network waiting, or stationary computation time do not automatically enter its denominator.

Four trajectory examples: in one straight corridor, the shortest distance from start to the exact goal is 10 m, the success threshold is 3 m, and every agent explicitly stops. The diagram shows behavioral categories, not spatial scale.

flowchart TB
    S["Common start: shortest distance to exact goal is 10 m"]
    S --> A["Direct arrival<br/>Travel 10 m; endpoint error 0 m"]
    S --> B["Arrival after detour<br/>Travel 20 m; endpoint error 0 m"]
    S --> C["Stop inside success region<br/>Travel 8 m; endpoint error 2 m"]
    S --> D["Stop too early<br/>Travel 6.5 m; endpoint error 3.5 m"]
    A --> AS["Success; SPL = 1"]
    B --> BS["Success; SPL = 0.5"]
    C --> CS["Success; SPL = 1"]
    D --> DS["Failure; SPL = 0"]
    classDef pass fill:#e4f2e8,stroke:#548565,color:#21452e;
    classDef fail fill:#fbe9e5,stroke:#b66b58,color:#612f25;
    class AS,BS,CS pass;
    class DS fail;

Together, these four yield SR 0.75 and SPL 0.625. The third travels less than the shortest distance to the exact goal because it stops within the success region, not because it beats the shortest path in the same geometry. max caps efficiency at 1. If a benchmark defines $l_i$ to the goal region, recalculate according to that implementation.

SPL assigns zero to failures and an inverse-length penalty to successful but excessive travel. It does not require visiting every instructed landmark. When instructions require a detour, faithful execution can lower SPL, motivating path-fidelity metrics.

8.3.2 CLS

For reference path $R$ and predicted path $P$, define point-to-path distance as $d(r,P)=\min_{p\in P}d(r,p)$. CLS combines soft coverage with length matching:

Here $L$ denotes path length and $d_{th}$ is a distance scale. CLS encourages coverage of route nodes but imposes no explicit visit order; use nDTW to assess sequence. See the original CLS paper.

Two steps in the formula: PC measures how far reference locations are from anywhere on the traveled path; LS checks whether actual length is proportionate to the covered portion. Because point-to-path distance takes a minimum, PC alone does not check visit order.

flowchart TB
    R["Each reference-path node"] --> D["Nearest distance to predicted path"]
    D --> PC["Average exponentially decayed distances<br/>Soft coverage PC"]
    PC --> EPL["EPL = PC × reference-path length"]
    EPL --> LS["Compare EPL with actual length<br/>Length score LS"]
    PC --> C["CLS = PC × LS"]
    LS --> C
    classDef score fill:#e8f1fb,stroke:#426a91,color:#18354f;
    class PC,LS,C score;

PC values here are chosen for illustration, not measured from a route diagram; 0.5 does not mean exactly half the nodes were visited. Even when every reference point lies within $d_{th}$, PC need not approach 1: a point exactly $d_{th}$ away contributes only $e^{-1}\approx0.368$.

CLS can reveal arrival that skips parts of the reference route, but identical coverage sets can have different visit orders. This motivates DTW alignment below.

8.3.3 nDTW and SDTW

DTW finds order-constrained path alignments while allowing different sample counts. Common normalized and success-weighted forms are:

Dataset nDTW is also calculated per episode and then averaged. Order alignment better captures route constraints than coverage alone, but still depends on reference quality. See the original nDTW paper.

Implementation limits: do not assume NE’s goal distance and DTW’s point-pair distance are identical. Continuous implementations also choose sampling, deduplication, and approximate-DTW procedures. Pin evaluator code rather than copying formulas alone. See the VLN-CE metric implementation.

Why not subtract corresponding steps directly? For reference A→B→C and prediction A→A→B→C, index-wise comparison mismatches B with A at the second step. DTW allows multiple samples to match one reference point while preserving order.

flowchart TB
    subgraph Ref["Reference path"]
        direction LR
        R1["A"] --> R2["B"] --> R3["C"]
    end
    subgraph Pred["Predicted path"]
        direction LR
        P1["A"] --> P2["A"] --> P3["B"] --> P4["C"]
    end
    R1 -. "Align" .-> P1
    R1 -. "Align" .-> P2
    R2 -. "Align" .-> P3
    R3 -. "Align" .-> P4

With exactly repeated positions in this example, alignment cost can be zero. Real displacement, sampling, and noise still affect DTW; it is not universally invariant to speed and sampling. A→C→B→C visits the same landmarks but generally cannot achieve the same zero-cost ordered alignment.

DTW’s accumulated cost can be expressed by this recurrence, where $D(a,b)$ is the minimum alignment cost for the two path prefixes, with standard DTW boundary initialization:

For example, if $\lvert R\rvert=4$, $d_{th}=3$ m, and DTW is 6 m, nDTW is $e^{-0.5}\approx0.607$. If that episode fails, its SDTW is 0. With a second successful episode at nDTW 0.8, mean nDTW is about 0.704 and mean SDTW is 0.4. Apply success weighting per episode; overall SR times overall nDTW is not a substitute.

8.4 Safety and deployment metrics

8.4.1 Collision and social safety

Collisions can be counted per action step, contact event, or episode, yielding different denominators. Define Collision Rate / Human Collision Rate explicitly and report completion alongside them, avoiding apparent safety from remaining stationary. Personal-space violations are also broader than physical contact.

8.4.2 Dynamic stability

Record falls, immobilization, emergency stops, and manual resets separately. Specify pose or time thresholds, recovery budgets, and whether resets terminate an episode. Different embodiments have different failure modes; do not combine them into an unconditional embodied-success score.

8.4.3 Efficiency and real-time control

Report camera-to-action end-to-end latency, including mean and tail latency, hardware, batch size, resolution, history length, and model invocation mode. Also report control frequency, GPU memory, API/token costs, task time, and human interventions. A model’s forward-pass FPS is not the robot’s effective control frequency.

8.5 Evaluation protocols

8.5.1 Val-Seen and Val-Unseen

Seen / Unseen usually split benchmark scenes to measure scene generalization within the same task. They do not guarantee that large-scale pretraining excluded related buildings, images, or derived assets, nor establish cross-robot, cross-language, or real-world generalization.

8.5.2 Fair comparison checklist

• Same inputs: camera count, field of view, depth, pose, and map access.
• Same task: data version, language subset, episodes, goals, and stopping criteria.
• Same execution: action space, controller, step/time budgets, sliding, and collision configuration.
• Transparent resources: extra training data, backbone, tool calls, search/retry counts, and compute budget.
• Inspectable statistics: success and total counts; repeated runs or confidence intervals for stochastic tasks, with sampling methods explained.

8.5.3 Reproducible evidence

Identify the paper version and result table first, then check the official evaluator. The VLN-CE repository has explicitly noted that SPL differences between a paper and leaderboard for the same baseline relate to hardware and Habitat builds. Protocols and implementations are themselves part of the experiment. See the official reproduction notes.

The companion paper readings help locate methods and original results. Until the above checks are completed, scores from different articles or versions are reading pointers, not evidence for causal claims of architectural superiority.

9. Learning resources and frameworks

A useful learning order is classic tasks → continuous environments → foundation models → real deployment. Reproduce R2R / VLN-CE baselines and learn Habitat episodes, sensors, and metrics; then study streaming VLA, fast-slow systems, and spatial memory; finally move to agents, world models, and real robots. Starting with the latest large model can obscure differences caused by protocols and action interfaces.

Companion resources on this site:

• VLN Papers: Instruction Following and Leaderboards: detailed readings, performance, and open-source status organized by task setting.
• Spatial Intelligence Survey: foundations of 3D representations, maps, and spatial reasoning.
• World Models Survey: predictive models, world-action models, and data engines.
• Traditional Robot Navigation Survey: SLAM, global/local planning, and controllers, complementing hierarchical control in Section 3.2, action interfaces in Section 3.4, and deployment metrics in Section 8.4.
• VLA Survey: vision-language-action training paradigms and action-interface design.

VLN-Survey-with-Foundation-Models

• Type: GitHub resource repository.
• Focus: VLN in the LLM/VLM era (2023–present), with ongoing paper updates.
• Audience: researchers studying how large models are changing VLN.

Awesome-Embodied-AI

• Type: full-stack resource collection.
• Focus: the embodied-intelligence stack, including VLN, VLA, and robot manipulation.
• Audience: researchers seeking a systematic overview of embodied AI.

Embodied-AI-Guide

• Type: introductory tutorials and practical guidance.
• Focus: code exercises, paper explanations, and learning paths.
• Audience: beginners or readers needing a structured learning path.

Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions

• Type: survey paper (Gu et al., ACL 2022).
• Focus: the development of VLN, covering tasks, methods, and evaluation from 2018–2022.
• Audience: researchers seeking a broad historical view of VLN.

Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models

• Type: survey paper (Zhang et al., 2024), accompanying the VLN-Survey-with-Foundation-Models repository above.
• Focus: reorganizes VLN around foundation models and discusses convergence with world models, human models, and VLA.
• Audience: researchers approaching VLN from the large-model perspective.

9.1 Conferences and workshops

Embodied-intelligence venues:

• Embodied AI Workshop — a CVPR workshop featuring recent directions and challenges.
• CoRL (Conference on Robot Learning) — a major venue for VLN transfer to real robots.
• RSS (Robotics: Science and Systems) — a leading robotics conference with emphasis on sim-to-real.

Common conference emphases:

9.2 A reproducible workflow

This survey proposes the following workflow for turning methods from the companion readings into comparable systems.

Completed inference does not guarantee a still-valid action. Camera capture, encoding, generation, communication, and actuation jointly determine loop latency. When the slow system returns a subgoal, the robot may already have left the relevant viewpoint. Define when old goals expire, when to replan, and what the fast system does while waiting. Report measurement boundaries, hardware, batch sizes, and control frequency alongside model FPS.

Choose methods by tracing back from failure evidence. Repeated visits on long routes suggest checking memory and map updates; reaching the goal vicinity without success suggests stopping and target verification; correct semantic goals with frequent collisions suggest reachability, action interfaces, and controllers. Metric gaps provide clues; trajectory replay and controlled experiments must establish causes.

9.3 Open questions

The reviewed work supports several conclusions with explicit scope: R4R and path metrics show the need to measure arrival and instruction fidelity separately; ETPNav and DualVLN demonstrate concrete planning/execution divisions; ScaleVLN establishes training resources as an important performance variable; VLN-PE provides experimental evidence that visual and physical gaps affect navigation. These findings come from path-following research, continuous graph planning, dual-system design, data scaling, and physical embodiment evaluation. They do not establish one optimal architecture for every task.

The table proposes open questions and evaluation approaches based on that evidence. These are research judgments, not a field-wide consensus.

This survey recommends first building replayable, attributable failure analysis, then choosing model changes. If local execution dominates failures, a larger language model may not be the most direct remedy. If stopping and route constraints dominate, mean SR alone may hide useful improvements. Progress should explain under which conditions a failure class was reduced and at what cost.

9.4 Scope and maintenance

This is a problem-oriented narrative survey covering route-instruction navigation and related goal-navigation, interaction, and embodied-control work. It is not a systematic review with exhaustive search and exclusion procedures. Boundaries are stated where other tasks inform mechanisms; general VLA and manipulation papers are not treated as direct performance evidence for standard VLN.

Data-counting conventions, platform support, and core metrics were verified together on 2026-09-26. On 2026-10-07, Section 3 was reorganized with the companion paper readings, adding checks of the mechanism descriptions for representative methods in action interfaces, progress and active perception, and predictive learning. On 2026-10-09, Section 3.8 was expanded, checking RL feedback levels, rewards, and same-backbone ablations against the specified versions of the original papers. These method-section revisions do not mean that every dataset, platform, or leaderboard was re-verified. Changeable data are tied to paper versions or data-card sources; entries without a confirmed independent protocol retain uncertainty notes. Other work is available through companion paper readings. This survey does not claim to have independently reproduced every experiment or verified all leaderboard results.

10. References

Detailed paper readings: see VLN Papers: Instruction Following and VLN Papers: Goal and Agentic Navigation. This section lists datasets, methods, simulators, and surveys directly discussed above, grouped by topic with continuous numbering. For preprint publication status and leaderboard values, consult paper homepages and the companion readings.

10.1 Datasets and benchmarks

10.1.1 Instruction-following and continuous

1. R2R — Anderson et al., Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments, CVPR 2018. [Paper]
2. R4R — Jain et al., Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation, ACL 2019. [Paper]
3. RxR — Ku et al., Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding, EMNLP 2020. [Paper]
4. VLN-CE — Krantz et al., Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments, ECCV 2020. [Paper]
5. RxR-CE — RxR adapted to continuous Habitat environments, released with VLN-CE for the RxR-Habitat challenge (CVPR Embodied AI Workshop). [Code]
6. ScaleVLN — Wang et al., Scaling Data Generation in Vision-and-Language Navigation, ICCV 2023. [Paper]
7. VLN-PE — Wang et al., Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparities, ICCV 2025. [Paper]
8. InternData-N1 — InternRobotics (Shanghai AI Laboratory), large-scale synthetic navigation data for InternVLA-N1 in LeRobot format, 2025. [Dataset] [Code]
9. VLNVerse — Lin et al., VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluation, arXiv 2025. [Paper]

10.1.2 Goal-oriented and long-horizon

1. REVERIE — Qi et al., REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments, CVPR 2020. [Paper]
2. REVERIE-CE — A family of continuous-environment adaptations without a unified official protocol. Cite the specific paper’s conversion implementation; see Section 6.3.2.
3. SOON — Zhu et al., SOON: Scenario Oriented Object Navigation with Graph-based Exploration, CVPR 2021. [Paper]
4. LHPR-VLN — Song et al., Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method, CVPR 2025. [Paper]

10.1.3 Dialog and social

1. CVDN — Thomason et al., Vision-and-Dialog Navigation, CoRL 2019. [Paper]
2. TEACh — Padmakumar et al., TEACh: Task-driven Embodied Agents that Chat, AAAI 2022. [Paper]
3. HA-VLN — Li et al., Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions, NeurIPS 2024 Datasets and Benchmarks Track. [Paper]
4. HA-VLN 2.0 — Dong et al., HA-VLN 2.0: An Open Benchmark and Leaderboard for Human-Aware Navigation in Discrete and Continuous Environments, IROS 2026 (first posted to arXiv in 2025; checked v5). [Paper]

10.1.4 Demand-driven, outdoor and aerial

1. DDN — Wang et al., Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigation, NeurIPS 2023. [Paper]
2. Touchdown — Chen et al., Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments, CVPR 2019. [Paper]
3. StreetLearn — Mirowski et al., The StreetLearn Environment and Dataset, arXiv 2019. [Paper]
4. AerialVLN — Liu et al., AerialVLN: Vision-and-Language Navigation for UAVs, ICCV 2023. [Paper]
5. CityNav — Lee et al., CityNav: A Large-Scale Dataset for Real-World Aerial Navigation, ICCV 2025（arXiv 2024）. [Paper]
6. OpenFly — Gao et al., OpenFly: A Comprehensive Platform for Aerial Vision-Language Navigation, ICLR 2026 (first posted to arXiv in 2025; checked v7). [Paper]

10.2 Core models and methods

10.2.1 Cross-modal pre-training

1. PREVALENT — Hao et al., Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training, CVPR 2020. [Paper]
2. VLN-BERT — Majumdar et al., Improving Vision-and-Language Navigation with Image-Text Pairs from the Web, ECCV 2020. [Paper]
3. Recurrent VLN-BERT — Hong et al., A Recurrent Vision-and-Language BERT for Navigation, CVPR 2021. [Paper]
4. HAMT — Chen et al., History Aware Multimodal Transformer for Vision-and-Language Navigation, NeurIPS 2021. [Paper]

10.2.2 Topological graphs and memory

1. DUET — Chen et al., Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation, CVPR 2022. [Paper]
2. ETPNav — An et al., ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments, IEEE TPAMI 2024. [Paper]
3. LagMemo — Zhou et al., LagMemo: Language 3D Gaussian Splatting Memory for Multi-modal Open-vocabulary Multi-goal Visual Navigation, arXiv 2025. [Paper]

10.2.3 Fast-slow dual systems

1. DualVLN — Wei et al., Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigation, arXiv 2025. [Paper]
2. NaVILA — Cheng et al., NaVILA: Legged Robot Vision-Language-Action Model for Navigation, RSS 2025. [Paper]
3. NavDP — Cai et al., NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance, arXiv 2025. [Paper]
4. InternVLA-N1 — InternRobotics (Shanghai AI Laboratory), InternVLA-N1: An Open Dual-System Vision-Language Navigation Foundation Model with Learned Latent Plans, 2025. [Code]
5. OmniNav — Xue et al., OmniNav: A Unified Framework for Prospective Exploration and Visual-Language Navigation, arXiv 2025. [Paper]
6. Qwen-RobotNav — Zhang et al., Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation System, arXiv 2026. [Paper]

10.2.4 Video VLMs and continuous policies

1. Waypoint Models — Krantz et al., Waypoint Models for Instruction-guided Navigation in Continuous Environments, ICCV 2021. [Paper]
2. NaVid — Zhang et al., NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation, RSS 2024. [Paper]
3. StreamVLN — Wei et al., StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling, ICRA 2026 (first posted to arXiv in 2025; checked v2). [Paper]

10.2.5 Low-level visual navigation

1. DD-PPO — Wijmans et al., DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames, ICLR 2020. [Paper]
2. GNM — Shah et al., GNM: A General Navigation Model to Drive Any Robot, ICRA 2023. [Paper]
3. ViNT — Shah et al., ViNT: A Foundation Model for Visual Navigation, CoRL 2023. [Paper]
4. NoMaD — Sridhar et al., NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration, ICRA 2024. [Paper]
5. ViPlanner — Roth et al., ViPlanner: Visual Semantic Imperative Learning for Local Navigation, ICRA 2024. [Paper]

10.2.6 World models

1. Navigation World Models (NWM) — Bar et al., Navigation World Models, CVPR 2025. [Paper] [Project]
2. Dynam3D — Wang et al., Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigation, NeurIPS 2025. [Paper]
3. WMNav — Nie et al., WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation, IROS 2025. [Paper] [Project]
4. AstraNav-World — Chen et al., AstraNav-World: World Model for Foresight Control and Consistency, arXiv 2025. [Paper]
5. NavWAM — Azuma et al., NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation, arXiv 2026. [Paper]
6. DreamVLA — Zhang et al., DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge, NeurIPS 2025. [Paper] [Project]
7. NVIDIA Cosmos — NVIDIA, Cosmos World Foundation Model Platform for Physical AI, arXiv 2025. [Paper] [Project]

10.2.7 Agents and general VLA

1. AgentVLN — Xin et al., AgentVLN: Towards Agentic Vision-and-Language Navigation, arXiv 2026. [Paper] [Code]
2. SE-VLN — Dong et al., SE-VLN: A Self-Evolving Vision-Language Navigation Framework Based on Multimodal Large Language Models, arXiv 2025. [Paper]
3. RT-2 — Brohan et al., RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control, CoRL 2023. [Paper]
4. OpenVLA — Kim et al., OpenVLA: An Open-Source Vision-Language-Action Model, CoRL 2024. [Paper]

10.3 Simulators and scene assets

1. Matterport3D Simulator — Anderson et al., CVPR 2018 (released with R2R). [GitHub]
2. Habitat — Savva et al., Habitat: A Platform for Embodied AI Research, ICCV 2019 [Paper]；Szot et al., Habitat 2.0: Training Home Assistants to Rearrange their Habitat, NeurIPS 2021 [Paper]；Puig et al., Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots, ICLR 2024 [Paper]. [GitHub]
3. AI2-THOR / ProcTHOR — Kolve et al., AI2-THOR: An Interactive 3D Environment for Visual AI, arXiv 2017 [Paper]；Deitke et al., ProcTHOR: Large-Scale Embodied AI Using Procedural Generation, NeurIPS 2022 [Paper]. [Website]
4. Gibson / iGibson / OmniGibson — Xia et al., Gibson Env: Real-World Perception for Embodied Agents, CVPR 2018 [Paper]; Li et al., iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks, CoRL 2021 [Paper]; Li et al., BEHAVIOR-1K, CoRL 2022 (extended arXiv version, 2024) [Paper]. [GitHub]
5. AirSim — Shah et al., AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles, FSR 2017. [GitHub] (see repository notes for maintenance and compatibility) / [Colosseum]
6. Isaac Sim / Isaac Lab — NVIDIA; Mittal et al., Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments (predecessor of Isaac Lab), IEEE RA-L 2023 [Paper]. [Docs]
7. MuJoCo / MJX — Todorov et al., MuJoCo: A Physics Engine for Model-based Control, IROS 2012; maintained by Google DeepMind. [GitHub] / [Docs]
8. InternUtopia (GRUtopia) — Wang et al., GRUtopia: Dream General Robots in a City at Scale, arXiv 2024. [Paper] [GitHub]

10.4 Survey papers

1. Gu et al., Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions, ACL 2022. [Paper]
2. Zhang et al., Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models, arXiv 2024. [Paper] [GitHub]

10.5 RL and post-training

1. DAgger — Ross et al., A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning, AISTATS 2011. [Paper]
2. RCM — Wang et al., Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation, CVPR 2019. [Paper]
3. EnvDrop — Tan et al., Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout, NAACL 2019. [Paper]
4. PPO — Schulman et al., Proximal Policy Optimization Algorithms, arXiv 2017. [Paper]
5. GRPO / DeepSeekMath — Shao et al., DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models, arXiv 2024. [Paper]
6. nDTW / SDTW — Ilharco et al., General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warping, NeurIPS ViGIL Workshop 2019. [Paper]
7. VLN-R1 — Vision-Language Navigation via Reinforcement Fine-Tuning, arXiv 2025 (checked v1). [Paper]
8. ActiveVLN — Towards Active Exploration via Multi-Turn RL in Vision-and-Language Navigation, arXiv 2025 (checked v1). [Paper]
9. ETP-R1 — Evolving Topological Planning with Reinforcement Fine-tuning for Vision-Language Navigation in Continuous Environments, arXiv 2025 (checked v1). [Paper]
10. JOP-VLN — Joint On-and-Off Policy Learning for Vision-and-Language Navigation, arXiv 2026 (checked v1). [Paper]
11. NavGRPO — Trajectory-Diversity-Driven Robust Vision-and-Language Navigation, arXiv 2026 (checked v1). [Paper]
12. GroundingVLN / GEAR — Reasoning and Acting with Grounding for Vision-Language Navigation, arXiv 2026 (checked v3). [Paper]