Operator Presentation

What

The Operator Presentation Module turns operator-side feedback into output. It may update a video surface, draw a predicted path, animate a timeline, play audio, or drive a haptic device. The paper treats these as presentation choices: the module determines what the operator sees or feels, not how the remote robot moved or how communication was delayed.

It always has access to Operator-side State Reconstruction and additionally receives output from Operator-side Assistance (downlink) when selected. Without assistance, reconstructed feedback is presented on its own.

Why

Presentation needs its own module because a study may change the operator interface without changing the feedback it receives. A delayed camera frame can be shown full-screen, placed beside a map, or combined with a haptic cue. A predicted path can be rendered as a line or an uncertainty region. Keeping those choices here makes the interface part of the study configuration rather than hidden inside transport or assistance code.

How

The presentation module receives explicitly reconstructed packages and optional assistance packages, then updates its local output sources once per kernel step. Study code may also read declared reconstructed buffers directly: the demo reads its latest video from 5.9 and composites overlays from 5.10.

• Package arrival. PresentPackage receives reconstructed information and assistance output in enqueue order.
• Composition. UpdatePresentation combines the latest reconstructed video with the selected overlay sources. Read the current video buffer each step; reconstruction owns its texture.
• Replacement. Any assistance that hides or replaces base feedback needs an explicit study-defined presentation contract. Omitting an overlay does not hide video.
• Source registry. Each OperatorPresentationSource owns its output target and has a unique id.

Boundary

Contract

The base module owns the package queue and validates the configured source registry. Your implementation provides the two hooks that receive packages and update the local output.

public abstract class OperatorPresentationModule
    : TeleroboticsModule, IPackageConsumer
{
    protected abstract void PresentPackage(
        Package package,
        double timestampSeconds);

    protected virtual void UpdatePresentation(
        double timestampSeconds)
    {
    }
}

• PresentPackage handles reconstructed information or assistance output and may route it to a source.
• UpdatePresentation runs every presentation step, including ticks with no new package.
• OperatorPresentationSource is the extension point for a concrete output target.
• The module should preserve package meaning; display transformations belong in the selected presentation source.

Examples

The four conditions keep delayed video fixed while changing the timeline, path, and envelope overlays.

public sealed partial class RoverViewPresentation : OperatorPresentationModule
{
    private void PresentReconstructedVideo()
    {
        ReconstructedVideoFrame view = stateReconstruction.LatestViewFrame;
        if (view == null || !view.Frame)
            return;

        operatorVideoImage.texture = view.Frame;
        operatorVideoImage.color = Color.white;
        predictionCamera.worldToCameraMatrix =
            view.WorldToCameraMatrix;
        predictionCamera.projectionMatrix =
            view.ProjectionMatrix;
    }
}

public sealed class NetworkTimelineVisualisation : EveryMoveVisualisation
{
    public override bool TryPresent(Package package, double timestampSeconds)
    {
        if (!(package.Payload is NetworkTimelinePackage timeline) ||
            timeline.Directions == null)
            return false;

        for (int index = 0;
             index < timeline.Directions.Length;
             index++)
        {
            SpawnTimelineNode(
                (int)timeline.Directions[index],
                timeline.CommandTimestampSeconds,
                timeline.RoundTripDelaySeconds);
        }
        return true;
    }
}

public sealed class PathRibbonVisualisation : EveryMoveVisualisation
{
    public override bool TryPresent(Package package, double timestampSeconds)
    {
        if (!(package.Payload is PathAssistancePackage path))
            return false;

        pending = path;
        return true;

    }

    public override void UpdateVisual(double timestampSeconds)
    {
        DrawRibbon(centreMesh, pending.Centre);
        DrawRibbon(leftMesh, pending.Left);
        DrawRibbon(rightMesh, pending.Right);
    }
}

public sealed class EnvelopeRegionVisualisation : EveryMoveVisualisation
{
    public override bool TryPresent(Package package, double timestampSeconds)
    {
        if (!(package.Payload is EnvelopeAssistancePackage envelope))
            return false;

        pending = envelope;
        return true;

    }

    public override void UpdateVisual(double timestampSeconds)
    {
        TrajectoryMeshing.BuildStudyEnvelopeMesh(
            pending.Centre,
            pending.Left,
            pending.Right,
            pending.ExtremaLeft,
            pending.ExtremaRight,
            terrain,
            regionMesh);
    }
}

Config

1. Implement OperatorPresentationModule in the study's Runtime assembly.
2. Attach one presentation implementation to the TeleroboticsAgent and assign its OperatorPresentationSource components.
3. Give every source a unique id. The base module validates the source registry before the trial starts.
4. Keep presentation targets local to the source: cameras, canvases, meshes, textures, audio outputs, or haptic devices.
5. Do not read the live Vehicle / Robot Model from presentation code. Use declared reconstructed information from 5.9 and assistance output from 5.10.