OceanX CTD MR  /  Case study

Mixed-reality learning prototype  ·  Meta Quest 2, 3 & 3S  ·  Created for the OceanX TIDE Science Impact Challenge

Ocean sampling in your room

A conductivity, temperature, and depth (CTD) rosette is an instrument ocean scientists lower from a ship to measure the water column and collect water samples on the way back up. This hand-tracked Meta Quest prototype brings a full-scale rosette through a portal in your floor, then guides you through preparing it, conducting a simulated ocean cast, collecting a sample, and exploring the resulting temperature profile.

The full-scale CTD rosette and its spatial temperature profile viewed underwater in the Meta Quest prototype Watch the prototype
Underwater view  The rosette and the recovered temperature profile, with the sample depth marked at 175 meters.
Role
Lead Developer
Build time
1 week
Platform
Meta Quest 2, 3 & 3S
Built with
Unreal Engine 5.7
Input
Hand tracking only
Length
2–3 min, replayable
Status
Proof of concept

The challenge

Ocean systems are connected. Their data is not.

The challenge called for an interactive 3D environment that could bring complex ocean-data layers together without overwhelming the user or sacrificing clarity and performance. The objective was not simply to display more information. It was to help users move from identifying what is present to investigating why it is there.

The OceanX TIDE Science Impact Challenge asked teams to rethink how complex ocean data could be interpreted and communicated. We selected Challenge 1: “Fusing Biological, Physical, and Geological Layers in 3D.”

Ocean scientists study biological observations, seafloor structure, CTD profiles, water chemistry, and ocean circulation within the same environment, yet these datasets are often examined using separate tools. This fragmentation makes it difficult to see how conditions relate spatially and to understand why a biological or environmental pattern occurs in a particular location.

Team and role

A small interdisciplinary team, led from concept through implementation

I formed and led the team, originated the CTD-focused concept, defined the experience, and carried the project through full implementation.

Adam Betts, a professional technical artist, served as art lead, created the CTD model, and helped write the proposal.

Humayra Rahman provided the project’s scientific backbone, helped write the proposal, and shaped what we would build next if it is accepted.

Our response

A focused prototype built around one complete CTD workflow

Challenge 1 covered a broad range of biological, physical, and geological data. Rather than attempting to combine every possible dataset in one prototype, we focused on the CTD cast as a clear and achievable starting point.

A CTD cast connects several parts of ocean research: the physical instrument, changing water-column measurements, the collection of samples at selected depths, and the interpretation of the resulting data. This made it a strong foundation for exploring how mixed reality could connect scientific equipment with the information it produces.

Our concept places a full-scale CTD rosette in the user’s room and guides them through the complete workflow. The learner completes the safety and startup procedure, defines the cast and sampling depths, observes the changing measurements during descent and ascent, fires a sampling bottle, and evaluates the resulting temperature profile and recovered sample.

This focused scope allowed us to develop a complete experience while creating a foundation that could incorporate additional biological, geological, and environmental datasets in future iterations.

Experience walkthrough

The complete sequence, from arrival to result

Full-scale CTD rosette ready for inspection in the headset experience
Arrival

The rosette comes up through your floor

The experience begins in mixed reality. A portal opens in the real room and a life-size CTD rosette rises beside the learner’s own furniture.

Its scale is the first thing that needs to land — and the one thing a photograph of a rosette cannot fully communicate.

Inspection The full-scale rosette at the start of the guided inspection, with the first instruction panel within reach.

Preflight · Guided inspection

Seven steps from a sealed instrument to a running cast

The inspection unfolds in a deliberate sequence, introducing each system only when it becomes relevant. The learner identifies each component through touch, confirms it with a deliberate press, and gradually prepares the rosette for deployment. With the instrument ready, control shifts back to the learner to choose the mission depths and begin the cast.

  • The protective rosette frame lit at step one, with its card pinned above the instrument
    Step 1 / 7

    Touch the frame

    The frame protects the sensors and bottles during deployment and keeps every component in a stable position as the instrument is winched over the side.

    Console TOUCH THE FRAME → FRAME IDENTIFIED

  • The lifting bail and cable lit at step two, with its card pinned above the instrument
    Step 2 / 7

    Touch the bail

    The bail is the lifting point the rosette hangs from. The cable above it lowers and recovers the CTD, and carries commands and measurements between the instrument and the ship.

    Console TOUCH THE BAIL → BAIL IDENTIFIED

  • The CTD sensor package lit at step three, with its card pinned above the instrument
    Step 3 / 7

    Touch the sensor

    The sensors continuously measure conductivity, temperature, and pressure as the CTD moves through the water. Those readings are what produce salinity and depth.

    Console TOUCH THE SENSOR → SENSOR IDENTIFIED

  • The Niskin bottle array lit at step four, with its card pinned above the instrument
    Step 4 / 7

    Touch a bottle

    The Niskin bottles stay open while the CTD descends, letting seawater flow through them. Individual bottles close at selected depths to preserve a sample.

    Console TOUCH A BOTTLE → BOTTLES IDENTIFIED

  • Step 5 / 7

    Pinch the ripcord

    Drawing the cord past a tenth of its travel opens the release mechanism. It is the first step that asks for a gesture rather than a touch.

    Console PINCH → CORD DRAWN

  • The rosette during preparation, with the console card and the pinch prompt at the interaction point
    Step 6 / 7

    Remove the sensor cap

    Removing the cap makes preparation tangible and gives the learner responsibility for the instrument. The cast cannot begin until they complete the action themselves.

    Console REMOVE THE CAP → CAP REMOVED

  • The mission plan panel with the sample depth set to 175 meters inside a 300 meter descent
    Step 7 / 7

    Begin the cast

    The plan sets how deep the cast will run and where Bottle #07 will close. Confirming 175 m inside a 300 m cast hands control back, and the descent begins on the learner’s own command.

    Console CONFIRM MISSION → BEGIN CAST

The cast

A planned depth becomes a physical sample

The inspection explains the instrument. The cast shows why its parts matter: live measurements build the water column while one bottle waits to collect water from the depth chosen by the learner.

  • Bottle 07 armed to fire while the rosette holds at the planned 175 meter sample depth
    Upcast · Physical sample

    You fire Bottle #07 at the depth you planned

    On the upcast, the rosette stops at the learner’s selected sampling depth. The live readings hold while Bottle #07 waits for the learner to fire it by hand.

    This is the hinge of the experience: an earlier decision becomes a physical action, connecting one bottle of water to one exact point in the ocean.

  • Live cast · Spatial data

    The water column builds around the cast

    As the CTD descends, the temperature profile grows inside the room. Changes in the water become a spatial structure rather than a series of numbers on a conventional graph.

    The learner can see where the temperature changes most sharply and understand the selected sample depth as a position within that larger profile.

  • Complete sequence · Inspection to results

    One decision carries through the entire experience

    The full playthrough shows the sequence without interruption: preparing the instrument, choosing a cast depth, following the descent, firing Bottle #07, and revealing the completed profile.

    Seen together, the steps form one continuous chain from scientific instrument to measurement, physical sample, and spatial result.

Three consequential decisions

In each of these, the easier option was available and would have made the experience worse

Why I chose Unreal Engine for Meta Quest

What I chose

With five years of Unreal Engine experience, I built the prototype in Unreal 5.7 using C++, OpenXR, and OculusXR. This gave me direct control over the experience architecture, rendering, interaction systems, and Quest optimization.

The easier alternative

Unity offers a more established Quest workflow, with extensive Meta-supported examples for passthrough, hand tracking, and mixed-reality interaction. It would have reduced the time required to establish the project’s core XR features.

Why I rejected it

Choosing Unity would have meant rebuilding my production workflow while working against a deadline. Unreal required more initial XR integration, but it allowed me to make confident decisions about architecture, materials, performance, and packaging from the beginning and carry the concept through to a polished on-device build.

The learner uses their hands, not controllers

What I chose

Every core interaction uses hand tracking. The learner inspects the rosette, completes its startup procedure, pulls the release cord, fires Bottle #07, and examines the recovered sample using direct gestures.

The easier alternative

Quest controllers would have provided more precise input, physical buttons, haptic feedback, and interaction patterns that are easier to implement and test reliably.

Why I rejected it

From my industry experience, hand tracking is the stronger default for short, public-facing mixed-reality experiences. It feels more intuitive, requires less instruction, and removes the controller learning curve. Unless a client or interaction specifically requires controllers, I design around hands first.

One continuous cast across mixed and virtual reality

What I chose

The experience begins in passthrough, placing the full-scale rosette inside the learner’s room. During the cast, they can enter an underwater virtual environment and return to mixed reality without pausing, restarting, or losing their progress.

The easier alternative

The simpler approach was to commit to a single presentation: keep the full experience in passthrough or build it entirely in virtual reality. This would remove the complexity of transitioning between environments, but it would also limit what the experience could communicate.

Why I rejected it

Passthrough places the full-scale CTD and its interactions directly inside a trade-show, classroom, or demonstration space. The learner can remain in conversation with a presenter while seeing and manipulating the virtual system within their physical surroundings. VR adds the underwater environment when deeper immersion is valuable, without separating it from the live demonstration.

Project status

A working prototype for a larger OceanX proposal

This prototype was developed for our submission to the OceanX TIDE Science Impact Challenge. It demonstrates one part of a broader concept: using mixed and virtual reality to connect scientific instruments, ocean data, and guided interaction within the same spatial experience.

The proposal has been submitted, and we are currently awaiting feedback. If selected, the next phase would expand beyond the CTD cast to include additional simulations, interactive data layers, and connections between biological, physical, and geological information. We would also incorporate feedback from OceanX scientists, test the experience with users, and evaluate how effectively each interaction communicates the underlying science.

This is an early working prototype, not a finished learning platform. Its purpose is to establish a direction that can be tested, refined, and expanded through collaboration with scientists, educators, and learners.

Development workflow

Parallel AI agents working inside one controlled architecture

AI agents accelerated development, but the architecture and technical direction remained under my control. I divided the project into bounded workstreams, defined how each system connected, and used automated verification to support rapid iteration.

Parallel development. I ran multiple coding agents concurrently across separate areas of the project, including interaction logic, spatial UI, automated testing, build tooling, and Android deployment. This allowed independent systems to progress simultaneously without giving any single agent responsibility for the entire project.

System boundaries. Each assignment defined the system’s responsibilities, inputs, outputs, state transitions, and relationship to the rest of the experience. Establishing these boundaries before implementation reduced ambiguity and helped independently developed systems remain compatible.

Review and integration. I reviewed every change before accepting it. This included evaluating the implementation, resolving overlapping changes, checking Unreal and Meta XR API usage, and ensuring each system remained consistent with the shared state-driven architecture.

Automated verification. Headless testing ran the complete experience without requiring a headset for every iteration. Additional tooling validated the Android configuration, packaged and installed the build, launched it on Quest, and collected runtime logs — allowing development, testing, and deployment to advance together.

Limitations

A focused prototype with room to grow

By design

  • The current prototype uses hand tracking only. Controller support could be added without changing the core experience.
  • The experience follows one cast, one collected sample, and one primary temperature structure — the minimum scope needed to demonstrate the complete workflow.
  • The self-guided sequence lasts approximately two to three minutes, making it suitable for short demonstrations at exhibitions, classrooms, and public events.

Known and scoped

  • The environmental profile is generated from predefined control points rather than live or archived scientific data.
  • Stage placement uses fixed spatial offsets and does not yet adapt to the geometry or available space of each room.
  • The scientific content has received interdisciplinary input but still requires formal review and validation by practicing ocean scientists.

Outside the current scope

  • The prototype is not a digital twin of a specific CTD instrument, research vessel, or recorded cast.
  • It is not yet a production-ready exhibit designed for sustained public deployment.
  • It is not a certified scientific or instructional system and should not be used for professional training, operational planning, or scientific analysis.

Technical appendix

Implementation details for XR builders

This section is for readers who build XR; everyone else can skip to Reflection.

Build access

To experience the prototype in your own environment, contact me at edm58@duke.edu to request the Meta Quest APK and installation instructions.

01 Engine and language

Unreal Engine 5.7, developed primarily in C++. Keeping gameplay logic in code makes the experience easier to review, version, and test through automated tools.

02 XR platform

Built for Meta Quest 2, 3, and 3S using OculusXR and OpenXR. Passthrough is the default presentation, with virtual reality available within the same session.

03 State-driven architecture

The rosette, spatial UI, stage, and data visualizations are spawned at runtime. A central state machine controls inspection, planning, casting, sampling, results, and reset.

04 Material-driven exploded view

The instrument separates through World Position Offset rather than multiple animated actors. This reduces object overhead while preserving a detailed close-range presentation on Quest.

05 Interaction model

Hand tracking supports inspection, pinch interactions, spatial UI, sampling, and stage repositioning. Interactions use generous targets and clear feedback to account for reduced precision.

06 Continuous MR and VR

Moving between passthrough and the underwater environment preserves the active cast. The simulation continues without restarting, pausing, or loading a separate level.

07 Automated validation

Automated tests verify state progression, actor creation and removal, sampling, reset, and replay. These checks validate software behavior — not usability or learning outcomes.

08 Build and deployment

Headless tools can run the sequence and capture staged frames without a headset. Deployment scripts package the Android build, validate its configuration, install it on Quest, launch it, and collect runtime logs.

Reflection

A focused CTD workflow created a stronger prototype

Focusing on one complete CTD workflow allowed us to connect the instrument, procedure, measurements, and recovered sample in a sequence that learners could follow from beginning to end.

The next stage requires more than additional features. It requires real ocean data, continued review from scientists, and evaluation with learners. These steps would determine whether the experience communicates the science effectively — not simply whether it creates a compelling immersive demonstration.

What I want to build next

Immersion matters when it makes the inaccessible understandable

This project reinforced the direction I want to continue pursuing: technically ambitious immersive experiences developed alongside people with deep subject-matter expertise.

I am interested in work that gives audiences access to instruments, environments, data, and procedures they could not ordinarily encounter firsthand, and that uses immersion to support meaningful learning outcomes rather than merely create a spectacular presentation.

Contact
Ethan Minnich · edm58@duke.edu
Demo film
youtu.be/EvIeLwiQ24s
Sources
Available on request
Institution
Duke University, Pratt School of Engineering