Skip to content

Add DMA-BUF side channel for zero-copy fd passing via SCM_RIGHTS #1570

Description

@julienzarka

Add DMA-BUF side channel for zero-copy fd passing via SCM_RIGHTS

Summary

This proposes a new crate iceoryx2-dmabuf that adds a Linux-only side channel to iceoryx2 for passing file descriptors (DMA-BUF, memfd, or any AsRawFd) between publisher and subscribers over a Unix domain socket using SCM_RIGHTS, paired with a typed-payload iceoryx2 service that carries metadata plus a handshake token.

The motivation is zero-copy interop with hardware pipelines (V4L2 cameras, GPU render targets, DRM/KMS display, hardware video codecs, NPU inference) where buffers do not live in CPU-addressable system RAM and therefore cannot be represented by iceoryx2's existing POSIX-shm transport.

Why this does not fit inside the current shm transport

iceoryx2's existing shared-memory transport is backed by shm_open / POSIX shared memory — system RAM, CPU-addressable. DMA-BUF exists precisely because buffers often do not live in system RAM:

  • GPU VRAM (Vulkan/EGL external memory)
  • V4L2 camera / codec DMA buffers (IOMMU-mapped, not always CPU-coherent)
  • DRM/KMS scanout (display controller consumes via drmModeAddFB2)
  • Format modifiers (AFBC, NV12 tiled, DCC — carried by DMA-BUF, lost under shm)
  • Memory bandwidth (4K60 RGBA = ~2 GB/s per stream; a camera→NPU→display pipeline via shm copies saturates DRAM on typical ARM SoCs)

Proposed design

Two-channel architecture:

  1. Metadata channel — a normal iceoryx2 publish-subscribe service carrying the user's typed message type plus a 64-bit monotonic DmabufToken.
  2. Fd channel — a Unix domain socket per service (path is deterministically derived via SHA-1 of the service name under /tmp) over which file descriptors are passed via SCM_RIGHTS control messages, each tagged with the same token.

The receiver reads the metadata sample, then reads the fd that matches the token. Token mismatch, missing fd, and service-drop cases map to typed DmabufError variants.

Public API surface:

  • DmabufPublisher<S: Service, Meta> with ::create and ::send(meta, &fd)
  • DmabufSubscriber<S: Service, Meta> with ::create and ::recv() -> Option<(Meta, OwnedFd)>
  • A new SideChannel trait in iceoryx2/src/port/side_channel.rs that documents the role contract for any out-of-band transport (upstream-visible change; kept minimal)

Optional features: memfd (memfd_create helpers), peercred (SO_PEERCRED UID check).

Platform: Linux only (non-Linux targets compile to Unsupported stubs so downstream code building for macOS/Windows remains portable).

Scope of the PR

Single PR landing:

  • iceoryx2-dmabuf/ - new crate (lib + 2 integration-test helper bins + examples + README + CHANGELOG)
  • iceoryx2/src/port/side_channel.rs - new SideChannel trait + Role enum in the core crate
  • benchmarks/dmabuf/ - latency / throughput / fanout benches for the 1080p RGBA8 memfd round-trip
  • doc/release-notes/iceoryx2-unreleased.md - Features entry

Around 12 logical commits. Tests colocated with features. No RED commits in the final history.

Open questions for maintainers (before the PR lands)

  1. First-party crate or external? The sidecar is Linux-only. Would maintainers prefer it inside the iceoryx2 workspace, or as an external crate that depends on iceoryx2?
  2. Placement of the FdSideChannel trait. Currently pub(crate) inside iceoryx2-dmabuf. Should this live alongside SideChannel in iceoryx2/src/port/?
  3. Roadmap alignment. Is there existing planning around fd-based side channels we should align with before rewriting commit history?

Status

An implementation exists on a fork at julienzarka/iceoryx2:feat/dmabuf-sidecar.

Validation complete as of 2026-04-20:

  • Linux (Ubuntu 25.10 arm64, Rust 1.93.0 via OrbStack) - all integration tests pass, clippy clean with -D warnings, --all-features build succeeds.
  • macOS (Darwin, cfg-gated stubs) - build + tests + clippy clean; non-Linux paths return UnsupportedPlatform.
  • The 10 Linux-gated integration tests all executed and passed: unit_scm::*, dmabuf_roundtrip, it_fd_identity, it_fanout, it_service_gone, it_socket_gone, peercred_mismatch (via --ignored), refcount_survival, prop_roundtrip. (it_crash_midsend is #[ignore]d by design - requires SIGSTOP timing.)

Happy to open a draft PR once directional feedback is in on the three open questions above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions