Implement RFC 3943: MIR move elimination by Amanieu · Pull Request #157943 · rust-lang/rust

Amanieu · 2026-06-15T22:42:55Z

This PR implements rust-lang/rfcs#3943: a MIR optimization pass that eliminates unnecessary copies. Since the new pass relies on the new MIR semantics from RFC 3943, it is gated behind the the -Z mir-move-elimination flag. Enabling this flag also disables the DestinationPropagation pass, which is completely superseded by this one.

There are 3 main parts to this optimization:

PreciseLiveness is an analysis that calculates, at a sub-statement granularity, the points in a function where a local requires storage to be allocated. This is more fine-grained than MaybeStorageLive, and takes borrows into account.
TailCopyToMove is a small pre-processing pass which turns copies into the return place just before a return into moves. This is necessary to allow the source of the copy to be unified with the return place.
MoveElimination is the main pass which optimizes away moves through place unification.

The RFC text and the top-level comments for the various passes have more details on the internals of the optimization.

r? @dianqk

cc @rust-lang/wg-mir-opt

rustbot · 2026-06-15T22:42:59Z

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

This PR changes MIR

cc @oli-obk, @RalfJung, @JakobDegen, @vakaras

This PR changes rustc_public

cc @oli-obk, @celinval, @ouz-a, @makai410

rustbot · 2026-06-15T22:43:01Z

dianqk is currently at their maximum review capacity.
They may take a while to respond.

- Changed `iter_intervals` to return `RangeInclusive` instead of `Range` - Added `clear_row`, `disjoint_rows` and `intersects_range` methods

This flag also has the effect of disabling DestinationPropagation, which is already covered by move elimination.

cjgillot

Can part of the code, in particular the liveness analysis be shared with the existing DestinationPropagation? Or is this meant to supersede it completely?

View changes since this review

cjgillot · 2026-06-21T17:11:02Z

+//
+// This pass has 2 outputs: a set of kill points that mark the last use
+// locations of locals and a per-block bitset indicating which locals are live
+// on entry to that block.


Can we use a named struct for this?

cjgillot · 2026-06-21T17:11:21Z

+    tcx: TyCtxt<'tcx>,
+    body: &mir::Body<'tcx>,
+    pass_name: Option<&'static str>,
+) -> (Vec<(Local, Location)>, IndexVec<BasicBlock, DenseBitSet<Local>>) {


BitMatrix<BasicBlock, Local>?

cjgillot · 2026-06-21T17:40:19Z

+        // Notably this kills any dead results produced by a predecessor's terminator.
+        state.intersect(&live_on_entry[block]);
+
+        for local in state.iter() {
+            builder.gen_(local, points.entry_point(block), SplitPointEffect::Early);
+        }
+
+        for (statement_index, statement) in block_data.statements.iter().enumerate() {
+            let location = Location { block, statement_index };
+            let point = points.point_from_location(location);
+
+            // StorageDead always kills a local, even if it has been borrowed.
+            if let mir::StatementKind::StorageDead(local) = statement.kind {
+                builder.kill(local, point, SplitPointEffect::Late);
+                continue;
+            }
+
+            // Kill moved operands if the whole local was moved.
+            VisitPlacesWith(|place: Place<'tcx>, ctxt| {
+                if ctxt == PlaceContext::NonMutatingUse(NonMutatingUseContext::Move) {
+                    if let Some(local) = place.as_local() {
+                        builder.kill(local, point, SplitPointEffect::Early);
+                    }
+                }
+            })
+            .visit_statement(statement, location);
+
+            // Kill any locals which are no longer used after this statement.
+            for &(local, _) in kill_point_map[point] {
+                builder.kill(local, point, SplitPointEffect::Early);
+            }
+
+            // Gen destination places.
+            VisitPlacesWith(|place: Place<'tcx>, ctxt| match DefUse::for_place(place, ctxt) {
+                DefUse::Def | DefUse::PartialWrite => {
+                    builder.gen_(place.local, point, SplitPointEffect::Late)
+                }
+                DefUse::Use | DefUse::NonUse => {}
+            })
+            .visit_statement(statement, location);
+
+            // Kill any dead destination places: they will only appear at
+            // the late point of the statement they are generated in, which is
+            // sufficient for determining overlap.
+            for &(local, _) in kill_point_map[point] {
+                builder.kill(local, point, SplitPointEffect::Late);
+            }
+        }
+
+        let location = Location { block, statement_index: block_data.statements.len() };
+        let point = points.point_from_location(location);
+        let terminator = block_data.terminator();
+
+        // Kill moved operands if the whole local was moved. Also kill dropped
+        // places if the entire local was dropped.
+        VisitPlacesWith(|place: Place<'tcx>, ctxt| {
+            if let PlaceContext::NonMutatingUse(NonMutatingUseContext::Move)
+            | PlaceContext::MutatingUse(MutatingUseContext::Drop) = ctxt
+            {
+                if let Some(local) = place.as_local() {
+                    builder.kill(local, point, SplitPointEffect::Early);
+                }
+            }
+        })
+        .visit_terminator(terminator, location);
+
+        // Kill any locals which are no longer used after this terminator.
+        for &(local, _) in kill_point_map[point] {
+            builder.kill(local, point, SplitPointEffect::Early);
+        }
+
+        // Gen destination places.
+        VisitPlacesWith(|place: Place<'tcx>, ctxt| match DefUse::for_place(place, ctxt) {
+            DefUse::Def | DefUse::PartialWrite => {
+                builder.gen_(place.local, point, SplitPointEffect::Late)
+            }
+            DefUse::Use | DefUse::NonUse => {}
+        })
+        .visit_terminator(terminator, location);
+
+        // Move arguments to a call are treated specially: the place that they
+        // represent is passed directly to the callee, which means that they are
+        // not allowed to alias any other move operand or the destination place.
+        // This is represented here by extending their live range to the late
+        // part, making it overlap with that of the destination place.
+        //
+        // Notably, this *doesn't* apply to TailCall.
+        if let mir::TerminatorKind::Call {
+            func: _,
+            args,
+            destination: _,
+            target: _,
+            unwind: _,
+            call_source: _,
+            fn_span: _,
+        } = &terminator.kind
+        {
+            for arg in args {
+                if let mir::Operand::Move(place) = arg.node {
+                    builder.gen_(place.local, point, SplitPointEffect::Late);
+                    builder.kill(place.local, point, SplitPointEffect::Late);
+                }
+            }
+        }


This duplicates a lot of code in impl Analysis for PreciseLiveness. Any way to deduplicate?

cjgillot · 2026-06-21T17:52:23Z

+            trace!("cannot unify {a:?} and {b:?} involving a rust-call tuple argument");
+            return None;
+        }
+


Can we remove a common projection tail from a and b before this match? Allowing to merge _1.foo.bar with _2.blah.foo.bar?

cjgillot · 2026-06-21T17:59:37Z

+    fn visit_aggregate_assign(
+        &mut self,
+        dest: Place<'tcx>,
+        project_field: impl Fn(TyCtxt<'tcx>, Place<'tcx>, FieldIdx, Ty<'tcx>) -> Place<'tcx>,


Should the callback return a &[PlaceElem<'tcx>] and call project_deeper here?

cjgillot · 2026-06-21T19:15:31Z

+                let Some(next) = next else {
+                    break;
+                };
+                state.block = next;
+            }


IIUC, the loop and this special case is meant to avoid cloning the state.used_after. What about just calling stack.push(TailState { block: next, state: state.used_after }), and letting the while let loop handle everything?

cjgillot · 2026-06-21T19:16:53Z

+            // Under the local lifetime semantics from RFC 3943, `StorageLive` does not allocate,
+            // and `StorageDead` has no effect if the local was already freed by a move. These
+            // markers therefore do not affect whether a copy can be treated as a final use.
+            StatementKind::StorageLive(_) | StatementKind::StorageDead(_) | StatementKind::Nop => {}


Why can't StorageDead(l) perform a used_after.remove(l)?

cjgillot · 2026-06-21T19:18:54Z

+                if !process_rvalue(&mut assign.1, used_after) {
+                    return false;
+                }


Suggested change

if !process_rvalue(&mut assign.1, used_after) {

return false;

}

return process_rvalue(&mut assign.1, used_after);

?

cjgillot · 2026-06-21T19:21:11Z

+    block: BasicBlock,
+    used_after: &mut DenseBitSet<Local>,
+    borrowed: &DenseBitSet<Local>,
+) -> bool {


Would it be better to return a Result or ControlFlow here? Just a bool is a bit obscure.

cjgillot · 2026-06-21T19:26:24Z

+            StatementKind::Assign(assign) => {
+                let dest = assign.0;


Could you untuple assign inside the match pattern?

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jun 15, 2026

rustbot assigned dianqk Jun 15, 2026

This comment has been minimized.

Sign in to view

Amanieu mentioned this pull request Jun 15, 2026

[WIP] MIR move elimination pass #156046

Closed

Amanieu added 7 commits June 15, 2026 23:46

Remove incorrect comments about Rvalue::Aggregate not being allowed

7cc4a3c

Pass block to visit_block_start and visit_block_end

28ea3e5

Various changes to SparseIntervalMatrix

bd561d1

- Changed `iter_intervals` to return `RangeInclusive` instead of `Range` - Added `clear_row`, `disjoint_rows` and `intersects_range` methods

Add precise MIR liveness analysis

5b8c927

Add MIR move elimination pass

f340209

Add a MIR pass which turns copies into moves just before a return

a20834e

Gate move elimination behind -Z mir-move-elimination

ca2c2ac

This flag also has the effect of disabling DestinationPropagation, which is already covered by move elimination.

Amanieu force-pushed the move-elimination branch from f12157e to ca2c2ac Compare June 15, 2026 22:47

cjgillot self-assigned this Jun 16, 2026

cjgillot reviewed Jun 21, 2026

View reviewed changes

Uh oh!

Conversation

Amanieu commented Jun 15, 2026

Uh oh!

rustbot commented Jun 15, 2026

Uh oh!

rustbot commented Jun 15, 2026

Uh oh!

This comment has been minimized.

cjgillot left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cjgillot left a comment •

edited by rustbot

Loading