Rust

6035 readers

3 users here now

Welcome to the Rust community! This is a place to discuss about the Rust programming language.

Wormhole

[email protected]

Credits

The icon is a modified version of the official rust logo (changing the colors to a gradient and black background)

founded 1 year ago

MODERATORS

Rust Optimization: Vec::into_iter().collect() (self.rust)

submitted 7 months ago by arendjr to c/rust

15 comments fedilink hide all child comments

I just had a random thought: a common pattern in Rust is to things such as:

let vec_a: Vec<String> = /* ... */;
let vec_b: Vec<String> = vec_a.into_iter().filter(some_filter).collect();

Usually, we need to be aware of the fact that Iterator::collect() allocates for the container we are collecting into. But in the snippet above, we've consumed a container of the same type. And since Rust has full ownership of the vector, in theory the memory allocated by vec_a could be reused to store the collected results of vec_b, meaning everything could be done in-place and no additional allocation is necessary.

It's a highly specific optimization though, so I wonder if such a thing has been implemented in the Rust compiler. Anybody who has an idea about this?

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 7 points 7 months ago* (last edited 7 months ago) (7 children)

map() can still be used with Vec::iter_mut(), filter_map() can be replaced with Vec::retain_mut().

[–] arendjr 4 points 7 months ago* (last edited 7 months ago) (6 children)

Yeah, that's helpful if I would be currently optimizing a hot loop now. But I was really just using it as an example. Also, retain_mut() doesn't compose as well.

I'd much rather write:

let vec_a: Vec<String> = /* ... */;
let vec_b: Vec<String> = vec_a
    .into_iter()
    .filter(some_filter)
    .map(some_map_fn)
    .collect();

Over:

let mut vec_a: Vec<String> = /* ... */;
vec_a.retain_mut(|x| if some_filter(x) {
    *x = some_map_fn(*x); // Yikes, cannot move out of reference.
    true
} else {
    false
});

And it would be nice if that would be optimized the same. After all, the point of Rust's iterators is to provide zero-cost abstractions. In my opinion, functions like retain_mut() represent a leakiness to that abstraction, because the alternative turns out to not be zero cost.

[–] porgamrer 4 points 7 months ago (2 children)

Is it really fair to say retain doesn't compose as well just because it requires reference-based update instead of move-based? I also think using move semantics for in-place updates makes it harder to optimise things like a single field being updated on a large struct.

It also seems harsh to say iterators aren't a zero-cost abstraction if they miss an optimisation that falls outside what the API promises. It's natural to expect collect to allocate, no?

But I'm only writing this because I wonder if I haven't understood your point fully.

(Side note: I think you could implement the API you want on top of retain_mut by using std::mem::replace with a default value, but you'd be hoping that the compiler optimises away all the replace calls when it inlines and sees the code can't panic. Idk if that would actually work.)

[–] arendjr 1 points 7 months ago

The composability doesn't have much to do with whether it's a reference or a move, it's because it bypasses usage of the Iterator methods. Iterators chains can consist of filter, map and other operations, which allows various functions and/or closures to be composed together. Whereas with retain_mut() there isn't really a chain and functions you may otherwise use in an iterator chain become harder to (re)use.

load more comments (1 replies)

load more comments (4 replies)