Repository navigation
Description
Coming from the "missing required bounds" error?
The "missing required bounds" error that was emitted is intended to ensure that current code is forwards-compatible with potential future changes. We'll provide a brief summary of the why the bounds are required, the potential future changes, and a workaround if the required bounds are too restrictive. We would appreciate any feedback for this issue.
Why are these bounds required
Let's start by imagining a trait:
trait Iterable {
type Item<'x>;
fn iter<'a>(&'a self) -> Self::Item<'a>;
}As-is, nothing looks wrong about this; without the "missing required bounds" error, this would compile. However, let's try to add an impl:
impl<T> Iterable for T {
type Item<'a> = &'a T;
fn iter<'a>(&'a self) -> Self::Item<'a> { self }
}This definition of Item<'a> = &'a T is invalid, since we don't know that T outlives 'a. So, our first thought might be to modify the definition to add a where Self: 'a clause. This is what we want. However, impls are not allowed to have more where clauses than the trait.
Luckily, we can detect in a trait when this type of problem might occur. To do this, we look at the trait methods that construct the GAT. There, we find the bounds that we know and require that those bounds be written on the GAT itself.
Here, on the Iterable trait, we construct the GAT Self::Item<'a> in the iter method. There, we have an argument &'a self, which allows us to know Self: 'a. Therefore, we require the where Self: 'a bound on the GAT.
Potential future changes
Following the above logic, if we know all the required bounds when the trait is written, why require them at all? In fact, allowed these bounds to be implied is a potential future change. However, to make this change in a backwards-compatible manner, we must require the bounds now that we want to eventually imply. At that time, the written bounds will be redundant and can be removed.
This breaks my code. Workaround?
First, if any code breaks from adding the required bounds, we really want feedback. Second, the workaround is to move the GAT into a super trait. Using the example above, our new code would look like:
trait IterableSuper {
type Item<'x>;
}
trait Iterable: IterableSuper {
fn iter<'a>(&'a self) -> Self::Item<'a>;
}Previous discussion
What is this bug?
We are moving towards stabilizing GATs (tracking issue: #44265) but there is one major ergonomic hurdle that we should decide how to manage before we go forward. In particular, a great many GAT use cases require a surprising where clause to be well-typed; this typically has the form where Self: 'a. It might be useful if we were to create some rules to add this rule by default. Once we stabilize, changing defaults will be more difficult, and could require an edition, therefore it's better to evaluate the rules now.
I have an opinion! What should I do?
To make this decision in an informed way, what we need most are real-world examples and experience reports. If you are experimenting with GATs, for example, how often do you use where Self: 'a and how did you find out that it is necessary? Would the default proposals described below work for you? If not, can you describe the trait so we can understand why they would not work?
Of great use would be example usages that do NOT require where Self: 'a. It'd be good to be able to evaluate the various defaulting schemes and see whether they would interfere with the trait. Knowing the trait and a rough sketch of the impls would be helpful.
Background: what where clause now?
Consider the typical "lending iterator" example. The idea here is to have an iterator that produces values that may have references into the iterator itself (as opposed to references into the collection being iterated over). In other words, given a next method like fn next<'a>(&'a mut self), the returned items have to be able to reference 'a. The typical Iterator trait cannot express that, but GATs can:
trait LendingIterator {
type Item<'a>;
fn next<'b>(&'b mut self) -> Self::Item<'b>;
}Unfortunately, this trait definition turns out to be not quite right in practice. Consider an example like this, an iterator that yields a reference to the same item over and over again (note that it owns the item it is referencing):
struct RefOnce<T> {
my_data: T
}
impl<T> LendingIterator for RefOnce<T> {
type Item<'a> where Self: 'a = &'a T;
fn next<'b>(&'b mut self) -> Self::Item<'b> {
&self.my_data
}
}Here, the type type Item<'a> = &'a T declaration is actually illegal. Why is that? The assumption when authoring the trait was that 'a would always be the lifetime of the self reference in the next function, of course, but that is not in fact required. People can reference Item with any lifetime they want. For example, what if somebody wrote the type <SomeType<T> as LendingIterator>::Item<'static>? In this case, T: 'static would have to be true, but T may in fact contain borrowed references. This is why the compiler gives you a "T may not outlive 'a" error (playground).
We can encode the constraint that "'a is meant to be the lifetime of the self reference" by adding a where Self: 'a clause to the type Item declaration. This is saying "you can only use a 'a that could be a reference to Self". If you make this change, you'll find that the code compiles (playground):
trait LendingIterator {
type Item<'a> where Self: 'a;
fn next<'b>(&'b mut self) -> Self::Item<'b>;
}When would you NOT want the where clause Self: 'a?
If the associated type cannot refer to data that comes from the Self type, then the where Self: 'a is unnecessary, and is in fact somewhat constraining. As an example, consider:
XXX finish this
What could we do about it?
There are a few options. Here is the list of ideas we've had so far.
- Status quo: require that people add the
where Self: 'abounds, and try to do better with diagnostics. - Simple, limited default: If a GAT has exactly one lifetime parameter
'a, addwhere Self: 'ato both traits and impls. Need some way to opt out. - More extensive defaults: e.g., for every lifetime parameter
'ato a GAT, addwhere Self: 'a, and maybewhere T: 'afor type parameters too. Need some way to opt out. - Add a syntactic sugar for this common case, e.g.
type Foo<'self>. This could be added later. - self-oriented defaults: Given some GAT
Foo<'a>, if each useSelf::Foo<'b>within the trait methods references a'bthat is the lifetime parameter forself, then addwhere Self: 'b. While kind of complex to explain, this captures the intuition that'ais meant to be the "lifetime of the self reference" in practice. We probably still want a way to opt out (though maybe not; maybe that way is "don't use&'a selfnotation"). - Even smarter defaults A: Look at the method signatures in the trait. If we find that each use of
Self::Item<'b>is associated with a lifetime'bwhereSelf: 'bis implied by the method arguments, then infer thatwhere Self: 'b. This is a more extensive, general version of self-oriented defaults. We probably still want a way to opt out (though maybe not).
In general, once we stabilize GATs, we likely cannot add defaults, except via an edition -- although we may be able to get away with it in this instance if the defaults are smart enough.
Activity
nikomatsakis commentedon Jul 26, 2021
cc @rust-lang/lang
cramertj commentedon Jul 26, 2021
One of the ones that seems most obvious to me is when the output borrows not from self, but from a parameter to the method, such as in the case of a parser:
Parser output wouldn't usually reference state internal to the parser, but it very well may reference the input data. In practice, you can express this instead with
for<'a> Parser<'a>, but it's often the case that lifetime-only GATs can be replaced with HRTB-based solutions. I think it might be a goal of ours to allow people to avoid HRTBs for these types of things-- if it is, then this use-case is worth considering.jplatte commentedon Jul 27, 2021
For another example where
Self: 'awould be wrong, in Ruma we have a traitIncomingRequestthat I've long wanted to use GATs for, which would be another example of this. Simplified:(full definition here if sbd. is curious)
My 2c regardingthe potential solutions:
type Foo<'self>looks really nice.ShadowJonathan commentedon Jul 30, 2021
I'm also in favour of keeping status quo and adding the implicit
'selflifetime, it balances the need for being explicit with DRY, is easy to hint at by the compiler (for one scenario or another), and easy to reference by the book & other resources.In general I think the compiler shouldn't try to be "smart" about this, I'm still a learning rustacean, but having the compiler try to infer multiple lifetimes by default, and implicitly, seems confusing and footgun-y to me.
ammkrn commentedon Aug 2, 2021
I have a use case of the kind @cramertj referenced that I ended up working around with macros, but I would like to try and remove them when GATs are available. Also for cramertj, I couldn't figure out how to implement your example with HRTBs, so if you can actually do that I'd be interested to learn how it's done.
My specific use case is a little more complex, but I think the idea is relevant to many applications of indirect data structures. I have tree structures that are managed with type/memory safe integer pointers:
The backing storage struct looks like:
The program is a type checker, so there may be multiple stores alive with lifetimes 'a > 'b > .. 'n, but they nest, so we're always dealing with the "outermost" one (with the shortest lifetime). The guarantees I'm currently able to enforce within Rust's type system are that a
Ptr<'a, A>can't outlive its backingStore<'a>, and that you can only create aPtr<'a, A>if you actually have aStore<A>. The macro-based solution runs really well, but GATs would be much more satisfying. The GAT solution I originally wanted was to have a trait for the stored types so that the output's lifetime was dependent on the lifetime of the backing store that was passed as an argument. As long as the underlying data is alive at least as long as the backing store I've been passed, I can both read aPtr<'a, A> -> Aand allocate anA -> Ptr<'a, A>.In this case, it seems like
type Carrier<'a> where Self: 'awould be incorrect, but I wrote up a basic implementation of it with the syntax proposed here and it seems to compile on nightly, I guess because the actual self lifetime is longer than the 'a bound, though I'm sure there are programs that would benefit from similar indirect structures in which that wouldn't always be the case.Also I have no idea if this is possible, but in playing around with this feature with the thought of making stuff generic over lifetimes, it would be cool if there was a way to add sugar or something to enable this:
P-E-Meunier commentedon Aug 6, 2021
GATs are probably the only feature I've been missing in Rust since 1.0, I'm delighted to see progress on that.
@nikomatsakis, I do have an example where
Self: 'aisn't always the case: in Sanakirja, an on-disk key-value store I wrote, memory management is a bit messy. I'd also like to extend it by implementing other transactional datastructures in the future.Since I don't know the kind of details yous need, here is a quick summary of how Sanakirja works:
db0is of some size given by the user, then when we run out of space in the middle of a transaction we startdb(n+1), which is twice the size ofdb(n)). This trick allows us to never unmap the file in the middle of a transaction. Some of the pointers (explained below) may still be valid after an increase.db0(of a size set when creating the database) is dedicated to independent versions of the database, and only the latest version is writable, and writable by a single (mutable) transaction. The only concurrency control is a simple read-write lock on each of these memory pages.Now, there's a choice about the lifetime of values retrieved from the database: we may choose to have them borrow the transactions, or to have them borrow individual tables. In the former case, a GAT may look like:
In the latter case, we'll have the following instead:
This could work with an extra mechanism to make sure
Dbisn't alive after the end of a transaction, for example.As a conclusion, I don't have a strong opinion about what the default should be, since I understand this is a rather niche use case. I haven't actually tried to implement these using nightly + GAT, but I could try if yous need more feedback.
eira-fransham commentedon Aug 10, 2021
I feel like you're always going to want to allow
where Self: 'aeven if you don't always require it, so why not stabilise it with the requirement that you must always annotate and add elision later as a separate RFC? That's what we did for lifetime elision in impl specs (e.g.impl Trait for &'_ Foo). This is now, as far as I know, the only remaining blocker for GATs being stabilised so it seems silly to hold that up while we bikeshed this considering that there exists a MVP solution that would be trivial to improve on later in a backwards-compatible way.nikomatsakis commentedon Aug 24, 2021
Hi all! I'm reading these responses and trying to summarize.
@P-E-Meunier what would be useful is to see not only the trait definition, but prospective impl definitions
@ammkrn the same! What does an impl for the trait
IsStoredlook like? You mentioned that you prototyped your setup on nightly, can you link to that on the Rust playground perhaps?ammkrn commentedon Aug 25, 2021
@nikomatsakis
I put it in a
gat_nightlybranch, here are the impls for one base type and its pointer.https://github.com/ammkrn/nanoda_lib/blob/4c46ec08cb80e12de1cf5a312f17c3fd67feb228/src/utils.rs#L370
https://github.com/ammkrn/nanoda_lib/blob/4c46ec08cb80e12de1cf5a312f17c3fd67feb228/src/utils.rs#L409
This application's state management is somewhat involved for performance reasons, but I think this issue boils down to what I believe is a GAT use case that's been discussed quite a bit, which is generalizing over the lifetime of a container.
ctxis the telescope of stores,'ais the lifetime of the outermost/shortest-lived backing store, and the one we have mutable access to.'xis any lifetime equal to or longer than'a. The two ideas are: "given a pointer to an object known to live at least as long as the shortest-lived store, I can return the pointed-to data, though I can only guarantee it lives as long as the shortest-lived store", and "given some data, I can allocate it and return a pointer, but I can only promise the pointer is good for the lifetime of the shortest-lived store".or:
read:
∀ 'x >= 'a, (Ptr<'x>, Store<'a>) -> Data<'a>alloc:
∀ 'x >= 'a, (Data<'x>, Store<'a>) -> Ptr<'a>jgarvin commentedon Aug 29, 2021
I ran into a simple case using GATs where I didn't need the clause. I have a trait representing mutable pointers so that I can write generic functions that work both with regular pointers and compressed pointers. Since it's for pointers, there are no lifetimes.
This is analogous in C++ to
std::static_pointer_castand related functions which work both on regular pointers and smart pointers likeshared_ptrandunique_ptr. One problem that still exists with this is thatMutPointer<U>could still be some other smart pointer type when it should really be the same generic with a different type plugged in (in other words you shouldn't be able to implement the trait such thatshared_ptr<T>is castable tounique_ptr<U>, onlyshared_ptr<U>). I think doing it right might require HKTs. Something like:dbdeviant commentedon Sep 11, 2021
I recently came across a scenario where I didn't need the clause. The library performs async serialization for any given user provided format (basically async serde). In my case, only the serializer trait needed the
Selflifetime annotation, as the returned future from the method holds on to a reference to the data until complete. The deserializer owns its own internal buffers, and as such just defines an associated function for which noSelfreference is needed.In this case, it would seem that option 6 would be necessary to properly infer the type. (Full disclosure: new rustacean here, so there might be a better way to do this.)
My type definitions:
madsmtm commentedon Oct 8, 2021
I have a use case (that may be similar to the others people have noted, if so, apologies).
A sealed marker trait
Ownershipwith two implementorsOwnedandSharedis used inobjc_idinside a reference-counted typeId<T, O: Ownership>, to specify whether you have mutable or immutable access to the data.In some cases I would like to consume the
Idand return the appropriate reference (bound to some other type, in this caseAutoreleasePool); either an&mut TforOwneddata or&TforShareddata.This doesn't require
Self: 'a.nikomatsakis commentedon Oct 8, 2021
Thanks @madsmtm! I believe your use case will work fine with the current plan.
57 remaining items
Kixunil commentedon Jan 20, 2024
I struggle to understand why not having the
wherebound is an issue with GAT but not with this trick I use to simulate GAT in older MSRVs:I don't see anything telling the compiler that
Self: 'ayet AFAICT this works perfectly fine. I've even implemented a macro that can use GAT syntax to impl the trait (so that once MSRV bumps it'll be a simple change). Is there a problem lurking there that I will hit eventually? Also why did the GAT work take so much time if it looks like it could be just a syntax sugar for what I wrote? I must be missing something.QuineDot commentedon Mar 23, 2024
@Kixunil There are situations where your pattern fails because the implementation of
EncodeTc<'a>requiresSelf: 'a.For example:
Can't work without the bound:
So you add the bound...
...but now the HRTB can't hold due to the extra constraint (which is not present in the HRTB).
(intermediate playground)
You can work around this like so:
And now the HRTB can be met, as the HRTB honors the implied bound.
The implied bound in the default type parameter is telling the compiler that
Self: 'a. With GATs, you use awhereclause on the GAT instead.Auto merge of #96709 - jackh726:gats-stabilization, r=compiler-errors
Auto merge of #96709 - jackh726:gats-stabilization, r=compiler-errors
ecryth commentedon Nov 17, 2024
I've also run into a case where this bound being required is causing issues. It's similar to the HRTB issues above, but in my case it's causing the code to fail to build outright, with no easy solution that I can see beyond the mentioned workaround (splitting the trait into two) or giving up on GATs altogether (moving the lifetime to a trait parameter).
Minimised example (playground):
Background motivation (playground):
I have some AST nodes used for code generation. There are two levels of the AST, the low-level AST that handles the code generation itself; below simplified:
and then a more strongly-typed higher-level AST which can be transformed into the lower-level AST; simplified:
(The purpose is to let the higher-level AST ignore certain syntactic details, and to allow multiple low-level backends for the same high-level AST.)
In order to be able to implement this
Add::make_nodemethod, we need to ensure thatTandUgenerate expression nodes, so to simplify matters my idea was to create a subtrait which is automatically implemented for all implementors ofMakeNodewhere the node type is anExprNode:However, the presence of the HRTB in the supertrait bound for
MakeExprNoderesults in this less-than-helpful error:I think this is a limitation of HRTBs applied to bounded GATs, where the upper bound of the HRTB doesn't take the
Self: 'bconstraint into account. But this issue is exacerbated by the compiler's insistence on the presence of the bound.If I use the suggested workaround and split the
MakeNodetrait in two and remove theSelf: 'abound, the definition ofMakeExprNodeworks as-is, and I can implement theMakeNodetrait forAddwithout issue:But it's very cumbersome to have to implement two traits for every node. And in my case all my GATs are concrete types known to live for
'a, meaning I have no need for theSelf: 'abound to begin with, so it would be nice if there was a less verbose way of opting out of the requirement.RishabhRD commentedon Feb 22, 2025
I feel that
where Self: 'acan become overly restrictive when used with HRTB.For example, consider we have a collection trait that supports forward iteration and slicing where slice is a collection itself:
Now for supporting a collection that supports backward iteration:
Here the intent is slice should be BidirectionalCollection. However, this doesn't work as intended. The problem is, this leads to deriving
staticlifetime for BidirectionalCollection asfor<'a>also includesfor<'static>.Now the solutions can be (and I guess this is well known) to support
for<'a where Self: 'a>syntax.Another way can be:
The lifetime check for
Item<'a>should be done only for use side i.e., let's say when someone returnsItem<'a>from a function and not on associated type declaration time itself. This would makeItemjust a type constructor and responsibility of lifetime check would be carried to use site. (I am not a language internal expert, so I am not sure how feasible this is.)Without any of these, lifetime GATs would not be much usable especially for generic programming purposes.
Move lifetimes around
Auto merge of #96709 - jackh726:gats-stabilization, r=compiler-errors
refactor(storage): split IterRead, decouple reverse
refactor(storage): split IterRead, decouple reverse