{ "@context": "https://schema.org", "@id": "https://aggr.aymericbeaumet.com/items/lobste-rs/2026-09-29-pining-for-arc-downcasting-in-rust/#webpage", "@type": [ "WebPage", "ArchiveComponent" ], "articleBody": "In the course of writing my [build driver](https://git.sr.ht/~polywolf/driver), I came across a bit of an unusual problem, for which I made a bit of an usual solution. I think the solution is interesting and would like to talk about it, but to understand anything we must first understand the problem at hand.\n\n## Consider The Case Of The Humble Concurrent Cache\n\nSuppose we have some expensive function we'd like to put a cache in front of. Furthermore, suppose we'd like to access this cache from multiple threads. A simple example follows ([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=cf4e4af4e40d6895d16d12779f395fa0)):\n\n```rust\nenum JSON {\n F64(f64),\n String(String),\n Vec(Vec),\n Object(HashMap),\n}\n\nstruct Proxy {\n client: HTTPClient,\n cache: RWLock>,\n}\n\nimpl Proxy {\n fn get(&self, key: &str) -> JSON {\n // 1.\n {\n let cache = self.cache.read().unwrap();\n if let Some(value) = cache.get(key) {\n return value.clone();\n }\n }\n\n // 2.\n let value = self.client.get(key);\n {\n let mut cache = self.cache.write().unwrap();\n cache.insert(key.to_string(), value.clone());\n }\n value\n }\n}\n```\n\nThis code has an \"early exit\" path (1) where it returns a value from the cache if it's present, and a \"late exit\" path (2) where it calls the expensive function, then inserts the resulting value into the cache.\n\nPlease ignore the many, many obvious problems with this implementation[^1]. Instead, let's focus on the one problem that bothers me the most: there's fair bit of `.clone()` action going on here!\n\nTechnically, it's just one `.clone()` per call: one on early exit to take value out of the cache, and one on late exit to put value into the cache. But if those values are big/tree-shaped/otherwise expensive to clone, this cost can dominate, minimizing the savings conferred by a cache. In my code, I found this to be the case, so we gotta do something about it.\n\n## Let's Get Rid Of The Clones?\n\nAssume that, with the way we use this data, read-only access is more than enough. Shared references are read-only & cheap to `Copy`, so using those instead of `.clone()`-ing the entire value seems good. If some later part of the code really needs to take ownership, we can just `.clone()` there, saving time in the average case. So, we'd like to change the signature for `get()` to be:\n\n```rust\nimpl Proxy {\n fn get<'a, 'b>(&'a self, url: &'b str) -> &'a JSON { ... }\n}\n```\n\nBut we can't do this!! Because our cache is behind a mutex, the only way we can get references to its contents is thru temporary handles. Those handles, while live, hold a lock on the cache, plus they only live for the body of the function, a not for all of `'a`. Even if we *could* return one of those handles, that'd be equivalent to holding the lock outside the function, which is very bad. Locks should only be held for VERY SHORT amounts of time unless ur into that sorta thing next month ;)\n\n## Let's Make The Clones Cheaper\n\nSo, no references. What other types can we use? What we want is something with all the following properties:\n\n1. It allows for read access to our data. That is, it allows us to get an `&JSON` somehow.\n2. It has no lifetime parameters (`'a`, the only lifetime we have access to, is too long).\n3. It is cheap to `.clone()`, even if the underlying value is *not* cheap to `.clone()`.\n\nThese requirements hint we should probably still be looking for some sort of pointer... Among standard library types, we have the following options[^2]:\n\n- [Raw pointers](https://doc.rust-lang.org/std/primitive.pointer.html): `*const JSON`\n- [Reference-counted pointers](https://doc.rust-lang.org/std/rc/index.html): `Rc`\n- [Atomically-reference-counted pointers](https://doc.rust-lang.org/std/sync/struct.Arc.html): `Arc`\n\nLike any good Rustacean, we care a lot about safety & concurrency, so `Arc` is the obvious pick here :3 Modifying the example to use it is straightforward ([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=f3e3821009472761d2297690f6262e17)):\n\n```rust\nstruct Proxy {\n client: HTTPClient,\n cache: RWLock>>, // new!\n}\n\nimpl Proxy {\n fn get(&self, url: &str) -> Arc { // new!\n {\n let cache = self.cache.read().unwrap();\n if let Some(value) = cache.get(url) {\n return value.clone();\n }\n }\n\n let value = Arc::new(self.client.get(url)); // new!\n {\n let mut cache = self.cache.write().unwrap();\n cache.insert(url.to_string(), value.clone());\n }\n value\n }\n}\n```\n\nOther than the three lines with `Arc` added to them, this implementation looks the exact same as before. But now our clones are cheaper, so we're happy, yay!!\n\n## So What's This About Downcasting?\n\nI hope the above section convinced you having an `Arc` \"owned value that acts like a reference\" is both normal to want & possible to achieve. Switching gears a bit, I'd like to discuss an interesting shortcoming with them: they don't fit into Rust's type system very well.\n\nSupposed we know for a fact that certain `JSON` values are strings, and we're only interested in the `JSON::String` variant of them. With an owned value or a shared reference, we can just pattern-match to \"downcast\" from a `JSON` to a `String`, or a `&JSON` to a `&String`.\n\n```rust\nimpl JSON {\n fn to_str(self) -> Option {\n match self {\n Self::String(s) => Some(s),\n _ => None,\n }\n }\n \n fn as_str(&self) -> Option<&String> {\n match self {\n Self::String(s) => Some(s),\n _ => None,\n }\n }\n}\n```\n\nBut if we have an `Arc`, we can't get another `Arc` the same way!\n\n```rust\nimpl JSON {\n fn doesnt_exist(value: Arc) -> Option> {\n match value.deref() {\n Self::String(s) => Some(s), // compile error!\n _ => None,\n }\n }\n}\n```\n\nThis is because `value.deref()` creates a reference to `value`, whose lifetime will end as soon as the function is over, because we don't return it, only a pointer somewhere inside it. The machinery for `Arc` only works if it has access to the original pointer, not any derived pointers. So if we wanted to return an `Arc`, we'd need to `.clone()` out of the `Arc`, which is what we've been trying to avoid this whole time.\n\nHowever! We don't necessarily *need* a full `Arc`! We'd be perfectly happy returning some other type, perhaps implementing `Deref`, so long as it still gives us those \"owned value that acts like a reference\" properties. If only we could extend the lifetime of `value`, perhaps by returning it alongside a reference to its contents, packaged together to implement `Deref` like we want...\n\n## Tying The Two Together With Evil Lesbian Shibari\n\nOur goal is some return type that looks like:\n\n```\n ,-----------------------+---------------------.\nval: owned --> | contents: *const JSON | refcnt: AtomicUsize |\n `-----------------------+---------------------'\n |\n /--------------------------/\n V\n ,------------------+--------------+------------+------------.\n | JSON::String tag | buf: *mut u8 | len: usize | cap: usize |\n `------------------+--------------+------------+------------'\n ^ |\n | |\nptr: ref --------------------/ |\n V\n ,---+---+---+---.\n |'A'|'C'|'A'|'B'|\n `---+---+---+---'\n```\n\nThat is, we want some `val` showing us how to get to the main value we care about, and then some pointer `ptr` into the memory `val` references. Then, as long as we keep those tied together, we know `ptr` will still be valid, because `val` is still alive, because we own `val`.\n\nA first attempt at writing this reveals an immediate issue[^3]\n\n```rust\nstruct Ref {\n val: V,\n ptr: &T, // What's the lifetime here?\n}\n```\n\nWe can't express `ptr` as a reference, because there's no obvious lifetime to attach it to. Without a way to spell \"lifetime of the containing struct\" in Rust, it looks like we're going to need a raw pointer instead. But what if.....\n\n```rust\nstruct Ref {\n val: V,\n ptr: &'static T,\n}\n\nimpl> {\n fn new(val: V) -> Self {\n let ptr: &T = val.deref();\n Self {\n // This is the easiest way to do lifetime extension\n // SAFETY: hm?\n ptr: unsafe { std::mem::transmute(ptr) },\n val,\n }\n }\n}\n```\n\nWhoa!! That's scary!!! Are they even allowed to hold hands like that...?\n\nIt's true this is exceedingly unsafe if users could extract that `ptr: &'static T` separately from the `val: V` it points into (`val`'s lifetime isn't `'static`!). But we could also just... not allow that, keeping them tied together always, providing access only via `Deref` implementation:\n\n```rust\nimpl Deref for Ref {\n type Target = T;\n fn deref(&self) -> &T {\n self.ptr\n }\n}\n```\n\nBecause the *effective* lifetime for which `ptr` can be accessed is a subset of the *actual* lifetime for which `val` lives, I believe we've properly rules-lawyered Rust's reference aliasing rules into submission. Or have we...\n\nOh noes... ([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=52455c2042bd9bc26f1bbbb8a239f9a4))\n\n```rust\nstruct SimpleWrapper(T);\n\nimpl Deref for SimpleWrapper {\n type Target = T;\n fn deref(&self) -> &Self::Target {\n &self.0\n }\n}\n\nfn main() {\n let r = Ref::new(SimpleWrapper(x));\n // Check what's stored vs what should be returned\n let ptr = r.ptr as *const i32 as usize;\n let actual_ptr = r.val.deref() as *const i32 as usize;\n println!(\"ptr: {ptr:x} actual_ptr {actual_ptr:x}\");\n}\n```\n\nRunning this, I got `ptr: 7fff85189b5c actual_ptr 7fff85189b88`. These are in fact different pointers!!! Turns out I messed up my earlier rules-lawyering: The act of moving `val` into `Ref::new()`, taking the `.deref()` on that stack frame, and then moving it back out to the parent stack frame invalidates `ptr`[^3]. Lifetimes exist precisely to prevent bugs like this, and our extension trick was foiled. Lesson learned! Guess we'll do this the hard way...\n\n## Ensure Address Stability With This One Simple Trick!\n\nTo fix our datastructure, we'll want a guarantee that each `.deref()` will give us the same pointer, even if move the container around. For this, we MUST NOT be able to move the `val: V` out of its location once we wrap it. Fortunately, Rust has a type exactly for this usecase!\n\nahem. anyways. Unfortunately, `Pin` is very hard to use, to the point I found a flaw in my initial implementation[^4] while writing this :( Still, the docs are *really* good, and we can pretty easily follow their example to make [a self-referential struct](https://doc.rust-lang.org/std/pin/index.html#a-self-referential-struct):\n\n```rust\nstruct MustPin {\n val: V,\n _pin: PhantomPinned,\n}\nstruct PinRef {\n val: Pin>>,\n // MUST point into [`val`].\n ptr: *const T,\n}\n\nimpl MustPin {\n fn new(val: V) -> Pin> {\n Arc::pin(Self {\n val,\n _pin: PhantomPinned,\n })\n }\n}\n\nimpl PinRef {\n fn new(val: Pin>>) -> Self {\n let ptr = &raw const val.val;\n Self { val, ptr }\n }\n}\n```\n\nComparing this to the example in the docs:\n\n1. We use `*const T` instead of `NonNull` because the latter is more like a `*mut T`, and we don't need all that power.\n2. We don't need `MaybeUninit` because we solve the \"knot-tying\" trick in a different way: we create the pinned data first, and then store a pointer into it out-of-line. This is still fine because of pin guarantees.\n3. We still need `PhantomPinned` because if we have `Pin>` where `V: Unpin`, all bets are off, literally every pin guarantee goes out the window.\n\n`Deref` is simple like before, just with a pointer instead of a reference:\n\n```rust\nimpl Deref for PinRef {\n type Target = T;\n fn deref(&self) -> &Self::Target {\n // SAFETY: by construction and pin guarantees, the pointer is still valid.\n unsafe { &*self.ptr }\n }\n}\n```\n\nNow, finally, we're all set up for the big reveal: how are we going to downcast these things?\n\n## She Downcast On My `Pin` 'Til I `Arc`\n\nOur rule for `ptr` is that it MUST point somewhere valid inside `val`. That's all we can assume, and that's what we have to uphold while doing our downcasts. Fortunately, we can use Rust's type-checking for \"standard\" downcasts to our advantage!\n\n```rust\nimpl PinRef {\n fn project(self, f: impl for<'a> FnOnce(&'a T) -> &'a U) -> PinRef {\n let Self { val, ptr } = self;\n // SAFETY: by validity of `ptr` and `f`\n let ptr = unsafe { f(&*ptr) as *const U };\n PinRef { val, ptr }\n }\n}\n```\n\nStating this signature more in more math-y terms, for those unfamiliar with Rust's syntax:\n\n$\\begin{gathered} \\text{PinRef} : (\\text{Type}, \\text{Type}) \\rightarrow \\text{Type}\\\\ \\forall\\ V,T,U : \\text{Type}.\\\\ \\text{project} : \\text{PinRef}(V,T) \\rightarrow (\\forall(a: \\text{Lifetime}).\\ \\&'a T \\rightarrow \\&'aU) \\rightarrow \\text{PinRef}(V,U) \\end{gathered}$\n\nHow we should interpret this is: If we can go from a `&T` to a `&U` for an *arbitrary* lifetime `'a`, that means `*U` is a fixed offset from `*T`[^5]. So, because `ptr` has a fixed address (it was derived from the pinned `val`), so will the output of `f`. Other pin guarantees like \"`val` will always remain valid at that address while it's pinned\" help too.\n\nIf I were a real type theorist, I would have pulled out some sort of commutative diagram and drawn a bunch of arrows, or perhaps even written down some inference rules, but alas, I cannot even abstract over monads... Anyways this argument works for what we originally wanted too:\n\n```rust\nimpl PinRef {\n fn filter_project(\n self,\n f: impl for<'a> FnOnce(&'a T) -> Option<&'a U>,\n ) -> Option> {\n let Self { val, ptr } = self;\n // SAFETY: by validity of `ptr`, `f`\n let ptr = unsafe { f(&*ptr)? as *const U };\n Some(PinRef { val, ptr })\n }\n \n fn try_project(\n self,\n f: impl for<'a> FnOnce(&'a T) -> Result<&'a U, E>,\n ) -> Result, E> {\n let Self { val, ptr } = self;\n // SAFETY: by validity of `ptr`, `f`\n let ptr = unsafe { f(&*ptr)? as *const U };\n Ok(PinRef { val, ptr })\n }\n}\n```\n\nYou see that??? We did the thing!! To celebrate, here's a full example using the original JSON projections ([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=9ba8b6f27dc48afb065a2923023fb1da)):\n\n```rust\nfn print(s: impl Deref) {\n println!(\"{}\", s.deref())\n}\n\nfn main() {\n let v = std::sync::Arc::pin(JSON::String(String::from(\"hello, world!\")));\n let v = PinRef::new(v);\n let s = v.filter_project(JSON::as_str).unwrap();\n print(s.clone());\n print(s);\n}\n```\n\nAll that remains in our original example is to replace all the plain `Arc` with `Pin>>` (wow what a mouthful), make a `Clone` implementation, account for `?Sized` types, etc. etc. This post is long enough as it is so I've omitted that, but if you want, you can find the full details in my [repository](https://git.sr.ht/~polywolf/driver/tree/cb9ee31e272f24311981a41f979d82f31f39f61f/item/packages/pin-downcast/src/pin_ref.rs). I might release this as a standalone crate if I feel like it, but this might still be riddled with UB I missed so maybe not (:\n\nAnyways!! Hope you learned something, until next time~\n\n[^1]: In increasing order of badness: too much string typing, no error handling, concurrent requests can race and end up doing extra work. Probably others I'm missing too. The solution to that last one is simultaneously very interesting & very boring, [read the code yourself if you want](https://git.sr.ht/~polywolf/driver/tree/3e0e9250a6657f9e5ed977ecdd08f3550e06f68b/item/packages/driver-db/src/database.rs#L150-270).\n\n[^2]: I'm only covering options from the Rust standard library for simplicity, but garbage-collected pointers from [`dumpster`](https://crates.io/crates/dumpster) or arena pointers from [`slotmap`](https://crates.io/crates/slotmap) can also be good ideas.\n\n[^3]: You might be thinking, \"why not `struct Ref { val: Arc, ptr: &'static T}`?\" and unfortunately a refutation is much more complex, and this example is more illustrative of why we need `Pin` later. Suffice to say, even though `Arc` on its own gives address stability in practice, Rust's type system doesn't enforce that it will [4](https://wolfgirl.dev/blog/2026-09-29-pining-for-arc-downcasting-in-rust/#fn-other-idea).\n\n[^4]: I previously thought `struct PinRef { val: Pin>, ptr: &'static T }` was enough, but turns out that's entirely insufficient due to the presence of `Unpin`. [↩2](https://wolfgirl.dev/blog/2026-09-29-pining-for-arc-downcasting-in-rust/#fnref-other-idea-2)\n\n[^5]: \"Arbitrary\" is key here. Means we can't do `fn project<'a, U>(self, f: impl FnOnce(&'a T) -> &'a U) -> PinRef`, because `'a` is bound too early, which would allow us to choose a smaller lifetime, letting us project things w/ interior mutability, which is bad. Wish I could formalize this better but I've thought about it really really hard and haven't been able to come up with a counterexample to my main function so I hope no one else will either.\n", "authors": [ "wolfgirl.dev by polywolf" ], "captured_at": "2026-10-01T06:33:51.242437607+00:00", "category": "news", "content_html": "

In the course of writing my build driver, I came across a bit of an unusual problem, for which I made a bit of an usual solution. I think the solution is interesting and would like to talk about it, but to understand anything we must first understand the problem at hand.

\n

Consider The Case Of The Humble Concurrent Cache

\n

Suppose we have some expensive function we'd like to put a cache in front of. Furthermore, suppose we'd like to access this cache from multiple threads. A simple example follows (playground link):

\n
enum JSON {\n    F64(f64),\n    String(String),\n    Vec(Vec<JSON>),\n    Object(HashMap<String, JSON>),\n}\n\nstruct Proxy {\n    client: HTTPClient,\n    cache: RWLock<HashMap<String, JSON>>,\n}\n\nimpl Proxy {\n    fn get(&self, key: &str) -> JSON {\n        // 1.\n        {\n            let cache = self.cache.read().unwrap();\n            if let Some(value) = cache.get(key) {\n                return value.clone();\n            }\n        }\n\n        // 2.\n        let value = self.client.get(key);\n        {\n            let mut cache = self.cache.write().unwrap();\n            cache.insert(key.to_string(), value.clone());\n        }\n        value\n    }\n}\n
\n

This code has an "early exit" path (1) where it returns a value from the cache if it's present, and a "late exit" path (2) where it calls the expensive function, then inserts the resulting value into the cache.

\n

Please ignore the many, many obvious problems with this implementation1. Instead, let's focus on the one problem that bothers me the most: there's fair bit of .clone() action going on here!

\n

Technically, it's just one .clone() per call: one on early exit to take value out of the cache, and one on late exit to put value into the cache. But if those values are big/tree-shaped/otherwise expensive to clone, this cost can dominate, minimizing the savings conferred by a cache. In my code, I found this to be the case, so we gotta do something about it.

\n

Let's Get Rid Of The Clones?

\n

Assume that, with the way we use this data, read-only access is more than enough. Shared references are read-only & cheap to Copy, so using those instead of .clone()-ing the entire value seems good. If some later part of the code really needs to take ownership, we can just .clone() there, saving time in the average case. So, we'd like to change the signature for get() to be:

\n
impl Proxy {\n    fn get<'a, 'b>(&'a self, url: &'b str) -> &'a JSON { ... }\n}\n
\n

But we can't do this!! Because our cache is behind a mutex, the only way we can get references to its contents is thru temporary handles. Those handles, while live, hold a lock on the cache, plus they only live for the body of the function, a not for all of 'a. Even if we could return one of those handles, that'd be equivalent to holding the lock outside the function, which is very bad. Locks should only be held for VERY SHORT amounts of time unless ur into that sorta thing next month ;)

\n

Let's Make The Clones Cheaper

\n

So, no references. What other types can we use? What we want is something with all the following properties:

\n
    \n
  1. It allows for read access to our data. That is, it allows us to get an &JSON somehow.
    2. \n
  2. It has no lifetime parameters ('a, the only lifetime we have access to, is too long).
    3. \n
  3. It is cheap to .clone(), even if the underlying value is not cheap to .clone().
    4. \n
\n

These requirements hint we should probably still be looking for some sort of pointer... Among standard library types, we have the following options2:

\n\n

Like any good Rustacean, we care a lot about safety & concurrency, so Arc is the obvious pick here :3 Modifying the example to use it is straightforward (playground link):

\n
struct Proxy {\n    client: HTTPClient,\n    cache: RWLock<HashMap<String, Arc<JSON>>>, // new!\n}\n\nimpl Proxy {\n    fn get(&self, url: &str) -> Arc<JSON> { // new!\n        {\n            let cache = self.cache.read().unwrap();\n            if let Some(value) = cache.get(url) {\n                return value.clone();\n            }\n        }\n\n        let value = Arc::new(self.client.get(url)); // new!\n        {\n            let mut cache = self.cache.write().unwrap();\n            cache.insert(url.to_string(), value.clone());\n        }\n        value\n    }\n}\n
\n

Other than the three lines with Arc added to them, this implementation looks the exact same as before. But now our clones are cheaper, so we're happy, yay!!

\n

So What's This About Downcasting?

\n

I hope the above section convinced you having an Arc "owned value that acts like a reference" is both normal to want & possible to achieve. Switching gears a bit, I'd like to discuss an interesting shortcoming with them: they don't fit into Rust's type system very well.

\n

Supposed we know for a fact that certain JSON values are strings, and we're only interested in the JSON::String variant of them. With an owned value or a shared reference, we can just pattern-match to "downcast" from a JSON to a String, or a &JSON to a &String.

\n
impl JSON {\n    fn to_str(self) -> Option<String> {\n        match self {\n            Self::String(s) => Some(s),\n            _ => None,\n        }\n    }\n    \n    fn as_str(&self) -> Option<&String> {\n        match self {\n            Self::String(s) => Some(s),\n            _ => None,\n        }\n    }\n}\n
\n

But if we have an Arc<JSON>, we can't get another Arc<String> the same way!

\n
impl JSON {\n    fn doesnt_exist(value: Arc<JSON>) -> Option<Arc<String>> {\n        match value.deref() {\n            Self::String(s) => Some(s), // compile error!\n            _ => None,\n        }\n    }\n}\n
\n

This is because value.deref() creates a reference to value, whose lifetime will end as soon as the function is over, because we don't return it, only a pointer somewhere inside it. The machinery for Arc only works if it has access to the original pointer, not any derived pointers. So if we wanted to return an Arc<String>, we'd need to .clone() out of the Arc<JSON>, which is what we've been trying to avoid this whole time.

\n

However! We don't necessarily need a full Arc<String>! We'd be perfectly happy returning some other type, perhaps implementing Deref<Target = String>, so long as it still gives us those "owned value that acts like a reference" properties. If only we could extend the lifetime of value, perhaps by returning it alongside a reference to its contents, packaged together to implement Deref like we want...

\n

Tying The Two Together With Evil Lesbian Shibari

\n

Our goal is some return type that looks like:

\n
               ,-----------------------+---------------------.\nval: owned --> | contents: *const JSON | refcnt: AtomicUsize |\n               `-----------------------+---------------------'\n                                     |\n          /--------------------------/\n          V\n        ,------------------+--------------+------------+------------.\n        | JSON::String tag | buf: *mut u8 | len: usize | cap: usize |\n        `------------------+--------------+------------+------------'\n                             ^          |\n                             |          |\nptr: ref --------------------/          |\n                                        V\n                                      ,---+---+---+---.\n                                      |'A'|'C'|'A'|'B'|\n                                      `---+---+---+---'\n
\n

That is, we want some val showing us how to get to the main value we care about, and then some pointer ptr into the memory val references. Then, as long as we keep those tied together, we know ptr will still be valid, because val is still alive, because we own val.

\n

A first attempt at writing this reveals an immediate issue3

\n
struct Ref<V, T> {\n    val: V,\n    ptr: &T, // What's the lifetime here?\n}\n
\n

We can't express ptr as a reference, because there's no obvious lifetime to attach it to. Without a way to spell "lifetime of the containing struct" in Rust, it looks like we're going to need a raw pointer instead. But what if.....

\n
struct Ref<V, T: 'static> {\n    val: V,\n    ptr: &'static T,\n}\n\nimpl<T, V: Deref<Target = T>> {\n    fn new(val: V) -> Self {\n        let ptr: &T = val.deref();\n        Self {\n            // This is the easiest way to do lifetime extension\n            // SAFETY: hm?\n            ptr: unsafe { std::mem::transmute(ptr) },\n            val,\n        }\n    }\n}\n
\n

Whoa!! That's scary!!! Are they even allowed to hold hands like that...?

\n

It's true this is exceedingly unsafe if users could extract that ptr: &'static T separately from the val: V it points into (val's lifetime isn't 'static!). But we could also just... not allow that, keeping them tied together always, providing access only via Deref implementation:

\n
impl<V, T> Deref for Ref<V, T> {\n    type Target = T;\n    fn deref(&self) -> &T {\n        self.ptr\n    }\n}\n
\n

Because the effective lifetime for which ptr can be accessed is a subset of the actual lifetime for which val lives, I believe we've properly rules-lawyered Rust's reference aliasing rules into submission. Or have we...

\n

Oh noes... (playground link)

\n
struct SimpleWrapper<T>(T);\n\nimpl<T> Deref for SimpleWrapper<T> {\n    type Target = T;\n    fn deref(&self) -> &Self::Target {\n        &self.0\n    }\n}\n\nfn main() {\n    let r = Ref::new(SimpleWrapper(x));\n    // Check what's stored vs what should be returned\n    let ptr = r.ptr as *const i32 as usize;\n    let actual_ptr = r.val.deref() as *const i32 as usize;\n    println!("ptr: {ptr:x} actual_ptr {actual_ptr:x}");\n}\n
\n

Running this, I got ptr: 7fff85189b5c actual_ptr 7fff85189b88. These are in fact different pointers!!! Turns out I messed up my earlier rules-lawyering: The act of moving val into Ref::new(), taking the .deref() on that stack frame, and then moving it back out to the parent stack frame invalidates ptr3. Lifetimes exist precisely to prevent bugs like this, and our extension trick was foiled. Lesson learned! Guess we'll do this the hard way...

\n

Ensure Address Stability With This One Simple Trick!

\n

To fix our datastructure, we'll want a guarantee that each .deref() will give us the same pointer, even if move the container around. For this, we MUST NOT be able to move the val: V out of its location once we wrap it. Fortunately, Rust has a type exactly for this usecase!

\n

ahem. anyways. Unfortunately, Pin is very hard to use, to the point I found a flaw in my initial implementation4 while writing this :( Still, the docs are really good, and we can pretty easily follow their example to make a self-referential struct:

\n
struct MustPin<V> {\n    val: V,\n    _pin: PhantomPinned,\n}\nstruct PinRef<V, T> {\n    val: Pin<Arc<MustPin<V>>>,\n    // MUST point into [`val`].\n    ptr: *const T,\n}\n\nimpl<V> MustPin<V> {\n    fn new(val: V) -> Pin<Arc<Self>> {\n        Arc::pin(Self {\n            val,\n            _pin: PhantomPinned,\n        })\n    }\n}\n\nimpl<V> PinRef<V, V> {\n    fn new(val: Pin<Arc<MustPin<V>>>) -> Self {\n        let ptr = &raw const val.val;\n        Self { val, ptr }\n    }\n}\n
\n

Comparing this to the example in the docs:

\n
    \n
  1. We use *const T instead of NonNull<T> because the latter is more like a *mut T, and we don't need all that power.
    2. \n
  2. We don't need MaybeUninit because we solve the "knot-tying" trick in a different way: we create the pinned data first, and then store a pointer into it out-of-line. This is still fine because of pin guarantees.
    3. \n
  3. We still need PhantomPinned because if we have Pin<Arc<V>> where V: Unpin, all bets are off, literally every pin guarantee goes out the window.
    4. \n
\n

Deref is simple like before, just with a pointer instead of a reference:

\n
impl<V, T> Deref for PinRef<V, T> {\n    type Target = T;\n    fn deref(&self) -> &Self::Target {\n        // SAFETY: by construction and pin guarantees, the pointer is still valid.\n        unsafe { &*self.ptr }\n    }\n}\n
\n

Now, finally, we're all set up for the big reveal: how are we going to downcast these things?

\n

She Downcast On My Pin 'Til I Arc

\n

Our rule for ptr is that it MUST point somewhere valid inside val. That's all we can assume, and that's what we have to uphold while doing our downcasts. Fortunately, we can use Rust's type-checking for "standard" downcasts to our advantage!

\n
impl<V, T> PinRef<V, T> {\n    fn project<U>(self, f: impl for<'a> FnOnce(&'a T) -> &'a U) -> PinRef<V, U> {\n        let Self { val, ptr } = self;\n        // SAFETY: by validity of `ptr` and `f`\n        let ptr = unsafe { f(&*ptr) as *const U };\n        PinRef { val, ptr }\n    }\n}\n
\n

Stating this signature more in more math-y terms, for those unfamiliar with Rust's syntax:

\n

PinRef : (Type, Type) → Type ∀ V,T,U : Type. project : PinRef(V,T) → (∀(a: Lifetime). ′a T → ′aU) → PinRef(V,U)

\n

How we should interpret this is: If we can go from a &T to a &U for an arbitrary lifetime 'a, that means *U is a fixed offset from *T5. So, because ptr has a fixed address (it was derived from the pinned val), so will the output of f. Other pin guarantees like "val will always remain valid at that address while it's pinned" help too.

\n

If I were a real type theorist, I would have pulled out some sort of commutative diagram and drawn a bunch of arrows, or perhaps even written down some inference rules, but alas, I cannot even abstract over monads... Anyways this argument works for what we originally wanted too:

\n
impl<V, T> PinRef<V, T> {\n    fn filter_project<U>(\n        self,\n        f: impl for<'a> FnOnce(&'a T) -> Option<&'a U>,\n    ) -> Option<PinRef<V, U>> {\n        let Self { val, ptr } = self;\n        // SAFETY: by validity of `ptr`, `f`\n        let ptr = unsafe { f(&*ptr)? as *const U };\n        Some(PinRef { val, ptr })\n    }\n    \n    fn try_project<U, E>(\n        self,\n        f: impl for<'a> FnOnce(&'a T) -> Result<&'a U, E>,\n    ) -> Result<PinRef<V, U>, E> {\n        let Self { val, ptr } = self;\n        // SAFETY: by validity of `ptr`, `f`\n        let ptr = unsafe { f(&*ptr)? as *const U };\n        Ok(PinRef { val, ptr })\n    }\n}\n
\n

You see that??? We did the thing!! To celebrate, here's a full example using the original JSON projections (playground link):

\n
fn print(s: impl Deref<Target = str>) {\n    println!("{}", s.deref())\n}\n\nfn main() {\n    let v = std::sync::Arc::pin(JSON::String(String::from("hello, world!")));\n    let v = PinRef::new(v);\n    let s = v.filter_project(JSON::as_str).unwrap();\n    print(s.clone());\n    print(s);\n}\n
\n

All that remains in our original example is to replace all the plain Arc<JSON> with Pin<Arc<MustPin<JSON>>> (wow what a mouthful), make a Clone implementation, account for ?Sized types, etc. etc. This post is long enough as it is so I've omitted that, but if you want, you can find the full details in my repository. I might release this as a standalone crate if I feel like it, but this might still be riddled with UB I missed so maybe not (:

\n

Anyways!! Hope you learned something, until next time~

\n
\n
    \n
  1. \n

    In increasing order of badness: too much string typing, no error handling, concurrent requests can race and end up doing extra work. Probably others I'm missing too. The solution to that last one is simultaneously very interesting & very boring, read the code yourself if you want. ↩

    \n
    2. \n
  2. \n

    I'm only covering options from the Rust standard library for simplicity, but garbage-collected pointers from dumpster or arena pointers from slotmap can also be good ideas. ↩

    \n
    3. \n
  3. \n

    You might be thinking, "why not struct Ref<V, T> { val: Arc<V>, ptr: &'static T}?" and unfortunately a refutation is much more complex, and this example is more illustrative of why we need Pin later. Suffice to say, even though Arc on its own gives address stability in practice, Rust's type system doesn't enforce that it will 4. ↩ ↩2

    \n
    4. \n
  4. \n

    I previously thought struct PinRef<V, T> { val: Pin<Arc<V>>, ptr: &'static T } was enough, but turns out that's entirely insufficient due to the presence of Unpin. ↩2 ↩

    \n
    5. \n
  5. \n

    "Arbitrary" is key here. Means we can't do fn project<'a, U>(self, f: impl FnOnce(&'a T) -> &'a U) -> PinRef<V, U>, because 'a is bound too early, which would allow us to choose a smaller lifetime, letting us project things w/ interior mutability, which is bad. Wish I could formalize this better but I've thought about it really really hard and haven't been able to come up with a counterexample to my main function so I hope no one else will either. ↩

    \n
    6. \n
\n
\n", "content_markdown": "In the course of writing my [build driver](https://git.sr.ht/~polywolf/driver), I came across a bit of an unusual problem, for which I made a bit of an usual solution. I think the solution is interesting and would like to talk about it, but to understand anything we must first understand the problem at hand.\n\n## Consider The Case Of The Humble Concurrent Cache\n\nSuppose we have some expensive function we'd like to put a cache in front of. Furthermore, suppose we'd like to access this cache from multiple threads. A simple example follows ([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=cf4e4af4e40d6895d16d12779f395fa0)):\n\n```rust\nenum JSON {\n F64(f64),\n String(String),\n Vec(Vec),\n Object(HashMap),\n}\n\nstruct Proxy {\n client: HTTPClient,\n cache: RWLock>,\n}\n\nimpl Proxy {\n fn get(&self, key: &str) -> JSON {\n // 1.\n {\n let cache = self.cache.read().unwrap();\n if let Some(value) = cache.get(key) {\n return value.clone();\n }\n }\n\n // 2.\n let value = self.client.get(key);\n {\n let mut cache = self.cache.write().unwrap();\n cache.insert(key.to_string(), value.clone());\n }\n value\n }\n}\n```\n\nThis code has an \"early exit\" path (1) where it returns a value from the cache if it's present, and a \"late exit\" path (2) where it calls the expensive function, then inserts the resulting value into the cache.\n\nPlease ignore the many, many obvious problems with this implementation[^1]. Instead, let's focus on the one problem that bothers me the most: there's fair bit of `.clone()` action going on here!\n\nTechnically, it's just one `.clone()` per call: one on early exit to take value out of the cache, and one on late exit to put value into the cache. But if those values are big/tree-shaped/otherwise expensive to clone, this cost can dominate, minimizing the savings conferred by a cache. In my code, I found this to be the case, so we gotta do something about it.\n\n## Let's Get Rid Of The Clones?\n\nAssume that, with the way we use this data, read-only access is more than enough. Shared references are read-only & cheap to `Copy`, so using those instead of `.clone()`-ing the entire value seems good. If some later part of the code really needs to take ownership, we can just `.clone()` there, saving time in the average case. So, we'd like to change the signature for `get()` to be:\n\n```rust\nimpl Proxy {\n fn get<'a, 'b>(&'a self, url: &'b str) -> &'a JSON { ... }\n}\n```\n\nBut we can't do this!! Because our cache is behind a mutex, the only way we can get references to its contents is thru temporary handles. Those handles, while live, hold a lock on the cache, plus they only live for the body of the function, a not for all of `'a`. Even if we *could* return one of those handles, that'd be equivalent to holding the lock outside the function, which is very bad. Locks should only be held for VERY SHORT amounts of time unless ur into that sorta thing next month ;)\n\n## Let's Make The Clones Cheaper\n\nSo, no references. What other types can we use? What we want is something with all the following properties:\n\n1. It allows for read access to our data. That is, it allows us to get an `&JSON` somehow.\n2. It has no lifetime parameters (`'a`, the only lifetime we have access to, is too long).\n3. It is cheap to `.clone()`, even if the underlying value is *not* cheap to `.clone()`.\n\nThese requirements hint we should probably still be looking for some sort of pointer... Among standard library types, we have the following options[^2]:\n\n- [Raw pointers](https://doc.rust-lang.org/std/primitive.pointer.html): `*const JSON`\n- [Reference-counted pointers](https://doc.rust-lang.org/std/rc/index.html): `Rc`\n- [Atomically-reference-counted pointers](https://doc.rust-lang.org/std/sync/struct.Arc.html): `Arc`\n\nLike any good Rustacean, we care a lot about safety & concurrency, so `Arc` is the obvious pick here :3 Modifying the example to use it is straightforward ([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=f3e3821009472761d2297690f6262e17)):\n\n```rust\nstruct Proxy {\n client: HTTPClient,\n cache: RWLock>>, // new!\n}\n\nimpl Proxy {\n fn get(&self, url: &str) -> Arc { // new!\n {\n let cache = self.cache.read().unwrap();\n if let Some(value) = cache.get(url) {\n return value.clone();\n }\n }\n\n let value = Arc::new(self.client.get(url)); // new!\n {\n let mut cache = self.cache.write().unwrap();\n cache.insert(url.to_string(), value.clone());\n }\n value\n }\n}\n```\n\nOther than the three lines with `Arc` added to them, this implementation looks the exact same as before. But now our clones are cheaper, so we're happy, yay!!\n\n## So What's This About Downcasting?\n\nI hope the above section convinced you having an `Arc` \"owned value that acts like a reference\" is both normal to want & possible to achieve. Switching gears a bit, I'd like to discuss an interesting shortcoming with them: they don't fit into Rust's type system very well.\n\nSupposed we know for a fact that certain `JSON` values are strings, and we're only interested in the `JSON::String` variant of them. With an owned value or a shared reference, we can just pattern-match to \"downcast\" from a `JSON` to a `String`, or a `&JSON` to a `&String`.\n\n```rust\nimpl JSON {\n fn to_str(self) -> Option {\n match self {\n Self::String(s) => Some(s),\n _ => None,\n }\n }\n \n fn as_str(&self) -> Option<&String> {\n match self {\n Self::String(s) => Some(s),\n _ => None,\n }\n }\n}\n```\n\nBut if we have an `Arc`, we can't get another `Arc` the same way!\n\n```rust\nimpl JSON {\n fn doesnt_exist(value: Arc) -> Option> {\n match value.deref() {\n Self::String(s) => Some(s), // compile error!\n _ => None,\n }\n }\n}\n```\n\nThis is because `value.deref()` creates a reference to `value`, whose lifetime will end as soon as the function is over, because we don't return it, only a pointer somewhere inside it. The machinery for `Arc` only works if it has access to the original pointer, not any derived pointers. So if we wanted to return an `Arc`, we'd need to `.clone()` out of the `Arc`, which is what we've been trying to avoid this whole time.\n\nHowever! We don't necessarily *need* a full `Arc`! We'd be perfectly happy returning some other type, perhaps implementing `Deref`, so long as it still gives us those \"owned value that acts like a reference\" properties. If only we could extend the lifetime of `value`, perhaps by returning it alongside a reference to its contents, packaged together to implement `Deref` like we want...\n\n## Tying The Two Together With Evil Lesbian Shibari\n\nOur goal is some return type that looks like:\n\n```\n ,-----------------------+---------------------.\nval: owned --> | contents: *const JSON | refcnt: AtomicUsize |\n `-----------------------+---------------------'\n |\n /--------------------------/\n V\n ,------------------+--------------+------------+------------.\n | JSON::String tag | buf: *mut u8 | len: usize | cap: usize |\n `------------------+--------------+------------+------------'\n ^ |\n | |\nptr: ref --------------------/ |\n V\n ,---+---+---+---.\n |'A'|'C'|'A'|'B'|\n `---+---+---+---'\n```\n\nThat is, we want some `val` showing us how to get to the main value we care about, and then some pointer `ptr` into the memory `val` references. Then, as long as we keep those tied together, we know `ptr` will still be valid, because `val` is still alive, because we own `val`.\n\nA first attempt at writing this reveals an immediate issue[^3]\n\n```rust\nstruct Ref {\n val: V,\n ptr: &T, // What's the lifetime here?\n}\n```\n\nWe can't express `ptr` as a reference, because there's no obvious lifetime to attach it to. Without a way to spell \"lifetime of the containing struct\" in Rust, it looks like we're going to need a raw pointer instead. But what if.....\n\n```rust\nstruct Ref {\n val: V,\n ptr: &'static T,\n}\n\nimpl> {\n fn new(val: V) -> Self {\n let ptr: &T = val.deref();\n Self {\n // This is the easiest way to do lifetime extension\n // SAFETY: hm?\n ptr: unsafe { std::mem::transmute(ptr) },\n val,\n }\n }\n}\n```\n\nWhoa!! That's scary!!! Are they even allowed to hold hands like that...?\n\nIt's true this is exceedingly unsafe if users could extract that `ptr: &'static T` separately from the `val: V` it points into (`val`'s lifetime isn't `'static`!). But we could also just... not allow that, keeping them tied together always, providing access only via `Deref` implementation:\n\n```rust\nimpl Deref for Ref {\n type Target = T;\n fn deref(&self) -> &T {\n self.ptr\n }\n}\n```\n\nBecause the *effective* lifetime for which `ptr` can be accessed is a subset of the *actual* lifetime for which `val` lives, I believe we've properly rules-lawyered Rust's reference aliasing rules into submission. Or have we...\n\nOh noes... ([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=52455c2042bd9bc26f1bbbb8a239f9a4))\n\n```rust\nstruct SimpleWrapper(T);\n\nimpl Deref for SimpleWrapper {\n type Target = T;\n fn deref(&self) -> &Self::Target {\n &self.0\n }\n}\n\nfn main() {\n let r = Ref::new(SimpleWrapper(x));\n // Check what's stored vs what should be returned\n let ptr = r.ptr as *const i32 as usize;\n let actual_ptr = r.val.deref() as *const i32 as usize;\n println!(\"ptr: {ptr:x} actual_ptr {actual_ptr:x}\");\n}\n```\n\nRunning this, I got `ptr: 7fff85189b5c actual_ptr 7fff85189b88`. These are in fact different pointers!!! Turns out I messed up my earlier rules-lawyering: The act of moving `val` into `Ref::new()`, taking the `.deref()` on that stack frame, and then moving it back out to the parent stack frame invalidates `ptr`[^3]. Lifetimes exist precisely to prevent bugs like this, and our extension trick was foiled. Lesson learned! Guess we'll do this the hard way...\n\n## Ensure Address Stability With This One Simple Trick!\n\nTo fix our datastructure, we'll want a guarantee that each `.deref()` will give us the same pointer, even if move the container around. For this, we MUST NOT be able to move the `val: V` out of its location once we wrap it. Fortunately, Rust has a type exactly for this usecase!\n\nahem. anyways. Unfortunately, `Pin` is very hard to use, to the point I found a flaw in my initial implementation[^4] while writing this :( Still, the docs are *really* good, and we can pretty easily follow their example to make [a self-referential struct](https://doc.rust-lang.org/std/pin/index.html#a-self-referential-struct):\n\n```rust\nstruct MustPin {\n val: V,\n _pin: PhantomPinned,\n}\nstruct PinRef {\n val: Pin>>,\n // MUST point into [`val`].\n ptr: *const T,\n}\n\nimpl MustPin {\n fn new(val: V) -> Pin> {\n Arc::pin(Self {\n val,\n _pin: PhantomPinned,\n })\n }\n}\n\nimpl PinRef {\n fn new(val: Pin>>) -> Self {\n let ptr = &raw const val.val;\n Self { val, ptr }\n }\n}\n```\n\nComparing this to the example in the docs:\n\n1. We use `*const T` instead of `NonNull` because the latter is more like a `*mut T`, and we don't need all that power.\n2. We don't need `MaybeUninit` because we solve the \"knot-tying\" trick in a different way: we create the pinned data first, and then store a pointer into it out-of-line. This is still fine because of pin guarantees.\n3. We still need `PhantomPinned` because if we have `Pin>` where `V: Unpin`, all bets are off, literally every pin guarantee goes out the window.\n\n`Deref` is simple like before, just with a pointer instead of a reference:\n\n```rust\nimpl Deref for PinRef {\n type Target = T;\n fn deref(&self) -> &Self::Target {\n // SAFETY: by construction and pin guarantees, the pointer is still valid.\n unsafe { &*self.ptr }\n }\n}\n```\n\nNow, finally, we're all set up for the big reveal: how are we going to downcast these things?\n\n## She Downcast On My `Pin` 'Til I `Arc`\n\nOur rule for `ptr` is that it MUST point somewhere valid inside `val`. That's all we can assume, and that's what we have to uphold while doing our downcasts. Fortunately, we can use Rust's type-checking for \"standard\" downcasts to our advantage!\n\n```rust\nimpl PinRef {\n fn project(self, f: impl for<'a> FnOnce(&'a T) -> &'a U) -> PinRef {\n let Self { val, ptr } = self;\n // SAFETY: by validity of `ptr` and `f`\n let ptr = unsafe { f(&*ptr) as *const U };\n PinRef { val, ptr }\n }\n}\n```\n\nStating this signature more in more math-y terms, for those unfamiliar with Rust's syntax:\n\n$\\begin{gathered} \\text{PinRef} : (\\text{Type}, \\text{Type}) \\rightarrow \\text{Type}\\\\ \\forall\\ V,T,U : \\text{Type}.\\\\ \\text{project} : \\text{PinRef}(V,T) \\rightarrow (\\forall(a: \\text{Lifetime}).\\ \\&'a T \\rightarrow \\&'aU) \\rightarrow \\text{PinRef}(V,U) \\end{gathered}$\n\nHow we should interpret this is: If we can go from a `&T` to a `&U` for an *arbitrary* lifetime `'a`, that means `*U` is a fixed offset from `*T`[^5]. So, because `ptr` has a fixed address (it was derived from the pinned `val`), so will the output of `f`. Other pin guarantees like \"`val` will always remain valid at that address while it's pinned\" help too.\n\nIf I were a real type theorist, I would have pulled out some sort of commutative diagram and drawn a bunch of arrows, or perhaps even written down some inference rules, but alas, I cannot even abstract over monads... Anyways this argument works for what we originally wanted too:\n\n```rust\nimpl PinRef {\n fn filter_project(\n self,\n f: impl for<'a> FnOnce(&'a T) -> Option<&'a U>,\n ) -> Option> {\n let Self { val, ptr } = self;\n // SAFETY: by validity of `ptr`, `f`\n let ptr = unsafe { f(&*ptr)? as *const U };\n Some(PinRef { val, ptr })\n }\n \n fn try_project(\n self,\n f: impl for<'a> FnOnce(&'a T) -> Result<&'a U, E>,\n ) -> Result, E> {\n let Self { val, ptr } = self;\n // SAFETY: by validity of `ptr`, `f`\n let ptr = unsafe { f(&*ptr)? as *const U };\n Ok(PinRef { val, ptr })\n }\n}\n```\n\nYou see that??? We did the thing!! To celebrate, here's a full example using the original JSON projections ([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=9ba8b6f27dc48afb065a2923023fb1da)):\n\n```rust\nfn print(s: impl Deref) {\n println!(\"{}\", s.deref())\n}\n\nfn main() {\n let v = std::sync::Arc::pin(JSON::String(String::from(\"hello, world!\")));\n let v = PinRef::new(v);\n let s = v.filter_project(JSON::as_str).unwrap();\n print(s.clone());\n print(s);\n}\n```\n\nAll that remains in our original example is to replace all the plain `Arc` with `Pin>>` (wow what a mouthful), make a `Clone` implementation, account for `?Sized` types, etc. etc. This post is long enough as it is so I've omitted that, but if you want, you can find the full details in my [repository](https://git.sr.ht/~polywolf/driver/tree/cb9ee31e272f24311981a41f979d82f31f39f61f/item/packages/pin-downcast/src/pin_ref.rs). I might release this as a standalone crate if I feel like it, but this might still be riddled with UB I missed so maybe not (:\n\nAnyways!! Hope you learned something, until next time~\n\n[^1]: In increasing order of badness: too much string typing, no error handling, concurrent requests can race and end up doing extra work. Probably others I'm missing too. The solution to that last one is simultaneously very interesting & very boring, [read the code yourself if you want](https://git.sr.ht/~polywolf/driver/tree/3e0e9250a6657f9e5ed977ecdd08f3550e06f68b/item/packages/driver-db/src/database.rs#L150-270).\n\n[^2]: I'm only covering options from the Rust standard library for simplicity, but garbage-collected pointers from [`dumpster`](https://crates.io/crates/dumpster) or arena pointers from [`slotmap`](https://crates.io/crates/slotmap) can also be good ideas.\n\n[^3]: You might be thinking, \"why not `struct Ref { val: Arc, ptr: &'static T}`?\" and unfortunately a refutation is much more complex, and this example is more illustrative of why we need `Pin` later. Suffice to say, even though `Arc` on its own gives address stability in practice, Rust's type system doesn't enforce that it will [4](https://wolfgirl.dev/blog/2026-09-29-pining-for-arc-downcasting-in-rust/#fn-other-idea).\n\n[^4]: I previously thought `struct PinRef { val: Pin>, ptr: &'static T }` was enough, but turns out that's entirely insufficient due to the presence of `Unpin`. [↩2](https://wolfgirl.dev/blog/2026-09-29-pining-for-arc-downcasting-in-rust/#fnref-other-idea-2)\n\n[^5]: \"Arbitrary\" is key here. Means we can't do `fn project<'a, U>(self, f: impl FnOnce(&'a T) -> &'a U) -> PinRef`, because `'a` is bound too early, which would allow us to choose a smaller lifetime, letting us project things w/ interior mutability, which is bad. Wish I could formalize this better but I've thought about it really really hard and haven't been able to come up with a counterexample to my main function so I hope no one else will either.\n", "dateCreated": "2026-10-01T06:33:51.242437607+00:00", "date_created": "2026-09-29T15:35:14+00:00", "date_modified": "2026-09-29T15:35:14+00:00", "date_published": "2026-09-29T15:35:14+00:00", "description": "Comments", "external_url": "https://wolfgirl.dev/blog/2026-09-29-pining-for-arc-downcasting-in-rust/", "git": { "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e", "path": "items/lobste-rs/2026/09/2026-09-29-pining-for-arc-downcasting-in-rust.md", "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/lobste-rs/2026/09/2026-09-29-pining-for-arc-downcasting-in-rust.md" }, "id": "urn:aggr:item:7364ac0546a231036aebeb40297fea333ca893e3", "isBasedOn": { "@id": "https://wolfgirl.dev/blog/2026-09-29-pining-for-arc-downcasting-in-rust/" }, "mainEntity": { "@id": "https://wolfgirl.dev/blog/2026-09-29-pining-for-arc-downcasting-in-rust/", "@type": "CreativeWork", "archivedAt": { "@id": "https://aggr.aymericbeaumet.com/items/lobste-rs/2026-09-29-pining-for-arc-downcasting-in-rust/#webpage" }, "author": [ { "@type": "Person", "name": "wolfgirl.dev by polywolf" } ], "dateModified": "2026-09-29T15:35:14+00:00", "datePublished": "2026-09-29T15:35:14+00:00", "headline": "Pining for Arc Downcasting in Rust", "timeRequired": "PT12M", "url": "https://wolfgirl.dev/blog/2026-09-29-pining-for-arc-downcasting-in-rust/", "wordCount": 2499 }, "name": "Archived snapshot: Pining for Arc Downcasting in Rust", "preview": null, "reading_minutes": 12, "replicated_at": null, "source": "lobste-rs", "source_name": "Lobsters: Top Stories of the Past Week", "summary": "Comments", "tags": [ "rust" ], "temporalCoverage": "2026-10-01T06:33:51.242437607+00:00", "title": "Pining for Arc Downcasting in Rust", "url": "https://aggr.aymericbeaumet.com/items/lobste-rs/2026-09-29-pining-for-arc-downcasting-in-rust/", "word_count": 2499 }