Repository navigation
Drop tokio requirement for object_store crate #273
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I'm a library developer. I would like my library to have async code, but not force developers into a specific async runtime (smol, tokio, etc). My library depends on object_store, which uses reqwest, which requires tokio to run:
- Reqwest panics due to
not currently running on the Tokio runtime.regardless of async runtime seanmonstar/reqwest#1108 - Reqwest always returns 'there is no reactor running, must be called from the context of a Tokio 1.x runtime' seanmonstar/reqwest#1961
My library can't use object_store without forcing all my users to adopt tokio as their async runtime.
Describe the solution you'd like
I think object_store should drop reqwest as a requirement for its cloud integrations. Perhaps adopt a different HTTP client that works with other async reactors as well?
Describe alternatives you've considered
We looked at dropping our dependency on object_store, but it's such a good library.
We are currently using futures-rs's block_on, but we get:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
Activity
tustvold commentedon Jul 15, 2024
It would be a fairly major undertaking to decouple the various implementations from
reqwest, as whilst efforts have been made to avoid it leaking overly out into the public APIs, the exposed feature set is pretty broad and there is a significant amount of client code. At the very least I would expect subtle behaviour changes between different client implementations.I have filed #62 to track making the dependency on
tokiooptional, and this would then allow interested parties to build third-party implementations that expose the sameObjectStoretrait for their runtime of choice.I have also filed apache/arrow-rs#6056 to track decoupling the HTTP client, but I don't really think this is actionable at this stage. Unless we can demonstrate tangible wins for our existing users from switching HTTP client, I think we would need to preserve the ability to use the battle-tested
reqwestimplementation.alamb commentedon Jul 15, 2024
Thank you for filing this issue @criccomini
I think starting with #62 is a great first step -- that way the crate could be used without tokio.
We could then have a subsequent discussion on how best to provide
ObjectStoretrait impls for various backing stores without tokio (e.g. apache/arrow-rs#6056 or improving reqwest upstream, etc)alamb commentedon Jul 15, 2024
In case anyone cares, here is a related twitter https://twitter.com/andrewlamb1111/status/1812442468853444855
tustvold commentedon Mar 26, 2025
I believe #1 is the only remaining item to achieve this