Replies: 9 comments 15 replies
-
|
I will say as well, looking at the output of Obviously we can expect the runtime performance of |
Beta Was this translation helpful? Give feedback.
-
|
As of 27/01/22 the code is now 730432 loc |
Beta Was this translation helpful? Give feedback.
-
|
Had some more digging today, this time using Sorting by time taken: > sort -nk 2 times.txt | tail -11 # top 10 plus total
time: 9.472; rss: 51MB -> 598MB ( +547MB) expand_crate
time: 9.472; rss: 51MB -> 598MB ( +547MB) macro_expand_crate
time: 10.330; rss: 1637MB -> 1969MB ( +332MB) generate_crate_metadata
time: 10.460; rss: 51MB -> 822MB ( +771MB) configure_and_expand
time: 11.072; rss: 1151MB -> 1551MB ( +401MB) MIR_borrow_checking
time: 12.394; rss: 1186MB -> 1140MB ( -45MB) item_bodies_checking
time: 16.134; rss: 1096MB -> 1142MB ( +46MB) type_check_crate
time: 23.106; rss: 2091MB -> 2232MB ( +141MB) LLVM_passes(crate)
time: 24.867; rss: 1963MB -> 2229MB ( +266MB) codegen_to_LLVM_IR
time: 24.976; rss: 1969MB -> 2229MB ( +259MB) codegen_crate
time: 80.185; rss: 40MB -> 165MB ( +125MB) totalInterestingly, we peak at over 2GB of RAM when compiling as well. |
Beta Was this translation helpful? Give feedback.
-
|
Just to add some more to the above: #[derive(serde::Deserialize, serde::Serialize)]
pub struct Discount {
// Always true for a deleted object
#[serde(default)]
pub deleted: bool,
/// The subscription that this coupon is applied to, if it is applied to a particular subscription.
#[serde(skip_serializing_if = "Option::is_none")]
pub subscription: Option<String>,
}
fn main() {}
Switching over to Based on the above findings and this, we can see that serde is producing almost all of the bloat in the crate. One solution we've discussed is simply splitting the crate into multiple crates based on the existing feature gates, eg ( |
Beta Was this translation helpful? Give feedback.
-
|
In light of this recent PR on serde, I thought it would be interesting to compare with the old results. The below was done with Therefore we are seeing near enough 1 million lines of serde generated code. The below was done with a patch of (serde)[https://github.com/nnethercote/serde/commit/f8cc37c8547589edb88123424b55cce19a5feaaf] from the PR: Delta of |
Beta Was this translation helpful? Give feedback.
-
|
nice! thanks. We should get this 'for free' since we only have a minimum bound on serde |
Beta Was this translation helpful? Give feedback.
-
|
Please don't take this as criticism, just as a data point. We are pulling |
Beta Was this translation helpful? Give feedback.
-
|
That said, I have had a project in the back of my mind to support manually spitting out impls for Deserialize for those parts of the api that aren't derivable for a while now but haven't had a strong enough urge to do this since for my personal use-case what we have now is 'good enough'. It is however on the roadmap :) Thanks for chiming in. |
Beta Was this translation helpful? Give feedback.
-
|
Our use case is very limited, just a handful of calls. I think a fair bit of "blame" (if that's the right word) can be places on the Rust toolchain here too. I love Rust, but I frequently lament the slow compile times. Switching to Go/C# briefly is enough to remind me just how bad they are lol. That and the decisions made around floating point comparisons. Both I can live with though, so 🤷♂️ |
Beta Was this translation helpful? Give feedback.
-
|
I'm in the same boat, the compilation requirements of this library breaks the box running our github actions CD, so I ended up just tearing it out and replacing it with reqwest calls to the few endpoints we actually use |
Beta Was this translation helpful? Give feedback.
-
|
We are working on a PR that replaces serde with miniserde here #314, so I would hope this would be mostly 'fixed' once that lands. |
Beta Was this translation helpful? Give feedback.
-
|
Broke several ec2 instances with GitLab runners in one day. The machine runs out of RAM and the server explodes. I tried to trim down the features and disable default features - no luck. Even some teammate computers can't handle compiling the library without freezing. |
Beta Was this translation helpful? Give feedback.
-
|
As a user of this crate, should I expect turning off unused features to significantly reduce compile times? |
Beta Was this translation helpful? Give feedback.
-
|
Hey folks, some lovely numbers from the rewrite (currently hiding in the
It is not ready for production but we are getting close to an RC |
Beta Was this translation helpful? Give feedback.
-
|
@arlyon This is super exciting news, thanks for the work on this! Any idea when a release candidate is expected? |
Beta Was this translation helpful? Give feedback.
-
|
Wow, that's awesome Alexander. Thank you so much for the work on this! |
Beta Was this translation helpful? Give feedback.
-
|
Not an expert on rust at all, but searching around came across this discussion. Compiling this on a t2.small doesn't work, it doesn't work also in t2.media (about 4 GB ) or ram. It just eats the RAM pretty quick until the computer becomes unresponsive, you can see with htop in ubuntu that it builds up in a few seconds and reaches the 4 gigs You have to stop the EC2 instance manually from AWS, and run again. It sounds that it might be just better for now not to use this crate ? and use an HTTP client and talk to Stripe API directly for the use-cases of each app. Not sure if this is helpful: |
Beta Was this translation helpful? Give feedback.
-
|
Thank you so much @iganev I've decided for now just to use direct calls to Stripe API. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, when you absolutely must compile a big codebase on a root vegetable, you'd want to have swap. I found it hard to compile a project using the stripe sdk with less than 16GB memory. 8GB RAM + 8GB swap saves the day, although the pipeline might run forever... |
Beta Was this translation helpful? Give feedback.
-
|
I got a similar error when upgrading to 0.38.0, we rolled back to 0.37.1 and were able to compile (0.38.0 was also causing my rust-analyzer+ide to get sigkilled, which ive never seen before) |
Beta Was this translation helpful? Give feedback.
-
|
Please consider trying the next branch if you are struggling with resource usage. While unstable it is the next major version and is getting very close to beta / rc level. In terms of improvements to compile performance, see this comment: #77 (comment) Beyond that, I would really love some testers and am trying to be responsive to PRs / issues. Alternatively, feel free to reach out directly at arlyon@me.com if you have any feedback on the |
Beta Was this translation helpful? Give feedback.
-
|
i didn't realize next was available for testing, i'd be happy to run it in our test enviornments, do you have a rough ETA on when you would consider it production stable? |
Beta Was this translation helpful? Give feedback.
-
It's no secret that this library has quite terrible compile perf:
Here is some data, based on the benchmark project (which still uses the blocking runtime):
Clearly, from this, serde is vastly increasing the size of the code, and is likely the primary culprit. Solutions?
Beta Was this translation helpful? Give feedback.
All reactions