Replies: 5 comments 13 replies
-
|
For the constraint and type level stuff, I think it would be interesting to have a metric about overload resolution, like how many overload resolutions were performed. |
Beta Was this translation helpful? Give feedback.
-
|
I am interested in this as well. More generally though, I'm curious what is really meant by this? Is this a reference to "value objects". |
Beta Was this translation helpful? Give feedback.
-
This depends on reference assembly support. Can you add this to the list at the top @cartermp ? Large gains (similar to the 2x gain from F# 4.5 -> 5 mentioned assuming that an average project is depended on by one other project on average, with much larger gains than this for large solutions), and low cost, so it should be on the radar. |
Beta Was this translation helpful? Give feedback.
-
|
Projects that depend on changed projects get built, even if the public surface area hasn't changed. Right? |
Beta Was this translation helpful? Give feedback.
-
|
Edit: below is incorrect, please ignore Not necessarily, no. It depends on what the up-to-date checker determines (note that VS has other heuristics as well, so this statement only applies to command-line builds). But with a simply Library -> Console solution you can verify that the Console project is not rebuilt unless you apply changes to it. Timings with no changes from an msbuild perf summary: Timings when only the library project changes: Exact values in ms will fluctuate, but it's roughly the same ratio each time. |
Beta Was this translation helpful? Give feedback.
-
|
Actually, I was incorrect here, as it will rebuild the projects that depend on the ones that change. I didn't link them up before :) That said, I don't know if emitting reference assemblies does anything here. Maybe if there is logic to check if the API surface area has changed? But this would also have to take into account internals via |
Beta Was this translation helpful? Give feedback.
-
|
Whoops on phone, didn't properly reply to this thread only. But yeah much of this was done a few years ago and afaik (though I've tried to find the most recent source, I think it was by @jaredpar ) that they'd like to bring it up to a level it can be turned on by default. Nothing scheduled, just aspirations as I read it, though it's good to take note of. |
Beta Was this translation helpful? Give feedback.
-
|
Okay, finally hunted it down and yes this would improve the incremental build scenario. Will update the list, thanks! |
Beta Was this translation helpful? Give feedback.
-
|
More info (and links to additional msbuild details) can be found here https://github.com/dotnet/roslyn/blob/master/docs/features/refout.md |
Beta Was this translation helpful? Give feedback.
-
|
@cartermp did you see the links at the bottom of this markdown? |
Beta Was this translation helpful? Give feedback.
-
|
Yep, it's accounted for up top |
Beta Was this translation helpful? Give feedback.
-
Our experiences in C# / VB have consistently shown that the compiler server is the greatest factor in our build performance. There are several other benefits that you didn't capture here:
For C# builds the ratio of MSBuild to C# overhead is roughly 3/1.
As a data point, C# used to have incremental compilation support and we eventually deprecated it. The wins don't justify the complication for batch build scenarios. Possible that F# can be more efficient here (can see how signature files could be heavily leveraged in incremental compilation). Fairly skeptical that this would pay off though and I'd definitely prioritize it below a server host or reference assemblies. |
Beta Was this translation helpful? Give feedback.
-
|
Incremental compilation within a project might be a win for F#, largely since:
But without a prototype it's all just speculation. I agree though that reference assembly emit and compiler server support would be a bigger bang for our buck though. |
Beta Was this translation helpful? Give feedback.
-
|
That for this info! I've read this a few times and read through my binlog, but I'm struggling to see the information you're discussing.
Where should we look in a binlog for this?
Can you say more about this? I can't see from this line how I would find out which targets are called, and whether they are expected. How would you call the expensive target only once?
Where do you look in the binlog for this information? |
Beta Was this translation helpful? Give feedback.
-
|
Binlogs are definitely challenging to grok.
This would be in the Fsc task for a given project. You can expand it to see the full command-line given to that project. If that command-line includes large
The binlog viewer will show a call hierarchy of tasks and targets. At least based on Dark's codebase, I don't think there's anything wonky here. This is more applicable for when someone were to write their own custom tasks and targets, and add those into their build.
In this case it would be like the first case, looking at the command-line passed to |
Beta Was this translation helpful? Give feedback.
-
I think people who state this didn't spend enough of their career trying to search through diagnostics logs. I will never insult a binlog for fear they will be taken away from me and I'd need to go back to diagnostic logs 😄 |
Beta Was this translation helpful? Give feedback.
-
|
@jaredpar "challenging to grok" is not an insult, otherwise plenty of people would be called out (and cancelled) for insulting F# 🙂 |
Beta Was this translation helpful? Give feedback.
-
Looking to improve your F# compile times? Read on!
Some context
The F# compiler has steadily been getting faster over time, and that's going to continue. But there's plenty of work to be done.
At the time of writing, the F# compiler could benefit from several items of work:
Aside from (1), these are all incredibly challenging and expensive things to do. That doesn't mean they won't get done, but it does mean that it may take time because the team has to balance other priorities and can't be caught in a world where nothing of value is delivered for users of a long period of time.
In the interim, there are some things you can do as a user to help everyone (including the F# team) understand your compile times.
Understand your overall build times
Most F# projects that run on .NET use the .NET build system, MSBuild. MSBuild has non-negligible overhead, and as a first step you should get a handle on how much time is spent "in MSBuild" as opposed to actually compiling code.
I recommend running these four commands at your solution level:
The first two commands clean your solution and produce an MSBuild performance summary with MSBuild building your solution how it normally would. The second two clean and force MSBuild to run with only one process.
Here is an example of a perf summary for a brand new
dotnet new mvc -lang F#project:As you can see, MSBuild overhead was actually higher than the time spend in the F# compiler!
For a larger project, here's a summary of the XPlot solution that I maintain:
We have two things to look at, Target and Task performance. Looking at Target data, we can see that MSBuild overhead is quite high (
ResolveProjectReferenceshas me raising my eyebrows), but theCoreCompiletarget is what's actually the most expenssive. That can be confirmed by looking at the specificFsctask, where you'll see the timings as nearly identical.Note that in these reports I did not restrict MSBuild to one process for building.
Here's what it looks like with
/m:1:As you can see, MSBuild overhead is reduced a lot, but my overall build time more than doubled since nothing was built in parallel.
Takeaways from this data
So, what can we learn from this? Several things:
This is good, since it's what you'd actually want to observe: more time spent compiling than not.
If you do not see data like this, and MSBuild targets/tasks are indeed your main source of timings, then something might be wrong with your build. Most of your build time should be in
CoreCompileandFsc. If it's not, then it's time to dig deeper into why MSBuild is spending so much time doing stuff in your solution.Binlogs
Binlogs are the best tool for figuring out exactly what your build is doing. If you have a lot of MSBuild overhead in your build times, this will tell you every single thing that MSBuild is doing when you build your codebase.
The next tool you can use is an MSBuild binary log. These are not mean to be used for performance analysis, because producing a binlog will incur overhead that can mess with your build. However, if you find that the timings in a binlog are proportional to your timings in a performance summary, it may be helpful to look at the reported timings.
To produce a binlog, and to avoid confusion over multiple processes, do this:
This will produce produce a binary log file that you can view online or install a tool to view on your own machine
Some specific things it will show that are relevant to building F# projects:
.dlls across your entire codebase? Are you referencing way too many.dlls per project? Is that needed? Recall that the F# compiler must crack metadata references during compilation since that's one of the inputs to typechecking. The more.dlls and the bigger they are, the more time is spent doing that.I'm not personally an MSBuild expert so I can't tell you everything about what the binlog would say about F# projects. But I've found value in the above two points in analyzing builds in the past, and have been able to adjust a build to shave off several seconds just be updating dependencies and project references.
ETL traces
The big one. If you're on Windows, you can use PerfView for this. If not, you'll need to install the
dotnet-traceglobal tool.An ETL trace of a build will give an extremely detailed view of your build from start to finish. You need specific tools to analyze them (PerfView, DotTrace, converting to chrome or SpeedScope). But their information is rich. It will show:
TcExprThenfunction in the typechecker, or how much is spent there and in all other functions that it calls.This is the meat and potatoes of detailed performance work in the F# compiler. It is not easy to do, and at this point it's often times best to just submit this data to the F# team and ask them if they notice anything that seems off.
Often times, even though build times are slow, the trace data indicates that everything looks normal. That would mean, unfortunately, that you've run into a case of "the F# compiler should be faster".
But sometimes, way more CPU or memory is being used by something than we'd expect. That's usually a signal that there's some low-hanging fruit for us to go pick. Sometimes the "low-hanging fruit" is actually very difficult to resolve, but it's isolated from any architectural changes and would thus be a low-risk kind of change to make. We'll either fix these outright or explain what the issue is and what a motivated contributor could consider doing to resolve the issue.
There is an existing catalogue of issues that usually have ETL trace data with them tagged here: https://github.com/dotnet/fsharp/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3ATenet-Performance
Hot tips
What some hot tips that will almost certainly make your life better? Here they are:
Make sure you're using the latest F# compiler
This one should be obvious, but I've ran into enough people who use older compilers that it has to come first.
The only way to get performance improvements is to update your compiler and toolset.
If you're still on an older .NET Core LTS or something, you will likely see a significant jump in compile times just by updating your compiler. Please do it!
Split up big projects into smaller ones
Do you have "I have one giant project with everything in it" syndrome like we do in dotnet/fsharp? Well I've got a quick fix for you!
You can proabbly improve your build times significantly if you split up a huge project in your solution. This is for several reasons:
Splitting things into different projects is, in a way, a form of incremental building. There are several circumstances that can lead to a rebuild of things unexpectedly, but you'll find that builds go much quicker if you can split things up more.
Just don't go overboard with this. Putting every little thing into its own project will just make MSBuild overhead too high, so try to group things logically.
Use F# signature files (tooling performance)
Do you have an unavoidably large project? If so, consider using F# signature files (.fsi) for each implementation file (.fs).
The F# compiler will now actively not typecheck several things in a tooling scenario if you have explicit signature files:
What this means when editing code is that all tooling should be significantly snappier for a large project with a lot of code.
Don't include type providers in your build transitive build graph
Type Providers are a great tool for a variety of applications, but have a non-negligible, unavoidable performance cost when you merely reference one: #7907
The reason is that the F# compiler needs to instantiate a type provider at build time and inspect provided types. This can add upwards of 500ms to each project that depends on one.
Look at your binlog. If you have projects that are being passed type provider references and you aren't actually using them in that project, fix your build!
This hot tip can easily shave several seconds off of your build. It's easy for dependencies to sneak into your build unexpectedly.
Don't get too fancy with type constraints and typelevel programming
Constraint programming in F# can be fun and expressive and useful, but it's possible to go overboard. There is a tendency for some people to get a little "type happy" and start to write typelevel-style programs using the F# constraint system.
In general, try not to write code that over-abstracts with constraints just for the sake of not repeating yourself a few times. Repeating yourself 2 or 3 times really isn't a big deal.
If you really don't want to repeat yourself, or you must make heavy use of typelevel abstractions and constraint-based programming, I suggest the following:
Use FSharpPlus instead instead of going all-in on your own. FSharpPlus is extremely well-written and likely has almost every abstraction you care about anyways. Just use that.
If you really want to do heavy constraint-based programming yourself, you're on your own. The authors of FSharpPlus have ran into and found ways to work around most problems (w.r.t performance) that the F# compiler has, so that's why I recommend using that. If you're set on doing things yourself, just know that you're giving up all of that knowledge as well. Constraint solving in the F# compiler can sometimes lead to exponential compile-time paths and you really don't want to be dealing with that. Just use FSharpPlus if you're going to make heavy use of this kind of programming.
It's also worth noting that heavy use of constraint-based and/or typelevel programming can make your code very difficult for others to understand. Most people really do struggle to understand these kinds of abstractions and you can often be better off by avoiding their use across a wider team or group of contributors.
Try to avoid generating enormous types and match expressions
Lastly, depending on what you do with nested types and matchin on their structures, you can quickly get exploding compile-times too. There's just way too many things the compiler has to solve when you have large, nested types that you're pattern matching on to a different degree (think a union with 100 cases, each made up of optional records with optional data, each of which you expand into sepcific patterns).
We'd love for this not to be a performance issue, but there's really no way around it. Extremely complex nested data with complex patterns torture the compiler to death.
Beta Was this translation helpful? Give feedback.
All reactions