Add an LLBC backend #3514

zhassan-aws · 2024-09-14T00:12:19Z

This PR adds a new codegen backend that generates low-level borrow calculus (LLBC), which is the format defined by Charon and Aeneas.

The backend can be invoked using -Zaeneas, and will generate a .llbc file. This file can then be passed to Aeneas to generate Lean for example. Currently, Aeneas needs to be manually run on the generated LLBC.

The PR translates stable MIR to unstructured LLBC (ULLBC) and then uses the Charon library to apply transformations that reconstruct the control flow to regain some of the high-level structure (loops, if statements, etc.).

In this PR, very few MIR constructs are handled, but the PR is already pretty big. More support to come in subsequent PRs.

Call-outs:

Charon is currently not released on crates.io, so the library is added as a dependency through GitHub with a particular commit.
The Kani driver is currenty heavily tailored towards CBMC, so currently it's expecting a goto file, and thus errors out when the Aeneas backend is invoked.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

tautschnig

Approving with the following understanding:

Some cleanup is needed as indicated by the comments.
The codegen for Aeneas does need further work, but this doesn't impact any existing user who continues to use the cprover back-end.

kani-driver/src/call_single_file.rs

kani-compiler/src/kani_compiler.rs

tautschnig · 2024-09-18T20:09:40Z

kani-compiler/Cargo.toml

@@ -10,6 +10,7 @@ publish = false

 [dependencies]
 cbmc = { path = "../cprover_bindings", package = "cprover_bindings", optional = true }
+charon = { git = "https://github.com/AeneasVerif/charon", optional = true, default-features = false, rev = "cdc1dcde447a50cbc20336c79b21b42ac977b7fd" }


Do we need some way to automatically update this commit id? Something akin to the CBMC update job?

That's a good idea. Currently, there are a lot of breaking changes though, so most updates will need to be handled manually.

tautschnig · 2024-09-18T20:10:16Z

.gitignore

+## Charon/Aeneas LLBC files
+*.llbc


How would those even come about?

These are emitted by the Kani compiler for the Aeneas backend. They're the equivalent of goto binaries for the CBMC backend. Kani makes sure to delete the goto binaries, but in the current PR, it doesn't yet delete the llbc ones.

kani-compiler/src/codegen_aeneas_llbc/compiler_interface.rs

tautschnig · 2024-09-18T20:13:17Z

kani-compiler/src/codegen_aeneas_llbc/compiler_interface.rs

+ info!("# LLBC resulting from control-flow reconstruction:\n\n{}\n", ccx);
+
+ // Run the micro-passes that clean up bodies.
+ for pass in charon_lib::transform::LLBC_PASSES.iter() {
+ pass.transform_ctx(&mut ccx)
+ }
+
+ println!("# Final LLBC before serialization:\n\n{}\n", ccx);
+
+ // Display an error report about the external dependencies, if necessary
+ ccx.errors.report_external_deps_errors();
+
+ trace!("Done");


That's info, println, trace - seems a bit mixed up? Should there be any use of println at all? (Also applies elsewhere.)

Removed this one

I made the printing of the final LLBC dependent on a new option (--print-llbc) that is used in the current expected tests. Now, by default, nothing will get printed except the name of the LLBC file.

tautschnig · 2024-09-18T20:15:26Z

kani-compiler/src/codegen_aeneas_llbc/mir_to_ullbc/mod.rs

+ parents.into_iter().rev().collect()
+ };
+
+ // Rk.: below we try to be as tight as possible with regards to sanity


What does "Rk." mean?

(Charon dev here) That's a comment copied from Charon, "Rk" is short for @R1kM I believe

I believe it stands for "Remark" here

Hah that makes a lot more sense x)

Fun conversation :)

I was hesitant to remove the comments because we may need to import any future upstream changes to this function, so I wanted to minimize the diff. @tautschnig Let me know if you think the comments create confusion, and I'll remove them.

tautschnig · 2024-09-18T20:15:59Z

kani-compiler/src/codegen_aeneas_llbc/mir_to_ullbc/mod.rs

+ }
+ DefPathData::Impl => todo!(),
+ DefPathData::OpaqueTy => {
+ // TODO: do nothing for now


When does this need to be changed?

This is copied over from Charon, I think the answer was "whenever we understand why we'd need the name of an opaque type". We could probably make this an error, I haven't tested if this can ever occur.

Same comment here @tautschnig

kani-compiler/src/codegen_aeneas_llbc/mir_to_ullbc/mod.rs

celinval

I haven't reviewed the backend code yet, but I'm curious about how this affects our released software. Are we shipping the new backend?

kani-compiler/src/args.rs

zhassan-aws

I haven't reviewed the backend code yet, but I'm curious about how this affects our released software. Are we shipping the new backend?

That was the plan (it's under an unstable feature). If we prefer to not include it in the release at this point, we can disable the feature by default and have a separate set of regressions for this feature.

kani-compiler/src/args.rs

zhassan-aws requested a review from a team as a code owner September 14, 2024 00:12

github-actions bot added the Z-BenchCI Tag a PR to run benchmark CI label Sep 14, 2024

zhassan-aws mentioned this pull request Sep 14, 2024

Expose charon as a rust library AeneasVerif/charon#178

Closed

zhassan-aws force-pushed the llbc4 branch 4 times, most recently from c56066c to 69840fb Compare September 17, 2024 17:24

Add an LLBC backend

e7dbbff

zhassan-aws force-pushed the llbc4 branch from 69840fb to e7dbbff Compare September 18, 2024 17:27

tautschnig approved these changes Sep 18, 2024

View reviewed changes

celinval reviewed Sep 19, 2024

View reviewed changes

kani-compiler/src/args.rs Outdated Show resolved Hide resolved

kani-compiler/src/args.rs Show resolved Hide resolved

zhassan-aws added 2 commits September 19, 2024 11:22

Add another test

08134b2

Merge branch 'main' into llbc4

9dd16d5

zhassan-aws commented Sep 20, 2024

View reviewed changes

kani-compiler/src/args.rs Show resolved Hide resolved

kani-compiler/src/args.rs Outdated Show resolved Hide resolved

zhassan-aws added 3 commits September 20, 2024 16:21

Address PR comments

21be282

Merge branch 'main' into llbc4

e41c5e7

Clarify TODO

f2fdc91

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add an LLBC backend #3514

Add an LLBC backend #3514

zhassan-aws commented Sep 14, 2024 •

edited

Loading

tautschnig left a comment

tautschnig Sep 18, 2024

zhassan-aws Sep 20, 2024

tautschnig Sep 18, 2024

zhassan-aws Sep 20, 2024

tautschnig Sep 18, 2024

zhassan-aws Sep 20, 2024

zhassan-aws Sep 20, 2024

tautschnig Sep 18, 2024

Nadrieril Sep 19, 2024 •

edited

Loading

R1kM Sep 20, 2024

Nadrieril Sep 20, 2024 •

edited

Loading

zhassan-aws Sep 20, 2024

tautschnig Sep 18, 2024

Nadrieril Sep 19, 2024 •

edited

Loading

zhassan-aws Sep 20, 2024

celinval left a comment

zhassan-aws left a comment

Add an LLBC backend #3514

Are you sure you want to change the base?

Add an LLBC backend #3514

Conversation

zhassan-aws commented Sep 14, 2024 • edited Loading

tautschnig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nadrieril Sep 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nadrieril Sep 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nadrieril Sep 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

celinval left a comment

Choose a reason for hiding this comment

zhassan-aws left a comment

Choose a reason for hiding this comment

zhassan-aws commented Sep 14, 2024 •

edited

Loading

Nadrieril Sep 19, 2024 •

edited

Loading

Nadrieril Sep 20, 2024 •

edited

Loading

Nadrieril Sep 19, 2024 •

edited

Loading