Add callback support to FileDescription::read

[shamb0]

Key Changes

• Introduces IoTransferOperation to manage atomic file transfers
• Adds callback support to FileDescription::read

Context

This work completes stages 2 of the async file operations roadmap discussed here:

1. ✅ Generalize callback system (PR Concurrency: Generalize UnblockCallback to MachineCallback #4106)
2. ✅ Add callback support to FileDescription::read (this PR)
3. 🔲 Implement readv() using callbacks (this PR)

[shamb0]

Hi @RalfJung,

I wanted to share some updates and seek your guidance on a few points:

1. Callback Integration
   I’ve implemented the callback using the Mutex synchronization object but would appreciate your insights on whether this is the most suitable concurrency synchronization approach for this case.

2. Clarification on read Behavior
   Regarding your statement:
   "The problem with read is that when it returns, the read may not actually have completed yet."
   I believe this behavior might be specific to the Miri interpreter context. Could you elaborate on this to help me better understand its implications?

3. Exploration of std::fs Library

   While reviewing the std::fs library on the Unix platform, I found the following relevant implementations:

   • FileDesc: Defines core operations like read (e.g., view code).

     #[derive(Debug)]
     pub struct FileDesc(OwnedFd);
     
     impl FileDesc {
         pub fn read(&self, buf: &mut [u8]) -> io::Result<usize> {
             let ret = cvt(unsafe {
                 libc::read(
                     self.as_raw_fd(),
                     buf.as_mut_ptr() as *mut libc::c_void,
                     cmp::min(buf.len(), READ_LIMIT),
                 )
             })?;
             Ok(ret as usize)
         }
     }

   • File (inner struct): Wraps FileDesc (view code).

     pub struct File(FileDesc);

   • File (outer struct): Provides higher-level abstractions (view code).

     use crate::sys::fs as fs_imp;
     
     #[stable(feature = "rust1", since = "1.0.0")]
     #[cfg_attr(not(test), rustc_diagnostic_item = "File")]
     pub struct File {
         inner: fs_imp::File,
     }

4. Understanding Read Operation
   Based on my analysis, it appears that the read operation blocks and fills the destination buffer before returning:

   let read_result = (file_handle.file).read(op.borrow_mut().buffer_mut());

   Could you please confirm if this understanding is correct? If not, I’d be grateful if you could point me toward relevant documentation or files for clarification.

Thanks in advance for your help and guidance! Looking forward to your insights.

[shamb0]

@rustbot ready

[RalfJung]

So this is blocked on #4106, I assume.

✅ Add callback support to FileDescription::read (this PR)
✅ Implement readv() using callbacks (this PR)

I would prefer if you could split this into two separate PRs. Refactoring an existing interface should not be done in the same changeset as adding new functionality, if it can be avoided. Splitting changes in smaller PRs greatly helps with making them easier to review.

[shamb0]

Hi @RalfJung,

Thank you for your feedback and for highlighting the review challenges—I completely understand.

This PR focuses on two key changes:

1. ✅ Generalizing the callback system (PR #4106)
2. ✅ Adding callback support to FileDescription::read (this PR).

Let me know if there's anything else you'd like me to adjust.

Thanks! 🙂

[shamb0]

@rustbot ready

[RalfJung]

I don't understand the point of all the new things you added here.

What I would expect is simply a new argument to read that has a type like DynMachineCallback<Result<u64, IoError>>. This can replace the existing dest: &MPlaceTy<'tcx>.

[shamb0]

I don't understand the point of all the new things you added here.

What I would expect is simply a new argument to read that has a type like DynMachineCallback<Result<u64, IoError>>. This can replace the existing dest: &MPlaceTy<'tcx>.

I need your guidance on the following:

• I have not modified the signature of the FileDescription::read() function. Instead, I introduced a new function, FileDescription::read_with_callback(), and integrated DynMachineCallback<Result<u64, IoError>> as a replacement for dest: &MPlaceTy<'tcx>. Below is the function definition for reference:

  fn read_with_callback<'tcx>(
      self: FileDescriptionRef<Self>,
      _communicate_allowed: bool,
      _ptr: Pointer,
      _len: usize,
      _completion_callback: DynFileIOCallback<'tcx>,
      _ecx: &mut MiriInterpCx<'tcx>,
  ) -> InterpResult<'tcx> { }

• Modifying the signature of FileDescription::read() will affect the following modules. I need your input on how to adapt the signature changes for these modules:

  FileDescription for io::Stdin  
  FileDescription for io::Stdout  
  FileDescription for io::Stderr  
  FileDescription for NullOutput  
  FileDescription for AnonSocket  
  FileDescription for Epoll  
  FileDescription for EventFd  

Let me know your thoughts and suggestions on how to proceed.

[RalfJung]

Now that #4106 landed, please rebase this over the latest miri HEAD.

[RalfJung]

You have changes in the first commit which are undone by the second commit -- please clea that up by squashing the two commits.