How do I use native (p/invoke) overlapped IO from C# utilizing async/await?

Question

I'd like to make use of native overlapped IO methods (via P/Invoke) in C# in an async/await friendly manner.

The following give good instructions on how to use overlapped IO in general:

• Using Overlapped IO from Managed Code
• Using Overlapped IO in .NET

Question: How can I make use of Overlapped IO using await to determine when the operation is complete?

For example, how can I call the method CfHydratePlaceholder utilizing overlapped IO and using async/await to determine when it is finished.

Answer 1

I used the information from the mentioned sites to create an async/await friendly class for doing Overlapped IO:

/// <summary>
/// Class to help use async/await with Overlapped class for usage with Overlapped IO
/// </summary>
/// <remarks>
/// Adapted from http://www.beefycode.com/post/Using-Overlapped-IO-from-Managed-Code.aspx
/// Other related reference: 
/// - https://www.codeproject.com/Articles/523355/Asynchronous-I-O-with-Thread-BindHandle
/// - https://stackoverflow.com/questions/2099947/simple-description-of-worker-and-i-o-threads-in-net
/// </remarks>
public unsafe sealed class OverlappedAsync : IDisposable
{
    [DllImport("kernel32.dll", SetLastError = true, ExactSpelling = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    static extern unsafe bool CancelIoEx([In] SafeFileHandle hFile, [In] NativeOverlapped* lpOverlapped);

    // HRESULT code 997: Overlapped I/O operation is in progress.
    // HRESULT code 995: The I/O operation has been aborted because of either a thread exit or an application request.
    // HRESULT code 1168: Element not found.
    // HRESULT code 6: The handle is invalid.
    // https://learn.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499-
    // https://learn.microsoft.com/en-us/windows/win32/debug/system-error-codes--500-999-
    // https://learn.microsoft.com/en-us/windows/win32/debug/system-error-codes--1000-1299-
    const int ErrorIOPending = 997;
    const int ErrorOperationAborted = 995;
    const int ErrorNotFound = 1168;
    const int ErrorInvalidHandle = 6;

    readonly NativeOverlapped* _nativeOverlapped;
    readonly TaskCompletionSource<bool> _tcs = new();
    readonly SafeFileHandle _safeFileHandle;
    readonly CancellationToken _cancellationToken;

    bool _disposed = false;

    /// <summary>
    /// Task representing when the overlapped IO has completed
    /// </summary>
    /// <exception cref="OperationCanceledException">The operation was cancelled</exception>
    /// <exception cref="ExternalException">An error occurred during the overlapped operation</exception>
    public Task Task => _tcs.Task;

    /// <summary>
    /// Construct an OverlappedAsync and execute the given overlappedFunc and safeHandle to be used in the overlappedFunc.
    /// </summary>
    /// <exception cref="OperationCanceledException">If the CancellationToken is cancelled</exception>
    public OverlappedAsync(SafeFileHandle safeFileHandle, Func<IntPtr, int> overlappedFunc, CancellationToken ct)
    {
        if (overlappedFunc == null) throw new ArgumentNullException(nameof(overlappedFunc));
        _safeFileHandle = safeFileHandle ?? throw new ArgumentNullException(nameof(safeFileHandle));

        _safeFileHandle = safeFileHandle;
        _cancellationToken = ct;

        // bind the handle to an I/O Completion Port owned by the Thread Pool
        bool success = ThreadPool.BindHandle(_safeFileHandle);
        if (!success)
        {
            throw new InvalidOperationException($"{nameof(ThreadPool.BindHandle)} call was unsuccessful.");
        }

        // Check if cancellation token is already triggered before beginning overlapped IO operation.
        // Check if cancellation token is already triggered before beginning overlapped IO operation.
        if (_cancellationToken.IsCancellationRequested)
        {
            _tcs.SetCanceled();
            return;
        }

        var overlapped = new Overlapped();
        _nativeOverlapped = overlapped.Pack(IOCompletionCallback, null);
        try
        {
            var nativeOverlappedIntPtr = new IntPtr(_nativeOverlapped);
            var result = overlappedFunc(nativeOverlappedIntPtr);
            ProcessOverlappedOperationResult(result);
        }
        catch
        {
            // If the constructor throws an exception after calling overlapped.Pack, we need to do the Dispose work
            // (since the caller won't have an object to call dispose on)
            Dispose();
            throw;
        }
    }

    ///<inheritdoc cref="OverlappedAsync.OverlappedAsync(SafeFileHandle, Func{IntPtr, HRESULT}, CancellationToken)"/>
    public OverlappedAsync(SafeFileHandle safeFileHandle, Func<IntPtr, int> overlappedFunc)
        : this(safeFileHandle, overlappedFunc, CancellationToken.None)
    {
    }

    ///<inheritdoc/>
    public void Dispose()
    {
        if (!_disposed)
        {
            return; // Already disposed
        }
        _disposed = true;

        if (_nativeOverlapped != null)
        {
            Overlapped.Unpack(_nativeOverlapped);
            Overlapped.Free(_nativeOverlapped);
        }
    }

    /// <summary>
    ///  Called when the cancellation is requested by the _cancellationToken.  
    ///  Cancels the IO request
    /// </summary>
    void OnCancel()
    {
        // If this is disposed, don't attempt cancellation.
        // If the task is already completed, then ignore the cancellation.
        if (_disposed || Task.IsCompleted)
        {
            return;
        }

        bool success = CancelIoEx(_safeFileHandle, _nativeOverlapped);
        if (!success)
        {
            var errorCode = Marshal.GetLastWin32Error();

            // If the error code is "Error not Found", then it may be that by the time we tried to cancel,
            // the IO was already completed and the handle and/or the nativeOverlapped is no longer valid.  This can be ignored.
            if (errorCode == ErrorNotFound)
            {
                return;
            }

            SetTaskExceptionCode(errorCode);
        }
    }

    /// <summary>
    /// Handles the HRESULT returned from the overlapped operation,
    /// If the IO is pending, register the OnCancel method with the _cancellationToken
    /// Otherwise, there is nothing to do (since the IO completed synchronously and IOCompletionCallback was already called)
    /// </summary>
    /// <param name="resultFromOverlappedOperation"></param>
    void ProcessOverlappedOperationResult(int resultFromOverlappedOperation)
    {
        // If the IO is pending (this is the normal case)
        if (resultFromOverlappedOperation == ErrorIOPending)
        {
            // Only register the OnCancel with the _cancellationToken in the case where IO is pending.
            _cancellationToken.Register(OnCancel);
            return;
        }

        // Invalid handle error will not result in a callback, so it needs to be handled here with an exception.
        if (resultFromOverlappedOperation == ErrorInvalidHandle)
        {
            Marshal.ThrowExceptionForHR(resultFromOverlappedOperation);
        }
    }

    /// <summary>
    /// Set the TaskCompletionSource into the proper state based on the errorCode
    /// </summary>
    void SetTaskCompletionBasedOnErrorCode(uint errorCode)
    {
        if (errorCode == 0)
        {
            _tcs.SetResult(true);
        }

        // If the error indicates that the operation was aborted and the cancellation token indicates that cancellation was requested,
        // Then set the TaskCompletionSource into the cancelled state.  This is expected to happen when cancellation is requested.
        else if (errorCode == ErrorOperationAborted && _cancellationToken.IsCancellationRequested)
        {
            _tcs.SetCanceled();
        }

        // Otherwise set the TaskCompletionSource into the faulted state
        else
        {
            SetTaskExceptionCode((int)errorCode);
        }
    }

    /// <summary>
    /// This callback gets called in the case where the IO was overlapped.
    /// This sets the TaskCompletionSource to completed 
    /// unless there was an error (in which case the TaskCompletionSource's exception is set)
    /// </summary>
    void IOCompletionCallback(uint errorCode, uint numBytes, NativeOverlapped* nativeOverlapped)
    {
        // It's expected that the passed in nativeOverlapped pointer always matches what we received
        // at construction (otherwise Dispose will be unpacking/freeing the wrong pointer).
        Debug.Assert(nativeOverlapped == _nativeOverlapped);

        // We don't expect the callback to be called if the TaskCompletionSource is already completed
        // (i.e. in the case where IO completed synchronously or had an error)
        Debug.Assert(!Task.IsCompleted);

        SetTaskCompletionBasedOnErrorCode(errorCode);
    }

    /// <summary>
    /// Set the TaskCompletion's Exception to an ExternalException with the given error code
    /// </summary>
    void SetTaskExceptionCode(int code)
    {
        Debug.Assert(code >= 0);
        try
        {
            // Need to throw/catch the exception so it has a valid callstack
            Marshal.ThrowExceptionForHR(code);

            // It's expected that for valid codes the above always throws, but when it encounters a code it isn't aware of
            // it does not throw.  Throw here for those cases.
            throw new Win32Exception(code);
        }
        catch (Exception ex)
        {
            // There is a race condition where both the Cancel workflow and the IOCompletionCallback flow
            // could set the Exception.  Only one of the errors will get translated into the Task's exception.
            bool success = _tcs.TrySetException(ex);
            Debug.Assert(success);
        }
    }
}

Sample usage:

using var overlapped = new OverlappedAsync(hFile, nativeOverlapped => CfHydratePlaceholder(hFile, 0, -1, 0, nativeOverlapped));
await overlapped.Task;

Note: It is important the the file handle remains valid until the OverlappedAsync.Task has completed.

Using this approach is convenient when using native methods that do not have counterparts in .NET. Here are some examples from the Cloud Filter API that can use this approach:

• CfHydratePlaceholder
• CfConvertToPlaceholder
• CfRevertPlaceholder
• CfSetPinState
• CfUpdatePlaceholder