The body of an async-iterator method must contain a 'yield' statement

[thefringeninja]

I have this method that ultimately reads from a database, and I want to differentiate between a stream not existing, vs existing but being empty:

public async IAsyncEnumerable<StreamMessage> ReadStreamBackwards(
    StreamId streamId,
    int fromVersionInclusive,
    int maxCount,
    bool prefetchJsonData = true,
    CancellationToken cancellationToken = default)
{
    Ensure.That(fromVersionInclusive, nameof(fromVersionInclusive)).IsGte(-1);
    Ensure.That(maxCount, nameof(maxCount)).IsGte(1);

    GuardAgainstDisposed();
    cancellationToken.ThrowIfCancellationRequested();

    Task<ReadStreamPage> ReadNext(int nextVersion, CancellationToken ct)
        => ReadStreamBackwardsInternal(streamId, nextVersion, maxCount, prefetchJsonData, ReadNext, ct); // returns a Page

    var page = await ReadStreamBackwardsInternal(
        streamId,
        fromVersionInclusive,
        maxCount,
        prefetchJsonData,
        ReadNext,
        cancellationToken);

    return page.Status == PageReadStatus.StreamNotFound 
        ? default(StreamNotFound) 
        : ReadStream(page, cancellationToken); // returns IAsyncEnumerable<StreamMessage>
}

private async IAsyncEnumerable<StreamMessage> ReadStream(
    ReadStreamPage page, 
    CancellationToken cancellationToken)
{
    while(page.Messages.Length > 0)
    {
        await foreach(var message in FilterExpired(page, cancellationToken))
        {
            yield return message;
        }

        if(page.IsEnd)
        {
            break;
        }

        page = await page.ReadNext(cancellationToken).NotOnCapturedContext();
    }
}

Then then caller could simply do

var messages = await streamStore.ReadStreamBackwards(...);
if (messages is StreamNotFound) { ... }

My alternatives are:

• Throw an exception. However, this is not an exceptional case.
• Return a tuple. it looks incredibly ugly for consumers though:

var messages = await (await store
    .ReadStreamForwards(streamId, StreamVersion.Start, 5))
    .messages
    .ToArrayAsync();

Is it at all possible to get what I want without resorting to the alternatives?

[ufcpp]

yield break?

[thefringeninja]

That will not help me differentiate between a stream that does not exist and a stream that does exist but has nothing in it.

[yaakov-h]

Seems like you might need Task<IAsyncEnumerable<StreamMessage>>?

[handyscripts]

Solved it for me. Thanks

[thefringeninja]

Same issue as the tuple solution. I would have to await it twice which is ugly.

[HaloFour]

You're dealing with two different asynchronous operations, using await twice is pretty much expected. But each await would be different, in that the first is scalar and the second is for the contents of the stream itself. Considering you want to replace the IAsyncEnumerable<T> itself with this "StreamNotFound" value I don't see an alternative.

[HaloFour]

Actually, if you declare ReadStreamBackwards as returning async Task<IAsyncEnumerable< StreamMessage>> that pretty much gets you the syntax you want:

var result = await ReadStreamBackwards(...);
if (result is StreamNotFound) {
    Console.WriteLine("The stream wasn't found, yo.");
    return;
}
var messages = await result.ToArrayAsync();
// do stuff with the array here

Sure, if you plan on consuming the IAsyncEnumerable<T> asynchronously you'll have to use a second await. You don't really have a choice there. You don't know whether or not the result is StreamNotFound until that first await given that operation is asynchronous.

The method can't return an IAsyncEnumerable<T> that then becomes StreamNotFound later. The compiler has to implement the internals of that interface through a state machine. And none of the code within the async iterator actually executes until you call MoveNextAsync (or call something else that does).

[Korporal]

@thefringeninja - Why not consider adding an out argument of type bool to ReadStreamBackwards that indicates presence of stream (or not) returning an empty sequence when you return that out argument as false?

Or return an new object that contains a bool property member and an async IAsyncEnumerable<StreamMessage> member? If the bool is true then enumerate the enumerable member. When you return that object withe bool false, make the enumerable empty.

[canton7]

@Korporal It's async. Async methods can't have out parameters.

[Korporal]

@canton7 @thefringeninja - OK so create a class that implements something like this:

    public interface IExistentialAsyncStream
    {
        IAsyncEnumerable<StreamMessage> BackwardMessageStream { get; }
        bool StreamExists { get; }
    }

and have ReadStreamBackwards return and instance of that class.

or possibly:

    public interface IExistentialAsyncStream<T>
    {
        IAsyncEnumerable<T> BackwardMessageStream { get; }
        bool StreamExists { get; }
    }

[YairHalberstadt]

I usually use tuples for this @Korporal

[thefringeninja]

After thinking about it I think I will go with @Korporal's or @YairHalberstadt's suggestion and just wrap it, and bite the bullet on the double async.

However, after playing with this, it seems to me anyway that something is missing. I can return a 'task like object' in an async method, but I cannot implement IAsyncEnumerable with some extra properties and combine that with yield. That's ok though, I can work around it.

Thanks for all the help!

[vladd]

I wonder why do you need to specialcase empty stream? You surely don't specialcase empty IEnumerable<T>, do you?

[thefringeninja]

Because this is for a stream store database - event sourced, domain driven design.

https://github.com/yreynhout/AggregateSource/blob/master/src/EventStore/AggregateSource.EventStore.IntegratedTests/AsyncRepositoryTests.cs#L73
vs
https://github.com/yreynhout/AggregateSource/blob/master/src/EventStore/AggregateSource.EventStore.IntegratedTests/AsyncRepositoryTests.cs#L82

[Korporal]

After thinking about it I think I will go with @Korporal's or @YairHalberstadt's suggestion and just wrap it, and bite the bullet on the double async.

However, after playing with this, it seems to me anyway that something is missing. I can return a 'task like object' in an async method, but I cannot implement IAsyncEnumerable with some extra properties and combine that with yield. That's ok though, I can work around it.

Thanks for all the help!

@thefringeninja - I'm curious about the problem you're seeing and I'd like to understand it more, can you possibly craft a simpler example of the problem so I can more easily see it? Get rid of stream and backwards and see if you can give a simple abstract example of the problem you're seeing.

My solution was just a way of representing what I thought you were driving at, but it seems its deeper...

[thefringeninja]

Well, i've been iterating on this a lot today, and figured out how to get rid of the double await. Turns out I don't need it if I implement IAsyncEnumerable<StreamMessage> and use some callback methods in its constructor:

public async IAsyncEnumerable<StreamMessage> ReadStreamBackwards(
    StreamId streamId,
    int fromVersionInclusive,
    int maxCount,
    bool prefetchJsonData = true,
    CancellationToken cancellationToken = default)
{
    Ensure.That(fromVersionInclusive, nameof(fromVersionInclusive)).IsGte(-1);
    Ensure.That(maxCount, nameof(maxCount)).IsGte(1);

    GuardAgainstDisposed();
    cancellationToken.ThrowIfCancellationRequested();

    Task<ReadStreamPage> Read(CancellationToken ct)
        => ReadStreamBackwardsInternal(streamId, fromVersionInclusive, maxCount, prefetchJsonData, ReadNext, ct); // returns the first page

    Task<ReadStreamPage> ReadNext(int nextVersion, CancellationToken ct)
        => ReadStreamBackwardsInternal(streamId, nextVersion, maxCount, prefetchJsonData, ReadNext, ct); // returns the next Page

    return new ReadStreamResult(Read, FilterExpired);
}

and ReadStreamResult uses its own IAsyncEnumerator<StreamMessage> on the inside, and has extra properties including if the stream was found or not.