Skip to main content

Time and cancellation

Some expectations take time: they await a task or an asynchronous delegate, enumerate an IAsyncEnumerable<T>, wait for a condition, an event or a callback, or measure how long a delegate runs. A timeout or a CancellationToken limits how long such an expectation may take, and every expectation reports them the same way.

Timeout​

You can set a global timeout that is applied for all expectations, e.g. in an assembly-level setup (see global defaults):

using aweXpect.Customization;

// Sets a global timeout of 10 seconds
Customize.aweXpect.Global.Settings().TestCancellation
    .Set(TestCancellation.FromTimeout(TimeSpan.FromSeconds(10)));

Like all customization options, the setter returns an IDisposable that removes the timeout again on Dispose(). Without Global, the timeout only applies to the current async flow, e.g. to a single test.

You can also apply a timeout on individual expectations, using the WithTimeout(TimeSpan) method:

IAsyncEnumerable<Track> playlist = // ...
await That(playlist).All().Satisfy(track => track.PlayCount > 0)
    .WithTimeout(TimeSpan.FromSeconds(10));
note

The tighter timeout wins. A local timeout that is longer than the global one, or than the limit of the expectation itself (e.g. ExecutesIn().AtMost(…) or Throws().Within(…)), does not loosen it.

How WithTimeout applies to Expect.ThatAll and Expect.ThatAny is described in combining expectations.

Repeated and infinite timeouts

Unlike other options, calling WithTimeout more than once applies the shortest timeout. Timeout.InfiniteTimeSpan imposes no limit.

CancellationToken​

You can set a global provider for getting a CancellationToken that is applied for all expectations:

// Uses the CancellationToken from the test context
Customize.aweXpect.Global.Settings().TestCancellation
    .Set(TestCancellation.FromCancellationToken(() => TestContext.Current.CancellationToken));

The setter again returns an IDisposable that removes the provider on Dispose().

note

The global timeout and the global CancellationToken are the same setting, Settings().TestCancellation, which holds either a timeout or a provider: the second Set replaces the first. A WithTimeout(…) on an individual expectation still applies together with a global CancellationToken.

TestCancellation is created with one of:

  • TestCancellation.FromTimeout(TimeSpan timeout), which cancels the CancellationToken that is used internally and forwarded to the delegates after the given timeout.
  • TestCancellation.FromCancellationToken(Func<CancellationToken> cancellationTokenFactory), which uses the returned CancellationToken internally and also forwards it to the delegates.
  • TestCancellation.None(), which applies neither a timeout nor a CancellationToken, e.g. to switch off a global TestCancellation in the current async flow.

You can also apply a CancellationToken on individual expectations, using the WithCancellation(CancellationToken) method:

using CancellationTokenSource cts = new(TimeSpan.FromSeconds(1));
IAsyncEnumerable<Track> playlist = // ...
await That(playlist).All().Satisfy(track => track.PlayCount > 0)
    .WithCancellation(cts.Token);
note

A local CancellationToken replaces the global one instead of being applied additionally. If you need both, pass a linked cancellation token.

How WithCancellation applies to Expect.ThatAll and Expect.ThatAny is described in combining expectations.

The timeout and the CancellationToken are also forwarded to a delegate that accepts a CancellationToken parameter.

Outcome​

A timeout and a cancellation are reported the same way by every expectation, whether it awaits a Task<T> subject, an asynchronous Whose member or delegate, enumerates an IAsyncEnumerable<T>, waits for a Signaler or for recorded events, or retries with Within(…) or Eventually():

  • When a timeout elapses (WithTimeout or TestCancellation.FromTimeout), the expectation fails with "did not finish within …" and a TimeoutException as inner exception.
  • When the CancellationToken is canceled (WithCancellation or TestCancellation.FromCancellationToken), the expectation is inconclusive: "could not be verified, because the evaluation was already canceled". It neither passes nor throws the OperationCanceledException, so e.g. DidNotSignal() does not pass because the cancellation ended the wait.

An inconclusive expectation throws the exception that your test framework uses for this outcome, see inconclusive.

Awaited tasks, async enumerables and cancellations from the delegate itself

A timeout or a cancellation also stops waiting for a task that the expectation awaits, such as a Task<T> subject or the task returned by an asynchronous delegate, even if it ignores the CancellationToken. A synchronous delegate cannot be abandoned and runs to completion. The outcome is the same whether the task was abandoned or reacted to the cancellation itself. With Eventually(), the timeout bounds each evaluation in the same way.

The CancellationToken of the expectation, which includes the timeout, is passed to an IAsyncEnumerable<T> subject, and the expectation stops waiting for the next item once it is canceled, even if the enumerable ignores the token. After that, the enumerable is not advanced any further.

An expectation that needs an item after the cancellation never reads the cancellation as the end of the enumerable, regardless of whether it occurs while waiting for an item or between two items. Expectations like HasCount or IsEmpty list the items received so far.

The enumerator is disposed when the expectation ends. A timeout or a cancellation also stops waiting for a DisposeAsync that does not complete. The outcome is already decided at that point and does not change: a met expectation stays met, and a failure is reported as it is.

A timeout or a cancellation only stops reading the items that still have to come from a collection, e.g. from a lazily evaluated IEnumerable<T>. A collection that is already complete in memory, such as an array or a List<T>, is evaluated completely like any other value, even when the timeout elapsed while a delegate created it. The same applies to a collection whose items an earlier expectation on the same subject has already read to the end.

An OperationCanceledException that a delegate throws for its own reasons, while neither the timeout elapsed nor the CancellationToken was canceled, is an ordinary exception: e.g. DoesNotThrow() fails with "did throw an OperationCanceledException".

Waiting​

Some expectations wait for something to happen. Within(…) limits how long they wait, and without it they behave as follows:

ExpectationWithout Within(…)
Satisfies(…), CompliesWith(…) and their negationsdoes not wait
Eventually()retries until the expectations are met, at most DefaultEventuallyTimeout (30 s)
Triggered(…), DidNotTrigger(…)does not wait, only the events recorded so far count
Signaled()waits until the callback is signaled, at most DefaultSignalerTimeout (30 s)
DidNotSignal()waits the full DefaultSignalerTimeout (30 s), unless the callback is signaled

Both defaults can be changed in the settings. When a default applied, the failure message names the wait, e.g. has never recorded the callback within 0:30 or eventually is equal to 2 within 0:30.

A condition​

When an object changes in the background, Within(…) lets Satisfies or CompliesWith wait until the object meets the condition, and DoesNotSatisfy or DoesNotComplyWith until it no longer does:

using aweXpect.Chronology; // from the aweXpect.Chronology package

Track track = new() { IsPlayed = false };
List<Track> playlist = new();
// Start a background task that plays the track and fills the playlist

await That(track).Satisfies(x => x.IsPlayed).Within(2.Seconds());
await That(playlist).CompliesWith(x => x.HasCount().GreaterThanOrEqualTo(4)).Within(2.Seconds());

The condition is checked again every DefaultCheckInterval (defaults to 100ms). Append CheckEvery(…) after Within(…) to change the interval for a single expectation:

using aweXpect.Chronology; // from the aweXpect.Chronology package

Track track = new();

await That(track).Satisfies(x => x.IsPlayed).Within(2.Seconds()).CheckEvery(50.Milliseconds());
Timeouts and cancellation while checking

As for Eventually(), a shorter WithTimeout or global timeout ends the checks with "did not finish within …", and a cancellation makes the expectation inconclusive.

Eventually​

Some values only become correct after a short delay, e.g. because a background task is still running. Instead of waiting for a fixed amount of time, Eventually() re-evaluates a delegate until the expectations are met. Because only the subject is re-evaluated, all expectations work as usual, including And, Or and Because:

Track track = new();
// Start a background task that plays the track

await That(() => track.PlayCount).Eventually().IsGreaterThan(5);
await That(() => track.Title).Eventually().IsNotNull().And.StartsWith("Let");

The delegate is re-evaluated every DefaultCheckInterval (defaults to 100ms) until the timeout configured in DefaultEventuallyTimeout (defaults to 30s) expires. You can overwrite the timeout per expectation with Within and the interval with CheckEvery:

using aweXpect.Chronology; // from the aweXpect.Chronology package

Track track = new();

await That(() => track.PlayCount).Eventually().Within(5.Seconds()).CheckEvery(50.Milliseconds())
  .IsGreaterThan(3)
  .WithTimeout(2.Seconds());

When the play count never exceeds 3, the retries end after 2 seconds, because the timeout is shorter than Within.

Within(Timeout.InfiniteTimeSpan) retries until the expectations are met or the expectation is canceled.

An exception thrown by the delegate counts as an unmet expectation and is retried. When the timeout expires while the delegate is still throwing, the expectation fails and the last exception is reported as the cause of the failure.

In addition to Func<T>, the asynchronous variant Func<Task<T>> is supported, and on .NET 8 or later also Func<ValueTask<T>>; each of them also accepts a CancellationToken (see the delegates page):

Func<Task<int>> loadPlayCountAsync = () => Task.FromResult(6);

await That(() => loadPlayCountAsync()).Eventually().IsGreaterThan(5);
How timeouts bound the retries

WithTimeout does not change the timeout of the retries, but cancels the evaluation like everywhere else, so the tighter timeout wins: a WithTimeout or a global TestCancellation.FromTimeout that is shorter than Within ends the retries and fails the expectation with "did not finish within …", and a longer one does not extend them. A cancellation before the timeout expires (via WithCancellation or TestCancellation.FromCancellationToken) makes the expectation inconclusive instead of failed.

The timeout also bounds each evaluation: an evaluation that is still running when the timeout is used up is abandoned, even if the delegate ignores its CancellationToken, which is canceled at that point, and the expectation fails with "did not finish within …" and a TimeoutException as inner exception. The last evaluation, which is made when the timeout is used up, still gets one check interval (at most the timeout) to finish. An expectation that waits while it checks the value, e.g. for the next item of an IAsyncEnumerable<T>, is canceled in the same way and fails with the same message. A synchronous delegate cannot be interrupted, so an evaluation that returns after that point fails the same way, whatever its result. Only Within(TimeSpan.Zero), which makes a single evaluation, does not bound it, for a synchronous and an asynchronous delegate alike; a WithTimeout or a global timeout still does.

Events​

Within(…) waits up to the given time for the expected events and finishes successfully as soon as they are recorded:

using aweXpect.Chronology; // from the aweXpect.Chronology package
using aweXpect.Recording;

class Player
{
  public event EventHandler? TrackStarted;
  public void Play(string title) => TrackStarted?.Invoke(this, EventArgs.Empty);
}
Player player = new();
IEventRecording<Player> recording = player.Record().Events();

_ = Task.Delay(1.Seconds()).ContinueWith(_ => player.Play("Let It Be"));

await That(recording).Triggered(nameof(Player.TrackStarted)).Within(3.Seconds());

Expectations with an upper bound (DidNotTrigger, Never(), AtMost(…), Exactly(…)) wait out the whole timeout and only return early when one event too many is recorded:

using aweXpect.Chronology; // from the aweXpect.Chronology package

IEventRecording<Player> recording = player.Record().Events();

// Waits for 3 seconds and expects that no TrackStarted event is triggered in that time
await That(recording).DidNotTrigger(nameof(Player.TrackStarted)).Within(3.Seconds());

Callbacks​

Within(…) limits how long to wait for a callback to be signaled:

using aweXpect.Signaling;

Signaler<string> signaler = new();

await That(signaler).Signaled().Within(TimeSpan.FromSeconds(5))
  .Because("it should take at most 5 seconds to complete");

An expectation without an upper bound (e.g. AtLeast) succeeds as soon as enough callbacks were signaled. Expectations with an upper bound, including DidNotSignal(), wait out the whole timeout and only return early when one signal too many is received. A canceled CancellationToken makes the expectation inconclusive, so use Within(…) to limit how long to wait.

On a virtual clock​

WithTimeProvider(timeProvider) lets an expectation wait, time out and measure on the clock of a TimeProvider instead of in real time, so that a test of a long wait does not take long. Pass any TimeProvider, e.g. the FakeTimeProvider from the Microsoft.Extensions.TimeProvider.Testing package, which the code under test can share:

using aweXpect.Chronology; // from the aweXpect.Chronology package
using aweXpect.Core;
using Microsoft.Extensions.Time.Testing; // from the Microsoft.Extensions.TimeProvider.Testing package

FakeTimeProvider timeProvider = new();
Track track = new(timeProvider); // plays on the same clock

async Task ExpectIsPlayed()
  => await That(track).Satisfies(x => x.IsPlayed).Within(2.Seconds())
    .WithTimeProvider(timeProvider);

Task expectation = ExpectIsPlayed();
timeProvider.Advance(2.Seconds());
await expectation;

It applies to the waits, timeouts and measured durations of the expectation, also to all expectations combined in Expect.ThatAll or Expect.ThatAny, but not to code that waits outside of it, e.g. in the subject. Without WithTimeProvider(…), an expectation runs on TimeProvider.System, the real clock.

warning

The waits and timeouts of the expectation are timers of the TimeProvider. A FakeTimeProvider only fires its timers when the test advances it, so an expectation that waits only progresses when the test advances the clock: start the expectation without awaiting it, advance the clock, and then await it. Awaiting it first waits forever, unless a timeout in real time, e.g. of the test framework, ends the test.

Execution time​

ExecutesIn() applies its upper bound as timeout: the maximum of AtMost, the end of the Between range, or the expected time plus the tolerance. A tighter timeout, e.g. from WithTimeout(…), still applies. A delegate that accepts a CancellationToken is canceled once the upper bound elapsed, and the expectation fails with "did not finish within …" instead of hanging. AtLeast has no upper bound and therefore applies no timeout. The duration of Throws().Within(…) is applied as timeout the same way. A task is already running when the expectation receives it, so only the duration that remains is measured.

Delegates that ignore the timeout

The task of an asynchronous delegate is abandoned once the timeout elapsed, even if the delegate ignores or does not accept a CancellationToken, and the expectation fails the same way. A synchronous delegate cannot be interrupted and runs to completion, however long that takes; neither WithTimeout nor WithCancellation changes that. If it returns after the timeout elapsed, the expectation fails the same way. ExecutesIn() and Throws().Within(…) report the duration it took instead, when the timeout is their own upper bound or the duration also violates their limit.