Assertions are a crucial part of tests. Even though most developers realize this, they often are rather limited. After all, they can be cumbersome to write. For example, if you’re running E2E tests for an endpoint that returns a list of 10 elements with 10 properties each - which isn’t even near the amount of data that often flows through production systems - you already have to write assertions for 100 properties. Wouldn’t it be easier to, say, assert the length of the list is 10 and move on? Or, if you’re feeling generous, assert the IDs of the elements, and assume the rest is fine too. There are many examples of such tests, and it isn’t a surprise given how much work writing and maintaining proper assertions would take in this case. This doesn’t have to be the case - you can write assertions that cover cases like that in just one line using snapshots.
Snapshot testing is a simple idea - instead of writing assertions by hand, you compare the result of your test with, well, a snapshot - a value captured during earlier tests. In the case of the previous example with a list, it’s as easy as serializing it to JSON and comparing it with a value that is already established to be correct. If there are any mismatches between those two, it’s a fail, if they’re the same - everything’s fine.
E2E testing an API endpoint
Before I show you how to do that, let’s see what we’re trying to test. I’ve written a simple API returning weather forecasts, which you’ve probably already seen if you’ve played around with any of the .NET example projects. Here’s the JSON returned by this API:
[
{
"date": "2023-06-20",
"temperatureC": 30,
"temperatureF": 85,
"summary": "Bracing"
},
{
"date": "2023-06-21",
"temperatureC": -11,
"temperatureF": 13,
"summary": "Warm"
},
{
"date": "2023-06-22",
"temperatureC": -8,
"temperatureF": 18,
"summary": "Chilly"
},
{
"date": "2023-06-23",
"temperatureC": 34,
"temperatureF": 93,
"summary": "Warm"
},
{
"date": "2023-06-24",
"temperatureC": -7,
"temperatureF": 20,
"summary": "Hot"
}
]
I’m not going to share the code for the API or the test setup, as it’s very rudimentary. For the snapshot testing, I’m going to use a library called Verify and xUnit. Here’s the test:
[UsesVerify]
public class E2eTests
{
private readonly HttpClient _client;
public E2eTests()
{
_client = new ApplicationFixture().CreateClient();
}
[Fact]
public async Task Test()
{
var result = await _client.GetFromJsonAsync<WeatherForecast[]>("api/weather-forecast");
await Verify(result);
}
}
The attribute [UsesVerify] above the class declaration is required by the Verify library. The line await Verify(result); is the assertion. We can run the test now, but it’s going to fail - it doesn’t have anything to compare with. I’ve got a plugin for Rider installed to see the result you can see below - depending on your setup it can look a little bit differently.
We can use this window to check what value we’ve received, and if we’re happy with it save it as a verified result. If I do that, the library will create a file called E2eTests.Test.verified.txt with the accepted result. That’s the reference for future tests. We can run it again, and it’s going to be a success!
What if we change the result during further development, either intentionally or by introducing a bug? Here’s the result of running the test again after modifying the result returned by the endpoint:
As you can see, we get the same window as before, but now it shows both the received and verified values with a highlight of the differences between them. We can save the updated values if that was an intentional change or, if it’s a result of a bug, fix the code.
There’s more to an HTTP call though - its consumers could depend on a specific status code or headers. The test can be modified to check the HttpResponseMessage instead of the content:
var result = await _client.GetAsync("api/weather-forecast");
await Verify(result);
But then the actual content isn’t present in the result, as internally it’s represented by a stream that won’t get serialized to JSON. Fortunately, there’s another package called Verify.Http that can be used to do that. After installing this package and modifying the test like so:
VerifyHttp.Initialize(); // this can be initialized elsewhere, I just put it here for simplicity
var result = await _client.GetAsync("api/weather-forecast");
await Verify(result);
There’s going to be much more than just the JSON content in the Verify result, like HTTP protocol version, the HTTP verb used in this request, headers and the details of the request.
Testing the integration with a database
Snapshot testing is useful in contexts other than E2E tests of the API as well. Let’s say you’d like to run an integration test checking whether we’re saving the data to the database correctly. It would not be the first test I’d write, as it’s better to test the behavior of the system through its inputs and outputs rather than testing its internals, but it can be difficult in some scenarios or you might want to check the integration in addition to other tests. If you’re using Entity Framework, there’s another Nuget package for you - Verify.EntityFramework. Here’s an example of a test that adds a WeatherForecast through a REST API and verifies the state of the data in the database.
[Fact]
public async Task Test()
{
await using var dbContext = GetDbContext();
await _client.PostAsync("api/weather-forecast",
JsonContent.Create(new WeatherForecast
{ Date = DateOnly.FromDateTime(DateTime.Now), TemperatureC = 30 }));
await Verify(dbContext.AllData()).AddExtraSettings(
serializer =>
serializer.TypeNameHandling = TypeNameHandling.Objects);
}
private static WeatherForecastContext GetDbContext()
{
var optionsBuilder = new DbContextOptionsBuilder<WeatherForecastContext>();
optionsBuilder.UseNpgsql("your-connection-string-here");
return new WeatherForecastContext(optionsBuilder.Options);
}
And here’s the received result:
[
{
$type: WeatherForecast,
Date: Date_1,
TemperatureC: 30,
TemperatureF: 85
}
]
Checking the state of the entire database might not solve your problem, but there are plenty of other options - if instead of creating another DbContext we used a mechanism provided by the package to record usage of the DbContext used by the application we could verify what entities it’s got in its change tracker or what SQL statements it’s running. If you’re not using Entity Framework, there’s a package that allows for similar tests for SqlServer, MongoDB, RavenDB and Azure Cosmos.
UI tests
Another feature I’d like to showcase is testing the UI. It might not be necessary for your project and depending on your frontend framework of choice you might run your tests in JS/TS, and the tooling is already there for tests like that. Even if you’re using different tools, the idea of snapshot testing still stands. Let’s take a look at how to use Verify and Verify.HeadlessBrowsers package to test a UI of a Blazor application.
[UsesVerify]
public class UITests
{
[Fact]
public async Task Test()
{
VerifyPlaywright.Initialize(installPlaywright: true);
var playwright = await Playwright.CreateAsync();
var browser = await playwright.Chromium.LaunchAsync();
var page = await browser.NewPageAsync();
await page.GotoAsync("https://localhost:44330/fetchdata");
await Verify(page);
}
}
In this case, the code is initializing Playwright, instructing it to retrieve a website and verifying its content. The received value is going to be the entire HTML of the web page, but also a screenshot that would cause a fail in case the content is the same, but for example one of the images doesn’t load properly or there was a change in the CSS.
There are tons of other options for using Verify - you can test PDF files, images, various libraries and integrations and multiple points of extensibility to write your own converters and comparers for your specific use cases. Before you do that, though, there are a few things to remember when doing snapshot testing. Firstly, it’s a little bit time-consuming to set up - you have to get everyone on your team on board with testing this way and have them set up their environments with plugins to see the diff tool. Secondly, you have to read the test results carefully before you accept them as verified - as carefully as you would read the assertions. It’s easy to just accept all received values as verified, but by doing that you’re undermining your efforts to properly test your code. Thirdly, you should keep the *.verified.txt files in your repository - I think it’s best to place them in a separate folder using the configuration options Verify provides.