Most of the time we use different approach: each integration test run in a separate transaction and automatically rollbacks at the end of the test. All data tracked by database itself and deleted after test. Of course there are some limitations but it's super simple and effective. Needless to say that library also seems to be interesting to investigate further. Thank you, Nick!
I've haven't gotten too far with automated integration tests against a real db, but this is what I have learned to use too. It seems very straight and worked well for my needs. Hopefully there aren't too many scenarios in the real world that need to be tested across many transactions, but if they are maybe they should get some special treatment anyways
What i do is create a "database per test". This have a overhead of create/restore the db for each test, but allow you to run all tests in parallel, wich is a really big win if you use a CI to validate the changes.
Nick says something like "it observes it, creates some sql and resets it back to a previous point" which would be fantastic, but the actual description from the github repo is "Respawn resets the database back to a clean, empty state by intelligently deleting data from tables." Resetting back to a previous point and resetting back to a clean, EMPTY state are two very different things. So does it actually reset back to a previous point or does it just empty the tables you tell it to empty?
What I do is at the beginning of the integration test suite run a database is created and seeded in MSSQLLocalDB and a snapshot is taken, then each test class restores the database to the snapshot, runs its tests against it, then tears it down. The solution limits the number of total databases that can be spun up. I find it to be pretty quick and allows tests classes to be run in parallel which is helpful becase we have 1000+ integration tests. However, if I didn't need parallelism and was looking for an easy to implement solution, Respawn definitely seems like a good option. Thanks for the info!
I imagine the tests still need to run sequentially? This has been really helpful. We have this very same ongoing pain point at work - having some tests that require a clean slate of a database and it taking a while (relatively) to set up.
@@llindeberg Because the reality is there will be race conditions in more complex integration tests. Especially when performing CRUD style operations over a shared context. Parallelization isn't always possible.
@@llindeberg It obviously depends in the situation. I do it before every test because my 2k tests run in 1 minute and I'm ok with it. It's easier than architecting everything sequentially. I'd rather spin up more databases and split collections than architect every test to be independent safe. That's a personal preference thing though
You can place it at the start then. ultimately it doesn’t matter. Start or end, rely or not rely, if the reset doesn’t happen before the next test runs, you are screwed
First, then your last test is never cleaned. Two, I find it really very weird as an approach, to clean others first... Not to mention that you do it for nothing on your first test. And it's you who are cleaning the work of others, not the way around. Idk, seems weird to me, but if it works for you then it's not a catastrophy. But, still, a bit weird to me
@@RaMz00z I'd rather do cleanup first and know for sure it was done and not depend on other tests. Guessing and not knowing for sure is a receipt for problems. In programming I'd rather take the defensive approach.
I really wish there were a library to really generate stub entities that respect column settings. Especially now that EF Core 6 requires non nullable fields to have a value at instantiation. I used to be able to only set a value for the field that's relevant to the test and now i need a bunch of junk too
That was never in question. The problem is that when you have operations such as "get all" it is very hard to deterministically return the data you want to assert against.
Great video, just my database docker image is taking 19 minutes to build because it's a big fat 2.5 gb database. What's your approach when you have this kind of SQL databases?
When it comes to integration tests I would just create a docker container update it using the migration scripts. when running unit tests I would just use a memory provider for entity framework (not 100% accurate but good enough)
The moment you introduce the memory provider of EF your tests are useless because they don't reflect the integration point. They are not integration tests at that point but rather functional tests. NEVER use in-memory database replacements when doing integration testing. It is the biggest mistake people make.
@@nickchapsas i might move not have bean clear but what i meant was that for unit tests i use the memory provider and for integration tests i use a clean Docker database.
You create a clean docker database per test? And you run migrations per database? This is such a slow and pointlessly inefficient approach for your resources.
Hi Nick, great stuff. Txs Any way to find the RespawnDatabase solution sample online somewhere ?
ปีที่แล้ว
Hi Nick!, this is very helpful locally all works fine, but if we need run this on Jenkins for example, we need to run in something like docker in docker?
Are these tests still running serially? We randomly generate PG schemas and allow tests to run in those. Individual tests can be grouped to share schemas (and associated setup data) or run stand alone. Integration tests run in parallel.
Looks interesting, one question: How does this stack up when you want to run tests in parallel? For example you have a couple thousand tests and it's taking a while so opting to run in parallel.
I would question why you have a couple of integration thousand tests in a single solution. For integration tests to run in parallel the data needs to be isolated and this is usually done with separate databases. I can't even begin to imagine the cost of spinning up 2k databases in docker to run your tests in parallel. This will be way more efficient and faster
@@martinshields7122 I have a project with around 700 integration tests using Respawn and it only takes a few seconds to run through them all using a MSSQL database on my localhost (test not run in parallel). I'm sure this will save you time.
@@nickchapsas I found creation and deletion of databases (not servers) fast after each test run in parallel. Maybe db pool with respawn would be a good idea. Most test runners run couple of dozens test in parallel only (though I am not sure), so maximum of dbs running in parallel would be the same. I like integration tests as they if done well (BDD style) can allow me to refactor internal code however I want without worrying about fixing tests (much if at all).
I would run them with in memory db, and for each test a different instance. Probably runs in 50ms or less :) parallel is also no issue. I know people argue this isn’t real enough but imo database adapters and software are tested elsewhere. To be honest; your Docker container is also not the actual “real” database that will be used :) I would probably do in memory locally and ci until you’re happy with your code and want to merge it or deploy it, then do the final check with the actual target db. In the meantime you save a lot of time and energy (literally). Tests are imo not about absolute safety, but about minimizing risk as best as possible at a reasonable cost. Developers waiting seconds, minutes, hours, is imo not worth it.
Running against an inmemory db defauts the purpose completely. It's the most common mistake that people do when it comes to integration testing. Docker, assuming you match the version of the database in production, absolutely is the real database. The moment you remove the real integration point to your test, your test becomes a functional test, which isn't 100% useless, but its value drops significantly.
@@nickchapsas The point is that you don't need to create two separate tests, you can create one, but run it fast/medium secure while developing, and slow/full secure just before the actual integration or deployment.
@@patrickroza3073 I'm very much against that approach. Your "fast" test is useless if it only produces reliable results when it's run against the real database. The first level of execution adds nothing to the table but the illusion of coverage. The actual integration point later is the only thing that matters in your example
@@nickchapsas I think Jimmy is on point in the conclusion of his blog post about in memory db testing; perhaps I’m lucky the applications are trivial enough :) (Im mostly using document dbs, no EF, not even .NET, but have in the past)
Hi. I recently make use of testcontainers. It works like a charm, UNTIL I decided to use redis. There will be an error of I use redis container: No conection is active/available to service this operation: Can you help me?
Hey Nick, great video! I don't reset my integ test database at all. What I do instead is I generate a unique test key (an integer) for each test. I do that using a special "TestExecution" table with an identity column. I insert a record into this table at the beginning of each test, then get the value of the identity column from the inserted record. Once I have a test key, I use it to construct names of my domain objects. For example, all tests that create a Customer object would use $"Test Customer {_testKey}" as the name. This avoids data collisions between tests and eliminates the need to reset the database after each test.
Ive seen Jason Taylor using a SliceFixture with using static SliceFixture (in his clean testing repo). Could you do something similar or with a base class to avoid having to copy paste the IAsyncLifetime setup for each test class you create? Also interested to hear where you migrate data. I tend to avoid doing it in api startup because it can mess up things if you deploy to production. Would you get a DbContext instance in initializeasync through a scope and run a migration there?
Yeah you can totally create a DatabaseFixture and extend from that. You can run your migrations in the InitializeAsync method of the CustomerApiFactory class to only do it once per the whole app outside of application startup
@@nickchapsas Yea I'm trying that, but I can't figure out how I would get the IServiceScopeFactory to resolve the DbContext during InitializeAsync in the WebApplicationFactory. I guess you just manually new up one inside a using statement? But then there's the problem with passing the correct connectionstring.
I don't get it. If tests run in parallel, they still might clash with each other because resetting database can reset data for other test executing in parallel.
Great library, but I dislike the fact to reseed using SQL Scripts. In my projects, I reset and reseed the database using an Action with EF into my databasefixture combined with EnsureDeleted and EnsureCreated. The drawback is that I have to use a scaffolding of the database using EF only for the tests (because we use dapper). it could be great, if that library could seed the database using Models or Fluent methods.
I think you missed the point of what Respawn does. It doesn't seed your DB. It clears data from tables by doing smarter SQL delete operations that take into account FK dependencies. This is way faster than truncating or dropping and recreating tables with whatever tool (EF, raw SQL, etc.). You can still use EF migrations to create the database and seed data, and then configure Respawn to include/exclude tables that it should reset when you run it. That way you don't lose data in look up tables which may have been seeded via EF, for example.
What if you have tens of thousands of records and you try to delete one by one based on their ID. If you have a relational DB like Postgresql or MS SQL, your transaction log will be flooded with delete instructions making it grow. If you need to delete entire table probably a Truncate will serve you better. In case you don’t want to delete an entire table it would be better to delete full sets at once in a single statement.
I don't get why people complain about MediatR or Automapper@@nickchapsas ... Jimmy Bogard has been a big contributor to the .NET community, and he has always provided enough info & examples to usecases. Also when NOT to use his libraries (especially when it comes to automapper for example).
@@psyaviah It's not a matter of hate towards Jimmy Bogard who's one of the biggest brain in the .NET community. It's more that some people tend to think that his products are bad from a (extremist) design point of view. Like AutoMapper has to much magiic in it, and MediatR is supposedly bad as a CQRS tool. What Nick got wrong is that, despite him apperently being one of those detractors, those two libs are vastly liked and used, cumulating 400 Million downloads... I would argue though that a lot of people are wrongly using the libraries. So when you arrive in a shitty project that misuse them, you could think that it's the lib's fault, which is not. Jimmy Bogard himself talked a lot about this issue in his blog, and made some radical changes to AutoMapper specifically to forbid what he wrongly allowed in the early years. TL;DR; Most people like them, some hate them, but they're very good libraries when used as intended.
I think AutoMapper brings more problems than it solves, but maybe it's nice to use in contexts that I have not come across. In general, I'd rather have normal mapping classes that make it easily to check out where setters are called rather than having to deal with a possible mix of implicit or explicit magic mappings that are also harder to set up and debug.
@@zabustifu that's tha opinion. In a clan architecture environment, it never bothered me because I never seek setters that do mappings. And to find object dependencies, you look for the object. And the biggest advantage for me to modern auto Mapper is that it's usuable in Linq To Sql to create the select expression. This is dope af (and yes pretty performant for web usage)
If all the tests are running in parallel, wouldn't it still have conflicts as you're resetting only after each test class completed it tests. so if CreateCustomerControllerTests and DeleteCustomerControllerTests runs in parallel, the state of the database from DeleteCustomerControllerTests perspective can contains data from CreateCustomerControllerTests.
When the classes are on the same "collection" they don't run in parallel. For more information you can read the xunit documentation (Running Tests in Parallel). Put this together with the default behavior of xunit where tests of a class don't run in parallel and nothing is running in parallel now.
The bad think about Respawn is that reseting to clean state what means no data at all what can be problematic, if your DB migrations create some data (like "system user") or when your migrations takes ages (in my project it takes ~1min * amount of test = hudge amount of time) the other aproch is to recreate DB from backup (or clone if your database support it) but in very similar way as Nick did with Respawn
You can reset to a state that already has data as long as you exclude the table (for example for join tables). In my case I am resetting to a database state AFTER i have run my migrations.
@@BillyBraga Updated my comment to reflect the behavior. What I meant was that you can still have data in your database and reset to that version with data, as long as they are in excluded schemas or tables. It doesn't make sense to need data for your "clean state" in non-join tables.
Hey! I tested db snapshot restore after my big big initial migration is applied of ~10 year project, and it execute reset very fast. But could Respawn work with HiLo id generator or I need to disposs ef instance after each test? because currently seems it is not support reset HiLo on EF cache level.
Nice way to write tests but isnt it a disadvantage that you cant use the same test against a real database after a deploy? When you manually delete things after each test, the same set of tests can be reused
Nick Chapsas must be one of the most beloved God's netrunner , a real demon at work . We love your content which is super profesional and outstanding . Blessings !
So… do you think FastEndpoints package is a good „alternative”? It’s not quite the same, but fixes the same / similar thing MediatR was made for. I think both work well when used wisely.
@@Tosde96 I like both MediatR & FastEndpoints. We use both actually, just depends on the scope of the api or type of application you are creating imho.
Does your system work with false transactions in production? Using transaction scopes and rollbacks doesn't reflect how your app will actually perform and ultimately fail in production when you deploy it. It is also limiting depending on the ORM tech you are using and also can't be used if your operations span multiple transactions. It's a naive approach.
The video covers the main two approaches and you can see how the tests got faster. This is faster than dacpac, snapshot load or transaction rollback (and more flexible).
@@kaczaczacza Because you want your integration tests to catch problems your real system will encounter before it actually encounters them. The more you move the variables between how your real integration points work compared to your test ones, the more likely it is you will have false positives or false negatives in your tests
@@nickchapsas Ok, so what you are saying is: rollback may look like the same solution but it can bring with it some behaviour we cannot predict? And that's why it is better to simulate production as much as possible?
Because integration tests using in memory providers (either EF or SQLite) are 100% useless integration tests. They aren't integration tests all together. They are functinal tests since you aren't using your integration point.
@@nickchapsas What is your definition of an entire "integration test"? (I'm sure you have videos or courses for that :p). We define Integration tests (for an web-api microservice) as: a test that covers code in; controller endpoints, business logic (services, helpers etc) and database fetchers). We use EF core to both SQLServer and PostGreSql, so we essentially want to test our data-fetching layers linq-queries, and leave the translation to the specific database query-languages to EF-core. So in our case we happily use InMemory databases in our integration tests, and to separate tests from each, we create the InMemoryDB with a Guid.NewGuid name.
InMemory EF Core is the worst You can bet on. As @Nick already said. Slightly better is to use even Sqlite with :memory database. But the best is real db 🙂
Because it's slow. I use Mapster which is 4x times faster and can be 10x times faster when using source generators. And I like the Mapster api way more
I don’t think auto mapper has much of a use case. Whenever you change the type of a property you have to be aware of any type incompatibilities you might create. Auto mapper will throw a runtime error when it encounters these. So if you want to avoid these errors then as a dev you have to spend time worrying about mapping logic anyway and still worrying about any run time errors you might miss. Much better just to write a mapping function, errors appear at compile time, no real loss of dev time (regular expressions exist) and performance is better.
BWAHAHA! "Made by the same person who made AutoMapper and MediatR, but don't worry, people like THIS library." BURN! Poor Jimmy - he gets so much crap for his OSS stuff, the man's gotta have some thick skin.
This does not work when executing tests in parallel. That starts to hurt when you have alot of tests. Ideally you want a db per test for full isolation but it's expensive to create a new db everytime. You could use pooling to prevent having to recreate the whole db everytime and reset the db before returning it to the pool. I think that's more scalable than what's shown here but obviously more complex and not needed everywhere.
This is what I do: How bad is this? It works, but is very slow... Will be switching to this method you showed protected override IHost CreateHost(IHostBuilder builder) { builder.UseEnvironment("Testing"); var host = builder.Build(); var serviceProvider = host.Services; using (var scope = serviceProvider.CreateScope()) { var scopedServices = scope.ServiceProvider; var appDbContext = scope.ServiceProvider.GetService()!; appDbContext.Database.EnsureDeleted(); appDbContext.Database.Migrate(); var userManager = scopedServices.GetRequiredService(); var roleManager = scopedServices.GetRequiredService(); Seeder.SeedUsersAsync(userManager, roleManager).GetAwaiter().GetResult(); var seeder = new Seeder(appDbContext); seeder.SeedInstancesAsync().GetAwaiter().GetResult(); } host.StartAsync(); return host; } protected override void ConfigureWebHost(IWebHostBuilder builder) { builder .UseSolutionRelativeContentRoot("Solution/Project"); }
![ALLY - OH MY! [ REACTION ] 'Amarin Nitibhon' #ALLY_OHMY #ALLY #allynitibhon #แอลลี่](http://i.ytimg.com/vi/nX9qpHjVfbQ/mqdefault.jpg)
Most of the time we use different approach: each integration test run in a separate transaction and automatically rollbacks at the end of the test. All data tracked by database itself and deleted after test. Of course there are some limitations but it's super simple and effective. Needless to say that library also seems to be interesting to investigate further. Thank you, Nick!
This won’t work if your test spans multiple transactions.
I've haven't gotten too far with automated integration tests against a real db, but this is what I have learned to use too. It seems very straight and worked well for my needs. Hopefully there aren't too many scenarios in the real world that need to be tested across many transactions, but if they are maybe they should get some special treatment anyways
Just to say a big thank you Nick. You helped me reduce my integrations tests from 30 min to 1 min 30 !
In my 20 years of experience i did not need this. But a few days ago i stumbled upon that problem. Now i know how to do this. :D
What i do is create a "database per test". This have a overhead of create/restore the db for each test, but allow you to run all tests in parallel, wich is a really big win if you use a CI to validate the changes.
I just sat back and thought about all the times I created code to rebuild and teardown my databases. Wow. Great vid as usual.
Nick says something like "it observes it, creates some sql and resets it back to a previous point" which would be fantastic, but the actual description from the github repo is "Respawn resets the database back to a clean, empty state by intelligently deleting data from tables."
Resetting back to a previous point and resetting back to a clean, EMPTY state are two very different things.
So does it actually reset back to a previous point or does it just empty the tables you tell it to empty?
This was what I was hoping it did as well. Deleting all the records from all the tables is not helpful for my requirements.
What I do is at the beginning of the integration test suite run a database is created and seeded in MSSQLLocalDB and a snapshot is taken, then each test class restores the database to the snapshot, runs its tests against it, then tears it down. The solution limits the number of total databases that can be spun up. I find it to be pretty quick and allows tests classes to be run in parallel which is helpful becase we have 1000+ integration tests.
However, if I didn't need parallelism and was looking for an easy to implement solution, Respawn definitely seems like a good option. Thanks for the info!
How do you create the snapshot of the db?
Thank you for the great tutorial, I created a Base class for the Integration Tests so it is much less Boilerplate.
That was very helpful and insightful video, Nick. Thank you. I need to revisit my integration tests and improve some parts.
Hi Nick, Nice video!
Can you please share this project repo as a some kind of template for using Respawn?
Nice library, it was always issue for me to make integration tests without clashes.
I like this library. I was using EF Core to clean the database (of course I was using the same DbContexts I had for my application)
I happen to like both MediatR and Automapper. Respawn looks really cool though... definitely have to check it out!
Same here, why would anyone hate MediatR? :)
I imagine the tests still need to run sequentially?
This has been really helpful. We have this very same ongoing pain point at work - having some tests that require a clean slate of a database and it taking a while (relatively) to set up.
Yeah they run sequentially but because of the sophisticated reset logic and the WAF, they are very fast
@@nickchapsas how can tests running sequentially be ok? Way slower
@@llindeberg Because the reality is there will be race conditions in more complex integration tests. Especially when performing CRUD style operations over a shared context.
Parallelization isn't always possible.
@Nick @ColinM
Here's Jimmy saying he doesn't respawn every test: th-cam.com/video/Q_plyaxL5K4/w-d-xo.html
@@llindeberg It obviously depends in the situation. I do it before every test because my 2k tests run in 1 minute and I'm ok with it. It's easier than architecting everything sequentially. I'd rather spin up more databases and split collections than architect every test to be independent safe. That's a personal preference thing though
Great tool! Thank you so much. It'll help me a lot
Cool, I've recently implemented something similar with RestorePoint from Effort
I have strong feelings about reset being done at the start and not on dispose. Do not rely on others to do cleanup!
You can place it at the start then. ultimately it doesn’t matter. Start or end, rely or not rely, if the reset doesn’t happen before the next test runs, you are screwed
First, then your last test is never cleaned.
Two, I find it really very weird as an approach, to clean others first...
Not to mention that you do it for nothing on your first test.
And it's you who are cleaning the work of others, not the way around.
Idk, seems weird to me, but if it works for you then it's not a catastrophy. But, still, a bit weird to me
@@RaMz00z I'd rather do cleanup first and know for sure it was done and not depend on other tests. Guessing and not knowing for sure is a receipt for problems. In programming I'd rather take the defensive approach.
I really wish there were a library to really generate stub entities that respect column settings. Especially now that EF Core 6 requires non nullable fields to have a value at instantiation. I used to be able to only set a value for the field that's relevant to the test and now i need a bunch of junk too
Thanks
Why should our tests need clean database to pass? Feature should work regardless if I have something in the database or not.
That was never in question. The problem is that when you have operations such as "get all" it is very hard to deterministically return the data you want to assert against.
Great video, just my database docker image is taking 19 minutes to build because it's a big fat 2.5 gb database. What's your approach when you have this kind of SQL databases?
When it comes to integration tests I would just create a docker container update it using the migration scripts. when running unit tests I would just use a memory provider for entity framework (not 100% accurate but good enough)
The moment you introduce the memory provider of EF your tests are useless because they don't reflect the integration point. They are not integration tests at that point but rather functional tests. NEVER use in-memory database replacements when doing integration testing. It is the biggest mistake people make.
@@nickchapsas i might move not have bean clear but what i meant was that for unit tests i use the memory provider and for integration tests i use a clean Docker database.
You create a clean docker database per test? And you run migrations per database? This is such a slow and pointlessly inefficient approach for your resources.
@@Petoj87 If you don't mock you db and use in-memory provider, your tests are not considered as unit tests
Hi Nick, great stuff. Txs
Any way to find the RespawnDatabase solution sample online somewhere ?
Hi Nick!, this is very helpful locally all works fine, but if we need run this on Jenkins for example, we need to run in something like docker in docker?
Are these tests still running serially?
We randomly generate PG schemas and allow tests to run in those. Individual tests can be grouped to share schemas (and associated setup data) or run stand alone. Integration tests run in parallel.
Looks interesting, one question: How does this stack up when you want to run tests in parallel? For example you have a couple thousand tests and it's taking a while so opting to run in parallel.
I would question why you have a couple of integration thousand tests in a single solution. For integration tests to run in parallel the data needs to be isolated and this is usually done with separate databases. I can't even begin to imagine the cost of spinning up 2k databases in docker to run your tests in parallel. This will be way more efficient and faster
Thanks for the reply Nick.
@@martinshields7122 I have a project with around 700 integration tests using Respawn and it only takes a few seconds to run through them all using a MSSQL database on my localhost (test not run in parallel). I'm sure this will save you time.
@@reikooters thank you
@@nickchapsas I found creation and deletion of databases (not servers) fast after each test run in parallel. Maybe db pool with respawn would be a good idea. Most test runners run couple of dozens test in parallel only (though I am not sure), so maximum of dbs running in parallel would be the same.
I like integration tests as they if done well (BDD style) can allow me to refactor internal code however I want without worrying about fixing tests (much if at all).
I would run them with in memory db, and for each test a different instance. Probably runs in 50ms or less :) parallel is also no issue.
I know people argue this isn’t real enough but imo database adapters and software are tested elsewhere. To be honest; your Docker container is also not the actual “real” database that will be used :)
I would probably do in memory locally and ci until you’re happy with your code and want to merge it or deploy it, then do the final check with the actual target db.
In the meantime you save a lot of time and energy (literally).
Tests are imo not about absolute safety, but about minimizing risk as best as possible at a reasonable cost. Developers waiting seconds, minutes, hours, is imo not worth it.
Running against an inmemory db defauts the purpose completely. It's the most common mistake that people do when it comes to integration testing. Docker, assuming you match the version of the database in production, absolutely is the real database. The moment you remove the real integration point to your test, your test becomes a functional test, which isn't 100% useless, but its value drops significantly.
@@nickchapsas The point is that you don't need to create two separate tests, you can create one, but run it fast/medium secure while developing, and slow/full secure just before the actual integration or deployment.
@@patrickroza3073 I'm very much against that approach. Your "fast" test is useless if it only produces reliable results when it's run against the real database. The first level of execution adds nothing to the table but the illusion of coverage. The actual integration point later is the only thing that matters in your example
@@nickchapsas I think Jimmy is on point in the conclusion of his blog post about in memory db testing; perhaps I’m lucky the applications are trivial enough :)
(Im mostly using document dbs, no EF, not even .NET, but have in the past)
Hi. I recently make use of testcontainers. It works like a charm, UNTIL I decided to use redis. There will be an error of I use redis container: No conection is active/available to service this operation:
Can you help me?
What about reseeding? Respawn wipes it clean, I want to keep my initial seeding for all test runs.
Hey Nick, great video! I don't reset my integ test database at all. What I do instead is I generate a unique test key (an integer) for each test. I do that using a special "TestExecution" table with an identity column. I insert a record into this table at the beginning of each test, then get the value of the identity column from the inserted record. Once I have a test key, I use it to construct names of my domain objects. For example, all tests that create a Customer object would use $"Test Customer {_testKey}" as the name. This avoids data collisions between tests and eliminates the need to reset the database after each test.
@nickchapsas - curious what you think about this approach.
Can't we use different databases in the same database container?
Ive seen Jason Taylor using a SliceFixture with using static SliceFixture (in his clean testing repo). Could you do something similar or with a base class to avoid having to copy paste the IAsyncLifetime setup for each test class you create?
Also interested to hear where you migrate data. I tend to avoid doing it in api startup because it can mess up things if you deploy to production. Would you get a DbContext instance in initializeasync through a scope and run a migration there?
Yeah you can totally create a DatabaseFixture and extend from that. You can run your migrations in the InitializeAsync method of the CustomerApiFactory class to only do it once per the whole app outside of application startup
@@nickchapsas Yea I'm trying that, but I can't figure out how I would get the IServiceScopeFactory to resolve the DbContext during InitializeAsync in the WebApplicationFactory.
I guess you just manually new up one inside a using statement? But then there's the problem with passing the correct connectionstring.
Tanx
I don't get it. If tests run in parallel, they still might clash with each other because resetting database can reset data for other test executing in parallel.
Great library, but I dislike the fact to reseed using SQL Scripts. In my projects, I reset and reseed the database using an Action with EF into my databasefixture combined with EnsureDeleted and EnsureCreated. The drawback is that I have to use a scaffolding of the database using EF only for the tests (because we use dapper). it could be great, if that library could seed the database using Models or Fluent methods.
I think you missed the point of what Respawn does. It doesn't seed your DB. It clears data from tables by doing smarter SQL delete operations that take into account FK dependencies. This is way faster than truncating or dropping and recreating tables with whatever tool (EF, raw SQL, etc.). You can still use EF migrations to create the database and seed data, and then configure Respawn to include/exclude tables that it should reset when you run it.
That way you don't lose data in look up tables which may have been seeded via EF, for example.
What Thiago said
@@tafs7 Thanks for the feedback, i didn't see that point of view. I will try this approach! As usual, great video Nick!
What if you have tens of thousands of records and you try to delete one by one based on their ID. If you have a relational DB like Postgresql or MS SQL, your transaction log will be flooded with delete instructions making it grow. If you need to delete entire table probably a Truncate will serve you better. In case you don’t want to delete an entire table it would be better to delete full sets at once in a single statement.
"But don't worry, people like this library."
They don't like MediatR? Also, what's the issue with Automapper?
There are plenty of people compaining about both libraries. Either google it or check the comments in my videos when I talk about those libraries.
I don't get why people complain about MediatR or Automapper@@nickchapsas ...
Jimmy Bogard has been a big contributor to the .NET community, and he has always provided enough info & examples to usecases. Also when NOT to use his libraries (especially when it comes to automapper for example).
@@psyaviah It's not a matter of hate towards Jimmy Bogard who's one of the biggest brain in the .NET community.
It's more that some people tend to think that his products are bad from a (extremist) design point of view.
Like AutoMapper has to much magiic in it, and MediatR is supposedly bad as a CQRS tool.
What Nick got wrong is that, despite him apperently being one of those detractors, those two libs are vastly liked and used, cumulating 400 Million downloads...
I would argue though that a lot of people are wrongly using the libraries. So when you arrive in a shitty project that misuse them, you could think that it's the lib's fault, which is not.
Jimmy Bogard himself talked a lot about this issue in his blog, and made some radical changes to AutoMapper specifically to forbid what he wrongly allowed in the early years.
TL;DR; Most people like them, some hate them, but they're very good libraries when used as intended.
I think AutoMapper brings more problems than it solves, but maybe it's nice to use in contexts that I have not come across. In general, I'd rather have normal mapping classes that make it easily to check out where setters are called rather than having to deal with a possible mix of implicit or explicit magic mappings that are also harder to set up and debug.
@@zabustifu that's tha opinion. In a clan architecture environment, it never bothered me because I never seek setters that do mappings.
And to find object dependencies, you look for the object.
And the biggest advantage for me to modern auto Mapper is that it's usuable in Linq To Sql to create the select expression.
This is dope af (and yes pretty performant for web usage)
If all the tests are running in parallel, wouldn't it still have conflicts as you're resetting only after each test class completed it tests. so if CreateCustomerControllerTests and DeleteCustomerControllerTests runs in parallel, the state of the database from DeleteCustomerControllerTests perspective can contains data from CreateCustomerControllerTests.
When the classes are on the same "collection" they don't run in parallel. For more information you can read the xunit documentation (Running Tests in Parallel).
Put this together with the default behavior of xunit where tests of a class don't run in parallel and nothing is running in parallel now.
The bad think about Respawn is that reseting to clean state what means no data at all
what can be problematic, if your DB migrations create some data (like "system user")
or when your migrations takes ages (in my project it takes ~1min * amount of test = hudge amount of time)
the other aproch is to recreate DB from backup (or clone if your database support it)
but in very similar way as Nick did with Respawn
You can reset to a state that already has data as long as you exclude the table (for example for join tables). In my case I am resetting to a database state AFTER i have run my migrations.
@@nickchapsas It looks like I use it incorrectly. My bad
@@nickchapsas I believe @KondzioSSJ4 is right. Respawn deletes every row.
@@BillyBraga Updated my comment to reflect the behavior. What I meant was that you can still have data in your database and reset to that version with data, as long as they are in excluded schemas or tables. It doesn't make sense to need data for your "clean state" in non-join tables.
@@nickchapsas For EF Migrations I would suggest to at least add
TablesToIgnore = new Table[]
{"__EFMigrationsHistory"},
to Respawner.CreateAsync(...)
Hey! I tested db snapshot restore after my big big initial migration is applied of ~10 year project, and it execute reset very fast. But could Respawn work with HiLo id generator or I need to disposs ef instance after each test? because currently seems it is not support reset HiLo on EF cache level.
Nice
Why not use database snapshots? (for the reset)
You can, but if you are after a cross-db solution that is very fast, then Respawn is better
why you dont use visual studio ?
Because I like Rider more
Nice way to write tests but isnt it a disadvantage that you cant use the same test against a real database after a deploy? When you manually delete things after each test, the same set of tests can be reused
Is there something like respawn for Mongo?
Not that I'm aware of
Nick Chapsas must be one of the most beloved God's netrunner , a real demon at work . We love your content which is super profesional and outstanding . Blessings !
Why people hate on MediatR? I've never seen such comments :o
There are plenty of people complaining about it. Just google "Is MediatR bad"
So… do you think FastEndpoints package is a good „alternative”? It’s not quite the same, but fixes the same / similar thing MediatR was made for. I think both work well when used wisely.
@@Tosde96 I like both MediatR & FastEndpoints. We use both actually, just depends on the scope of the api or type of application you are creating imho.
Hi, could you please share your demo codes?
The source code is available to my patreons
Hey Nick, why you don't use the transaction mechanism? It should cover all problems that you mentioned in your video.
Does your system work with false transactions in production? Using transaction scopes and rollbacks doesn't reflect how your app will actually perform and ultimately fail in production when you deploy it. It is also limiting depending on the ORM tech you are using and also can't be used if your operations span multiple transactions. It's a naive approach.
@@nickchapsas Thank you for your anserw. Good to know that are some extra limitations with "my" setup.
Could you make some video showing how test suite execution time scales using this and alternative solutions?
The video covers the main two approaches and you can see how the tests got faster. This is faster than dacpac, snapshot load or transaction rollback (and more flexible).
Why don't we just run a test in rollback scope?
Is that how your system integrates with your database in production?
@@nickchapsas No, but why is it important? (I ask seriously)
@@kaczaczacza Because you want your integration tests to catch problems your real system will encounter before it actually encounters them. The more you move the variables between how your real integration points work compared to your test ones, the more likely it is you will have false positives or false negatives in your tests
@@nickchapsas Ok, so what you are saying is: rollback may look like the same solution but it can bring with it some behaviour we cannot predict? And that's why it is better to simulate production as much as possible?
@@kaczaczacza Because the whole purpose of testing is to catch issues before they happen in production where they will cost you money and time to fix.
Can you compare Testcontainer and Ductus.FluentDocker?
Please like if you wanna send Jimmy some love for his hard work!
You've talked about an EF Core unit testing video before. Is it still in your backlog or cancelled?
Why not to use just InMemory EF Core database for tests?
Because integration tests using in memory providers (either EF or SQLite) are 100% useless integration tests. They aren't integration tests all together. They are functinal tests since you aren't using your integration point.
@@nickchapsas What is your definition of an entire "integration test"? (I'm sure you have videos or courses for that :p).
We define Integration tests (for an web-api microservice) as: a test that covers code in; controller endpoints, business logic (services, helpers etc) and database fetchers).
We use EF core to both SQLServer and PostGreSql, so we essentially want to test our data-fetching layers linq-queries, and leave the translation to the specific database query-languages to EF-core.
So in our case we happily use InMemory databases in our integration tests, and to separate tests from each, we create the InMemoryDB with a Guid.NewGuid name.
InMemory EF Core is the worst You can bet on. As @Nick already said. Slightly better is to use even Sqlite with :memory database. But the best is real db 🙂
Why do people hate automapper? lol
Because it's slow. I use Mapster which is 4x times faster and can be 10x times faster when using source generators. And I like the Mapster api way more
I don’t think auto mapper has much of a use case. Whenever you change the type of a property you have to be aware of any type incompatibilities you might create. Auto mapper will throw a runtime error when it encounters these.
So if you want to avoid these errors then as a dev you have to spend time worrying about mapping logic anyway and still worrying about any run time errors you might miss.
Much better just to write a mapping function, errors appear at compile time, no real loss of dev time (regular expressions exist) and performance is better.
Incredible
BWAHAHA! "Made by the same person who made AutoMapper and MediatR, but don't worry, people like THIS library." BURN! Poor Jimmy - he gets so much crap for his OSS stuff, the man's gotta have some thick skin.
Found it -> Patreon ... :-)
This does not work when executing tests in parallel. That starts to hurt when you have alot of tests.
Ideally you want a db per test for full isolation but it's expensive to create a new db everytime. You could use pooling to prevent having to recreate the whole db everytime and reset the db before returning it to the pool.
I think that's more scalable than what's shown here but obviously more complex and not needed everywhere.
I feel this pain daily
Who the hell hates AutoMapper
Just google "why i hate automapper" and see your results light up like Christmas
I would kill to know what he does. I would e a multi. Millionaire
Can I pay you for an hour of 1to1 advice nick?
This is what I do:
How bad is this? It works, but is very slow... Will be switching to this method you showed
protected override IHost CreateHost(IHostBuilder builder)
{
builder.UseEnvironment("Testing");
var host = builder.Build();
var serviceProvider = host.Services;
using (var scope = serviceProvider.CreateScope())
{
var scopedServices = scope.ServiceProvider;
var appDbContext = scope.ServiceProvider.GetService()!;
appDbContext.Database.EnsureDeleted();
appDbContext.Database.Migrate();
var userManager = scopedServices.GetRequiredService();
var roleManager = scopedServices.GetRequiredService();
Seeder.SeedUsersAsync(userManager, roleManager).GetAwaiter().GetResult();
var seeder = new Seeder(appDbContext);
seeder.SeedInstancesAsync().GetAwaiter().GetResult();
}
host.StartAsync();
return host;
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder
.UseSolutionRelativeContentRoot("Solution/Project");
}