If you write Rails code, or any Ruby code that interacts with a database, and you also write automated tests, chances are you have heard of or used the database_cleaner gem. It’s a terrific gem that abstracts away the various ORM APIs for getting the DB into a “blank slate” state.
Periodically I start a new project using Rails, with RSpec, Capybara, and Selenium for acceptance testing, and a short way into it I find myself banging my head against bizarre inconsistencies with the test database. I’ll set up some records in the test DB, only to have the Selenium-driven browser-based tests act like those records never existed. Eventually, I’ll realize what I did wrong and curse my feeble brain for not remembering the last time I solved the same problem.
The problem is always the same: the tests are being wrapped in database transactions, so any code running outside the actual test process (like, say, a server process servicing a Selenium-driven browser request) does not see the database fixture I’ve so carefully assembled.
I just walked a pairing client through the same fix, so in the interests of remembering the steps, and hopefully preventing some other folks from tearing their hair out, here are the steps needed.
First of all, and this is very important, go into spec/spec_helper.rb
and change this line:
config.use_transactional_fixtures = true
To:
config.use_transactional_fixtures = false
This will disable rspec-rails’ implicit wrapping of tests in a database transaction. Without disabling this, none of the following configuration will matter.
Now configure database_cleaner
. I usually create a separate file called spec/support/database_cleaner.rb
for this. Inside, I put something like this:
RSpec.configure do |config| config.before(:suite) do DatabaseCleaner.clean_with(:truncation) end config.before(:each) do DatabaseCleaner.strategy = :transaction end config.before(:each, :js => true) do DatabaseCleaner.strategy = :truncation end config.before(:each) do DatabaseCleaner.start end config.after(:each) do DatabaseCleaner.clean end end
Let’s take that step by step.
config.before(:suite) do DatabaseCleaner.clean_with(:truncation) end
This says that before the entire test suite runs, clear the test database out completely. This gets rid of any garbage left over from interrupted or poorly-written tests—a common source of surprising test behavior.
config.before(:each) do DatabaseCleaner.strategy = :transaction end
This part sets the default database cleaning strategy to be transactions. Transactions are very fast, and for all the tests where they do work—that is, any test where the entire test runs in the RSpec process—they are preferable.
config.before(:each, :js => true) do DatabaseCleaner.strategy = :truncation end
This line only runs before examples which have been flagged :js => true
. By default, these are the only tests for which Capybara fires up a test server process and drives an actual browser window via the Selenium backend. For these types of tests, transactions won’t work, so this code overrides the setting and chooses the “truncation” strategy instead.
config.before(:each) do DatabaseCleaner.start end config.after(:each) do DatabaseCleaner.clean end
These lines hook up database_cleaner
around the beginning and end of each test, telling it to execute whatever cleanup strategy we selected beforehand.
And that’s it!
Note that this is all for RSpec, and does not cover Cucumber configuration.
Hopefully this will help someone else out there avoid the frustrations I’ve run into!
EDIT: A few people have asked me why I don’t just force all threads to share the same ActiveRecord connection, as demonstrated in this Gist. A few reasons:
- Using
database_cleaner
implies that I want ORM neutrality.database_cleaner
supports ActiveRecord, DataMapper, MongoMapper, and others. The solution linked above only works for ActiveRecord. - It’s a monkey-patched kludge which will only work so long as AR refrains from changing its connection-sharing internals. And frankly I’m not sure I trust it to work across all Ruby VM and database combinations (UPDATE: And indeed, I’ve now seen two different people say there are race conditions with the current Postgres adapter). I’d be more inclined to use it if ActiveRecord had a published configuration option which was known to work in all contexts.
- As I stressed above, I’m careful to set things up so that only the tests that need them fall back to truncation. Since
:js => true
tests generally don’t form the bulk of my suite (and since they are unavoidably slow anyway, due to the overhead of driving a browser), I’m not overly concerned about the added overhead. Perhaps if all of my acceptance tests drove a live browser I’d be more worried about it. - In cases where database truncation is taking up a significant amount of test time, you can usually speed things up with some judicious control of which subset of tables get truncated for a given test. This is something
database_cleaner
makes pretty easy. Maybe that would make a good topic for a followup post. - UPDATE: Oh yeah, and as @donaldball points out, sharing a transaction between test and test server means acceptance tests don’t run quite the same as they would in production. Specifically, they’ll never trigger
after_commit
hooks.
Why didn’t you just share activerecord connection between threads – https://github.com/jnicklas/capybara#transactions-and-database-setup ?
I’ve added a section at the end addressing this.
I’ve screwed this up a few times – thanks for writing the definitive guide!
You’re welcome!
This is one of those things I take for granted, but get bitten by on occasion, so nice work identifying it and writing it up.
🙂
Nice consolidation! 🙂
does not work with multiple database connection with activerecord
Thanks! This is really useful. I was doing something similar, but I believe your approach works better and is a little bit clearer to use!
Hi, Thanks for the post. I’ve been struggling with some database issues in my tests, and this seems to help, but my specs no take 4 x’s longer. It is a pretty large test suite, but 20 minutes seems prohibitive. Got any advice?
Do you have a many short capybara tests?
Check also http://gist.github.com/moonfly/4950750.
I really like the code in this post (actually more than the one in the gist), but it can be optimized slightly further if you need that extra bit of performance.
Also worth adding that it’s important, that in the config you put the before(:each, :js => true) block above the DatabaseCleaner.start block (as it is in the example) – I just spent a while figuring out why my capybara tests were hanging, and this was the problem (clearly setting the strategy after starting the cleaner doesn’t work so well…!). Thanks a bundle for the post though, extremely helpful.
How did you worked it around?
I just had it the wrong way around – if you put the before(:each) { DatabaseCleaner.start} block above the before(:each, :js=>true) block, it starts the database cleaner before applying the truncation strategy. Which is bad. If you follow the configuration in the post above you shouldn’t have any problems.
Very useful! Thanks
I decided I wasn’t entirely happy with relying on the :js tag to determine the strategy, so I modified the code to use Capybara.current_driver to determine if :rack_test was being used. This permits a single before(:each) block instead of the three listed above. See https://gist.github.com/tovodeverett/5817365 for a synopsis.
One note – both Avdi’s approach and my approach can fall victim to an issue with RSpec < 2.14 where before(:each) blocks activated by an include get called before those activated from config.before. See https://github.com/rspec/rspec-core/issues/903 for one of several issue reports that ended up all being resolved by https://github.com/rspec/rspec-core/pull/845.
If your using FactoryGirl sequences you’ll need to reset them when you use the truncation strategy.
config.before(:each, :js => true) do
DatabaseCleaner.strategy = :truncation
FactoryGirl.reload
end
OMGGGGG thank you for this!!!
If you’re running into stupid deadlock or other transaction errors because after hooks with databasecleaner are being run before the server finishes whatever it’s doing, try this! https://gist.github.com/timonv/7026067
The trick is to force the driver to wait till the page finishes loading.
thanks
You could dry up your code a bit using an around filter:
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.around(:each) do |example|
DatabaseCleaner.strategy = example.metadata[:js] ? :truncation : :transaction
DatabaseCleaner.start
example.run
DatabaseCleaner.clean
end
Thanks for this great post! We started to get a lot of random deadlock errors on a few different projects we have, so I wanted to update you a slight change that I think will help people avoid having this issue.
We ended up changing your:
config.after(:each) do DatabaseCleaner.clean end
to:
config.append_after(:each) do DatabaseCleaner.clean end
This is because, as it was, DatabaseCleaner.clean was being run before Capyara was resetting the driver, meaning that you could have situations where there’s an active connection while you’re trying to truncate the tables. By changing it to an append, it means that the Capybara.reset! that is added in capybara/rspec is called before your DatabaseCleaner.clean call.
Capybara 2.2 fixed an issue where calling Capybara.reset! wouldn’t wait for the driver to reset before continuing, which could have been a cause for the deadlock. So, combine Capybara 2.2 with the above change and you should be golden.
Thanks for the note!
You just saved me. Thanks
thanks!!!
Great post. Thank you!
I pretty much ignored this post when it came out. Now I’m splitting an extant app apart into API and UI halves, and trying to test the UI half. This is exactly what I need, so that the API server will see the items created by the UI-side tests! Thanks once again Avdi!
As far as I can tell, this methodology causes rspec reload the fixtures before each test, even when the database cleaner is using the transaction strategy. Needless to say, this is really bad for test performance. Is this expected? Can you recommend a way to load the fixtures only once (or, I suppose, have them only get reloaded after tests with the truncation strategy)? Thanks.
I don’t use fixtures, so I don’t know. Sorry!
No problem. Thanks for the note. There are some ideas here http://pivotallabs.com/fixture-builder-and-rspec-acceptance/ but this approach is just generating an exception for me.
Great post Avdii! This was a current problem I was having (had some Postgres race conditions in my continuous integration build) due to the ActiveRecord connection sharing, and using database_cleaner this way, together with the tip of appending the each hook for cleaning the db, worked like a charm.
Hi Avdi,
Great post, I am also a great fan of Ruby Rouges, so thank you for all that. I have two questions:
If I have a feature with several scenarios, if I want to create some records once (because I don’t manipulation the actual records, but test some things like that there are exactly 5 records per page, there is pagination, pagination works etc.), I can’t use rspec’s before(:all) hook, as config.after(:each) { DatabaseCleaner.clean } will detroy these records. Is there a way to tell database_cleaner gem to not clean the db just for a certain scenario?
Cheers,
Alex
I’ve referred back to this post a lot, but recently discovered that I was using truncation globally, even though I thought I was only doing it on my feature specs. Just tossed up this gist of how I fixed the situation:
https://gist.github.com/mockdeep/9904695
Rspec runs
before
hooks in the order they are defined, andafter
hooks in reverse order, so I opted to explicitly useprepend_before
andappend_after
to avoid confusion. Also, I added a couple of methods to help debug database cleaner. Unfortunately, they don’t have a neat API that I’ve found for finding out what strategy is active.Another side note, I use
:type => :feature
instead of:js => true
since we don’t always need or remember to tag our specs as using javascript.great post. thank you
Hey there!
Great post and very useful for setting this up with rspec 🙂
Got another problem in setting this up with “native” rails testing (minitest)
When I use the ActiveRecord Connection Gist I get this weird error:
ActiveRecord::StatementInvalid: Mysql2::Error: This connection is in use by: #
And for the database_cleaner attempt, I turn off transactional fixtures
class ActionDispatch::IntegrationTest self.use_transactional_fixtures = true end
In my Feature file I then use:
class Feature < ActionDispatch::IntegrationTest def setup DatabaseCleaner.strategy = :truncation DatabaseCleaner.start end def teardown DatabaseCleaner.clean end
But this isn’t working, Fixtures / Factories created in a before block for the feature test aren’t available for capybara / webkit.
You got any hint how to solve this?
Cheers benny
Wow Avdi, thank you so much for this–been struggling with this exact issue for 3 hours. Fixed now.
Beautiful! Thanks!
Thank you, thank you, thank you!!!!!
@avdi:disqus thank you for this material of crucial importance!
I want to ask Capybara’s maintainer to add a link to this post in the gem’s README, if this is ok for you!
Do you plan to update your post’s code with the information supplied by @casetaintor:disqus ?
Fine by me…
Sometimes you don’t want to clean the db after each example, but after the whole example group. This can be done in rspec 3 using the metadata filters.
#rails_helper.rb config.around(:example) do |example| unless example.metadata[:noclean] DatabaseCleaner.strategy = example.metadata[:js] ? :truncation : :transaction DatabaseCleaner.cleaning do example.run end end end <pre><code> config.before(:context, :noclean => true) do |example_group| DatabaseCleaner.strategy = example_group.class.metadata[:js] ? :truncation : :transaction DatabaseCleaner.start end config.after(:context, :noclean => true) do DatabaseCleaner.clean end #some_spec.rb require "rails_helper" describe "database_cleaner", :noclean do describe "one feature" do context "when working" do before do FactoryGirl.create :foo end it "should have one foo" do expect(Foo.count).to eq 1 end end end describe "another feature" do context "when still working" do it "still have on foo" do expect(Foo.count).to eq 1 end end end end
I was struggling with this post for a while. In the end I needed an additional non_transactional filter for some model specs where I use some transaction breaking feature (load infile etc..). So, just sharing the config I ended up with:
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do |example|
if example.metadata.slice(:non_transactional, :js).values.any?
DatabaseCleaner.strategy = :truncation
else
DatabaseCleaner.strategy = :transaction
end
DatabaseCleaner.start
end
config.append_after(:each) do
DatabaseCleaner.clean
end
Thanks all for the help in figuring this out.
(can’t edit so follow up post)
OK, found some relevant nuances:
Using a truncation strategy after your test, doesn’t ensure the record IDs are reset prior to the test. Sometimes you need this for special tables (arguably code smell…). Anyway, new flag: reset_auto_increments. Only reset the couple of tables which have ID related logic to avoid an overall reset. Also this is completely unrelated to the cleaning strategy afterwards.
By default truncation is not smart. Of you don’t care about the IDs being reset, use the pre_count and reset_ids options. Or go with deletion instead.
Here is my new config:
config.before(:suite) do
ensure_databases_are_innodb
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do |example|
if example.metadata.slice(:reset_auto_increments).values.any?
%w(migration_batches appointment_id_seeds).each do |table_name|
ActiveRecord::Base.connection.execute(“truncate table #{table_name}”)
end
end
if example.metadata.slice(:non_transactional, :js).values.any?
#DatabaseCleaner.strategy = :deletion
DatabaseCleaner.strategy = :truncation, {pre_count: true, reset_ids: false}
else
DatabaseCleaner.strategy = :transaction
end
DatabaseCleaner.start
end
config.append_after(:each) do
DatabaseCleaner.clean
end
Thanks! \m/ 😀
Yes! Thanks for this posting. I was stumped for 1/2 a day on this at one point.
I have written a guide on configuring Rails 4.1 for Testing with Test::Unit, Minitest, and Rspec. It also reviews the uses of mocking, stubbing, Page Object pattern, and debugging hints. You can find it on my Ruby Reflections blog: http://www.ironhorserails.com/blog
When you create spec/support/database_cleaner.rb, do you “require ‘database_cleaner'” from spec_helper.rb? Is there anything else to do to get it to find your database_cleaner.rb? I can’t seem to get it to find database_cleaner.rb.
If I copy/paste the contents of your database_cleaner.rb into my spec_helper.rb’s config block directly, it works beautifully.
But if I just put it in spec/support/database_cleaner.rb it never clears the data. I kinda thought rspec would automatically load files in spec/support but it doesn’t seem to. I tried “require ‘database_cleaner'” from spec_helper.rb and that didn’t help. Probably something little I’m missing.
(Ruby 2.1.5, rails 4.1.8, rspec-rails 3.1.0)
@gayle – by default, files in spec/support directories aren’t loaded by rspec any longer, so if you uncomment the line in spec helper, it should be ok
Dir[Rails.root.join(“spec/support/**/*.rb”)].each { |f| require f }
Again, this post was really helpful, but when I ran my feature specs along with the rest of the suite, it slowed the rest of my specs to a grinding hault. Any idea why adding this may cause the rest of the specs to take so long that I’ve had to stop it after 20 minutes when normally it takes 2?
Hey Avdi my favorite rails author and and speaker thanks for the lovely books and yoir deep thoughts on elixir (Im trying it)
I use Mongoid mostly on rails can you also show us a mongoid version , I think its just using truncation instead o transaction but thats me 🙂
Wow, thanks for this. I was using the truncation strategy for both :suite and :each and my tests were taking up to an hour to run. After switching to the transaction strategy for :each we’re down to about 10 minutes.
One question I had, though:
Why not just use the transaction support built right into Rspec?
Many Thanks,
Josh
Here’s a battle-tested
spec/support/database_cleaner.rb
with bad config detection and explaining variables to try to make the config self-documenting and clearer to newer developers: https://github.com/eliotsykes/rspec-rails-examples/blob/master/spec/support/database_cleaner.rb (contributions welcome!)