Configuring database_cleaner with Rails, RSpec, Capybara, and Selenium

If you write Rails code, or any Ruby code that interacts with a database, and you also write automated tests, chances are you have heard of or used the database_cleaner gem. It’s a terrific gem that abstracts away the various ORM APIs for getting the DB into a “blank slate” state.

Periodically I start a new project using Rails, with RSpec, Capybara, and Selenium for acceptance testing, and a short way into it I find myself banging my head against bizarre inconsistencies with the test database. I’ll set up some records in the test DB, only to have the Selenium-driven browser-based tests act like those records never existed. Eventually, I’ll realize what I did wrong and curse my feeble brain for not remembering the last time I solved the same problem.

The problem is always the same: the tests are being wrapped in database transactions, so any code running outside the actual test process (like, say, a server process servicing a Selenium-driven browser request) does not see the database fixture I’ve so carefully assembled.

I just walked a pairing client through the same fix, so in the interests of remembering the steps, and hopefully preventing some other folks from tearing their hair out, here are the steps needed.

First of all, and this is very important, go into spec/spec_helper.rb and change this line:

config.use_transactional_fixtures = true

To:

config.use_transactional_fixtures = false

This will disable rspec-rails’ implicit wrapping of tests in a database transaction. Without disabling this, none of the following configuration will matter.

Now configure database_cleaner. I usually create a separate file called spec/support/database_cleaner.rb for this. Inside, I put something like this:

RSpec.configure do |config|

  config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
  end

  config.before(:each) do
    DatabaseCleaner.strategy = :transaction
  end

  config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.after(:each) do
    DatabaseCleaner.clean
  end

end

Let’s take that step by step.

config.before(:suite) do
  DatabaseCleaner.clean_with(:truncation)
end

This says that before the entire test suite runs, clear the test database out completely. This gets rid of any garbage left over from interrupted or poorly-written tests—a common source of surprising test behavior.

config.before(:each) do
  DatabaseCleaner.strategy = :transaction
end

This part sets the default database cleaning strategy to be transactions. Transactions are very fast, and for all the tests where they do work—that is, any test where the entire test runs in the RSpec process—they are preferable.

config.before(:each, :js => true) do
  DatabaseCleaner.strategy = :truncation
end

This line only runs before examples which have been flagged :js => true. By default, these are the only tests for which Capybara fires up a test server process and drives an actual browser window via the Selenium backend. For these types of tests, transactions won’t work, so this code overrides the setting and chooses the “truncation” strategy instead.

config.before(:each) do
  DatabaseCleaner.start
end

config.after(:each) do
  DatabaseCleaner.clean
end

These lines hook up database_cleaner around the beginning and end of each test, telling it to execute whatever cleanup strategy we selected beforehand.

And that’s it!

Note that this is all for RSpec, and does not cover Cucumber configuration.

Hopefully this will help someone else out there avoid the frustrations I’ve run into!

EDIT: A few people have asked me why I don’t just force all threads to share the same ActiveRecord connection, as demonstrated in this Gist. A few reasons:

  • Using database_cleaner implies that I want ORM neutrality. database_cleaner supports ActiveRecord, DataMapper, MongoMapper, and others. The solution linked above only works for ActiveRecord.
  • It’s a monkey-patched kludge which will only work so long as AR refrains from changing its connection-sharing internals. And frankly I’m not sure I trust it to work across all Ruby VM and database combinations (UPDATE: And indeed, I’ve now seen two different people say there are race conditions with the current Postgres adapter). I’d be more inclined to use it if ActiveRecord had a published configuration option which was known to work in all contexts.
  • As I stressed above, I’m careful to set things up so that only the tests that need them fall back to truncation. Since :js => true tests generally don’t form the bulk of my suite (and since they are unavoidably slow anyway, due to the overhead of driving a browser), I’m not overly concerned about the added overhead. Perhaps if all of my acceptance tests drove a live browser I’d be more worried about it.
  • In cases where database truncation is taking up a significant amount of test time, you can usually speed things up with some judicious control of which subset of tables get truncated for a given test. This is something database_cleaner makes pretty easy. Maybe that would make a good topic for a followup post.
  • UPDATE: Oh yeah, and as @donaldball points out, sharing a transaction between test and test server means acceptance tests don’t run quite the same as they would in production. Specifically, they’ll never trigger after_commit hooks.

55 comments

  1. Hi, Thanks for the post. I’ve been struggling with some database issues in my tests, and this seems to help, but my specs no take 4 x’s longer. It is a pretty large test suite, but 20 minutes seems prohibitive. Got any advice?

  2. Also worth adding that it’s important, that in the config you put the before(:each, :js => true) block above the DatabaseCleaner.start block (as it is in the example) – I just spent a while figuring out why my capybara tests were hanging, and this was the problem (clearly setting the strategy after starting the cleaner doesn’t work so well…!). Thanks a bundle for the post though, extremely helpful.

      1. I just had it the wrong way around – if you put the before(:each) { DatabaseCleaner.start} block above the before(:each, :js=>true) block, it starts the database cleaner before applying the truncation strategy. Which is bad. If you follow the configuration in the post above you shouldn’t have any problems.

  3. I decided I wasn’t entirely happy with relying on the :js tag to determine the strategy, so I modified the code to use Capybara.current_driver to determine if :rack_test was being used. This permits a single before(:each) block instead of the three listed above. See https://gist.github.com/tovodeverett/5817365 for a synopsis.

    One note – both Avdi’s approach and my approach can fall victim to an issue with RSpec < 2.14 where before(:each) blocks activated by an include get called before those activated from config.before. See https://github.com/rspec/rspec-core/issues/903 for one of several issue reports that ended up all being resolved by https://github.com/rspec/rspec-core/pull/845.

  4. If your using FactoryGirl sequences you’ll need to reset them when you use the truncation strategy.

    config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation
    FactoryGirl.reload
    end

  5. You could dry up your code a bit using an around filter:

    config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
    end

    config.around(:each) do |example|
    DatabaseCleaner.strategy = example.metadata[:js] ? :truncation : :transaction
    DatabaseCleaner.start
    example.run
    DatabaseCleaner.clean
    end

  6. Thanks for this great post! We started to get a lot of random deadlock errors on a few different projects we have, so I wanted to update you a slight change that I think will help people avoid having this issue.

    We ended up changing your:
    config.after(:each) do DatabaseCleaner.clean end
    to:
    config.append_after(:each) do DatabaseCleaner.clean end

    This is because, as it was, DatabaseCleaner.clean was being run before Capyara was resetting the driver, meaning that you could have situations where there’s an active connection while you’re trying to truncate the tables. By changing it to an append, it means that the Capybara.reset! that is added in capybara/rspec is called before your DatabaseCleaner.clean call.

    Capybara 2.2 fixed an issue where calling Capybara.reset! wouldn’t wait for the driver to reset before continuing, which could have been a cause for the deadlock. So, combine Capybara 2.2 with the above change and you should be golden.

  7. I pretty much ignored this post when it came out. Now I’m splitting an extant app apart into API and UI halves, and trying to test the UI half. This is exactly what I need, so that the API server will see the items created by the UI-side tests! Thanks once again Avdi!

  8. As far as I can tell, this methodology causes rspec reload the fixtures before each test, even when the database cleaner is using the transaction strategy. Needless to say, this is really bad for test performance. Is this expected? Can you recommend a way to load the fixtures only once (or, I suppose, have them only get reloaded after tests with the truncation strategy)? Thanks.

  9. Great post Avdii! This was a current problem I was having (had some Postgres race conditions in my continuous integration build) due to the ActiveRecord connection sharing, and using database_cleaner this way, together with the tip of appending the each hook for cleaning the db, worked like a charm.

  10. Hi Avdi,

    Great post, I am also a great fan of Ruby Rouges, so thank you for all that. I have two questions:

    1. Instead of setting config.use_transactional_fixtures from true to false, couldn’t I just delete the line config.use_transactional_fixtures = true altogether. In other words – what is the default value? If the default value is true, I wonder why rspec creates this line by default at all.
    2. If I have a feature with several scenarios, if I want to create some records once (because I don’t manipulation the actual records, but test some things like that there are exactly 5 records per page, there is pagination, pagination works etc.), I can’t use rspec’s before(:all) hook, as config.after(:each) { DatabaseCleaner.clean } will detroy these records. Is there a way to tell database_cleaner gem to not clean the db just for a certain scenario?

    Cheers,
    Alex

      1. I always prefer explicit configuration, especially when I’m doing something that may go against people’s expectations (regardless of what the RSpec default is).
      2. Probably, but I don’t know offhand. Sorry.
  11. I’ve referred back to this post a lot, but recently discovered that I was using truncation globally, even though I thought I was only doing it on my feature specs. Just tossed up this gist of how I fixed the situation:

    https://gist.github.com/mockdeep/9904695

    Rspec runs before hooks in the order they are defined, and after hooks in reverse order, so I opted to explicitly use prepend_before and append_after to avoid confusion. Also, I added a couple of methods to help debug database cleaner. Unfortunately, they don’t have a neat API that I’ve found for finding out what strategy is active.

    1. Another side note, I use :type => :feature instead of :js => true since we don’t always need or remember to tag our specs as using javascript.

    2. Unfortunately, they don’t have a neat API that I’ve found for finding out what strategy is active.
      It’s because database_cleaner can handle multiple connections at the same time with different strategies.

  12. Hey there!

    Great post and very useful for setting this up with rspec 🙂

    Got another problem in setting this up with “native” rails testing (minitest)

    When I use the ActiveRecord Connection Gist I get this weird error:
    ActiveRecord::StatementInvalid: Mysql2::Error: This connection is in use by: #

    And for the database_cleaner attempt, I turn off transactional fixtures

    class ActionDispatch::IntegrationTest self.use_transactional_fixtures = true end

    In my Feature file I then use:

    class Feature < ActionDispatch::IntegrationTest def setup DatabaseCleaner.strategy = :truncation DatabaseCleaner.start end def teardown DatabaseCleaner.clean end

    But this isn’t working, Fixtures / Factories created in a before block for the feature test aren’t available for capybara / webkit.

    You got any hint how to solve this?

    Cheers benny

  13. Wow Avdi, thank you so much for this–been struggling with this exact issue for 3 hours. Fixed now.

  14. @avdi:disqus thank you for this material of crucial importance!
    I want to ask Capybara’s maintainer to add a link to this post in the gem’s README, if this is ok for you!

    Do you plan to update your post’s code with the information supplied by @casetaintor:disqus ?

  15. Sometimes you don’t want to clean the db after each example, but after the whole example group. This can be done in rspec 3 using the metadata filters.

    #rails_helper.rb config.around(:example) do |example| unless example.metadata[:noclean] DatabaseCleaner.strategy = example.metadata[:js] ? :truncation : :transaction DatabaseCleaner.cleaning do example.run end end end <pre><code> config.before(:context, :noclean =&gt; true) do |example_group| DatabaseCleaner.strategy = example_group.class.metadata[:js] ? :truncation : :transaction DatabaseCleaner.start end config.after(:context, :noclean =&gt; true) do DatabaseCleaner.clean end #some_spec.rb require "rails_helper" describe "database_cleaner", :noclean do describe "one feature" do context "when working" do before do FactoryGirl.create :foo end it "should have one foo" do expect(Foo.count).to eq 1 end end end describe "another feature" do context "when still working" do it "still have on foo" do expect(Foo.count).to eq 1 end end end end

  16. I was struggling with this post for a while. In the end I needed an additional non_transactional filter for some model specs where I use some transaction breaking feature (load infile etc..). So, just sharing the config I ended up with:

    config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
    end

    config.before(:each) do |example|
    if example.metadata.slice(:non_transactional, :js).values.any?
    DatabaseCleaner.strategy = :truncation
    else
    DatabaseCleaner.strategy = :transaction
    end

    DatabaseCleaner.start

    end

    config.append_after(:each) do
    DatabaseCleaner.clean
    end

    Thanks all for the help in figuring this out.

    1. (can’t edit so follow up post)
      OK, found some relevant nuances:

      Using a truncation strategy after your test, doesn’t ensure the record IDs are reset prior to the test. Sometimes you need this for special tables (arguably code smell…). Anyway, new flag: reset_auto_increments. Only reset the couple of tables which have ID related logic to avoid an overall reset. Also this is completely unrelated to the cleaning strategy afterwards.
      By default truncation is not smart. Of you don’t care about the IDs being reset, use the pre_count and reset_ids options. Or go with deletion instead.

      Here is my new config:

      config.before(:suite) do
      ensure_databases_are_innodb
      DatabaseCleaner.clean_with(:truncation)
      end

      config.before(:each) do |example|
      if example.metadata.slice(:reset_auto_increments).values.any?
      %w(migration_batches appointment_id_seeds).each do |table_name|
      ActiveRecord::Base.connection.execute(“truncate table #{table_name}”)
      end
      end
      if example.metadata.slice(:non_transactional, :js).values.any?
      #DatabaseCleaner.strategy = :deletion
      DatabaseCleaner.strategy = :truncation, {pre_count: true, reset_ids: false}
      else
      DatabaseCleaner.strategy = :transaction
      end
      DatabaseCleaner.start
      end

      config.append_after(:each) do
      DatabaseCleaner.clean
      end

  17. Yes! Thanks for this posting. I was stumped for 1/2 a day on this at one point.

    I have written a guide on configuring Rails 4.1 for Testing with Test::Unit, Minitest, and Rspec. It also reviews the uses of mocking, stubbing, Page Object pattern, and debugging hints. You can find it on my Ruby Reflections blog: http://www.ironhorserails.com/blog

  18. When you create spec/support/database_cleaner.rb, do you “require ‘database_cleaner'” from spec_helper.rb? Is there anything else to do to get it to find your database_cleaner.rb? I can’t seem to get it to find database_cleaner.rb.

    If I copy/paste the contents of your database_cleaner.rb into my spec_helper.rb’s config block directly, it works beautifully.

    But if I just put it in spec/support/database_cleaner.rb it never clears the data. I kinda thought rspec would automatically load files in spec/support but it doesn’t seem to. I tried “require ‘database_cleaner'” from spec_helper.rb and that didn’t help. Probably something little I’m missing.
    (Ruby 2.1.5, rails 4.1.8, rspec-rails 3.1.0)

    1. @gayle – by default, files in spec/support directories aren’t loaded by rspec any longer, so if you uncomment the line in spec helper, it should be ok

      Dir[Rails.root.join(“spec/support/**/*.rb”)].each { |f| require f }

  19. Again, this post was really helpful, but when I ran my feature specs along with the rest of the suite, it slowed the rest of my specs to a grinding hault. Any idea why adding this may cause the rest of the specs to take so long that I’ve had to stop it after 20 minutes when normally it takes 2?

  20. Hey Avdi my favorite rails author and and speaker thanks for the lovely books and yoir deep thoughts on elixir (Im trying it)

    I use Mongoid mostly on rails can you also show us a mongoid version , I think its just using truncation instead o transaction but thats me 🙂

  21. Wow, thanks for this. I was using the truncation strategy for both :suite and :each and my tests were taking up to an hour to run. After switching to the transaction strategy for :each we’re down to about 10 minutes.

    One question I had, though:

    Why not just use the transaction support built right into Rspec?

    Many Thanks,

    Josh

Leave a Reply to Edgars Beigarts Cancel reply

Your email address will not be published. Required fields are marked *