If you’ve ever worked with C or C++ you no doubt remember that one of continual headaches of working with those languages is avoiding double-inclusions of header files. Most C headers start and end with preprocessor directives in order to avoid this scenario:

#ifndef HELLO_H
#define HELLO_H

/* ... C code ... */

#endif

At first the situation in Ruby seems much improved. We have require, after all, which ensures that a given file will only be loaded once. Or does it?

As it turns out all is not rosy in Ruby-land. Require works great for gems and system libraries. But when we start loading files relative to the current file, the old double-load problem rears it’s ugly head once more.

Let’s take a look at why this happens. First, we’ll define a file to load:

# foo.rb
class Foo
  puts "I'm being loaded!"
end

Now we’ll define a client file which requires foo.rb:

# client.rb
require File.join(File.dirname(__FILE__), './foo')
require File.join(File.dirname(__FILE__), '../lib/foo')

When we run the script above, we get the following output:

I'm being loaded!
I'm being loaded!

Of course, we’d never write a file like that, with the same library being required twice using two different paths. But in larger projects it is all too common for a file to be required using a different path in different files. Because Ruby does not use canonicalized pathnames to check if it has already loaded a file, it assumes that the different paths must refer to different files and loads the file over and over again.

Is this a problem? Besides for slower application startup, the most common ill effect of repeated file loads is constant redefinition warnings. If you have a project that outputs a lot of warnings that look like this on startup…

../lib/foo.rb:1: warning: already initialized constant FOO

…you probably have some files being loaded twice or more times.

More serious and subtle side-effects of double-loading can occur though, especially if the files being reloaded do any class-level metaprogramming. Errors caused by double-loading can be strange and very difficult to track down.

So what do do? Well, first we need to find where the offending loads are originating from. In a large project this can be a daunting task. Here’s some code I wrote to help track down double loads at Devver:

ROOT_PATH = File.expand_path('..', File.dirname(__FILE__))

def require_with_reload_check(raw_path)

  unless $LOADED_FEATURES.include?(raw_path)
    $require_sites ||= {}
    site, line, info = caller.first.split(':')
    expanded_site = File.expand_path(site)
    load_dir = $LOAD_PATH.detect{|dir|
      File.exist?(File.expand_path(raw_path + ".rb", dir))
    }
    expanded_path = File.expand_path(raw_path, load_dir)

    if (expanded_path.index(ROOT_PATH) == 0) &&
        $require_sites.key?(expanded_path) &&
        $require_sites[expanded_path][:as] != raw_path &&
        expanded_path !~ /test_helper$/
      warn "!" * 80
      warn "#{expanded_path} is being reloaded!"
      warn "It was originally loaded as: #{$require_sites[expanded_path][:as]}"
      warn "From #{$require_sites[expanded_path][:in]}"
      warn "But now it is being loaded as: #{raw_path}"
      warn "In #{expanded_site}"
      warn "!" * 80
    end

    $require_sites[expanded_path] = {
      :as => raw_path,
      :in => expanded_site
    }
  end
end

unless defined?($reload_guard_enabled)
  alias require_without_reload_check require
  alias require require_with_reload_check
  $reload_guard_enabled = true
end

This code should be loaded as early as possible in your project. Once loaded, it spits out some a very noisy warning every time a file is re-loaded using a different path:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
/home/avdi/articles/double-load-guards/lib/foo is being reloaded!
It was originally loaded as: ././foo
From /home/avdi/articles/double-load-guards/lib/client.rb
But now it is being loaded as: ./../lib/foo
In /home/avdi/articles/double-load-guards/lib/client.rb
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

But how do we avoid reloads, once we have located the offenders? The simplest remedy is to always expand relative paths before requiring them. I prefer to use the two-argument form of File.expand() to construct fully-qualified paths:

# client.rb
require File.expand_path('../lib/foo', File.dirname(__FILE__))

Eliminate your double-loads, and your Ruby code will load faster, produce fewer warnings, and be that much less prone to bugs.

Published by Avdi Grimm

7 Comments

  1. I disagree. Doing require File.expand_path is way to ugly. Just setup you $LOAD_PATH correctly and you will not encounter any of such problems. Treat your code as if it was a library.

    Reply
    • Konstantin, thanks for your comment. Setting up $LOAD_PATH is essential and I make sure to add the current project to the load path in all but the smallest of projects.

      However, there are always a few files where relative requires can't be avoided. For instance, it is conventional to set up unit test/spec files so that they can be run standalone. In order for this to work each test file has to start out by requiring a test_helper.rb or a spec_helper.rb which then sets up $LOAD_PATH, requires additional libraries, etc. Because this test helper is the first thing to be loaded and can't rely on anything else to be configured, it has to be loaded with a relative path.

      This is where I see the most double-loads occurring, because when all of the tests are run together they each require the test helper file, often with differing relative paths. Using expand_path is a way to ensure that files that must loaded relatively are required in a consistent way.

      I may update the post to make it clear that I'm not suggesting you use relative requires throughout a project.

      Reply
  2. Why not use Ruby 1.9, which fixed the problem by storing the expanded path in the require table? http://eigenclass.org/hiki/Changes+in+Ruby+1.9#l25

    Reply
  3. Another awesome post Avdi! We were having some hard-to-trace double constant definitions and using this code in my Rails pre-initializer TOTALLY surfaced them. Thanks!

    Reply
  4. I found the File.expand_path two-argument trick useful for writing some code that works both in Ruby 1.8 and 1.9 (where require behavior changes)

    Reply
  5. I found the File.expand_path two-argument trick useful for writing some code that works both in Ruby 1.8 and 1.9 (where require behavior changes)

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *