If you’ve ever worked with C or C++ you no doubt remember that one of continual headaches of working with those languages is avoiding double-inclusions of header files. Most C headers start and end with preprocessor directives in order to avoid this scenario:
#ifndef HELLO_H #define HELLO_H /* ... C code ... */ #endif
At first the situation in Ruby seems much improved. We have require
, after all, which ensures that a given file will only be loaded once. Or does it?
As it turns out all is not rosy in Ruby-land. Require works great for gems and system libraries. But when we start loading files relative to the current file, the old double-load problem rears it’s ugly head once more.
Let’s take a look at why this happens. First, we’ll define a file to load:
# foo.rb class Foo puts "I'm being loaded!" end
Now we’ll define a client file which require
s foo.rb:
# client.rb require File.join(File.dirname(__FILE__), './foo') require File.join(File.dirname(__FILE__), '../lib/foo')
When we run the script above, we get the following output:
I'm being loaded! I'm being loaded!
Of course, we’d never write a file like that, with the same library being required twice using two different paths. But in larger projects it is all too common for a file to be require
d using a different path in different files. Because Ruby does not use canonicalized pathnames to check if it has already loaded a file, it assumes that the different paths must refer to different files and loads the file over and over again.
Is this a problem? Besides for slower application startup, the most common ill effect of repeated file loads is constant redefinition warnings. If you have a project that outputs a lot of warnings that look like this on startup…
../lib/foo.rb:1: warning: already initialized constant FOO
…you probably have some files being loaded twice or more times.
More serious and subtle side-effects of double-loading can occur though, especially if the files being reloaded do any class-level metaprogramming. Errors caused by double-loading can be strange and very difficult to track down.
So what do do? Well, first we need to find where the offending loads are originating from. In a large project this can be a daunting task. Here’s some code I wrote to help track down double loads at Devver:
ROOT_PATH = File.expand_path('..', File.dirname(__FILE__)) def require_with_reload_check(raw_path) unless $LOADED_FEATURES.include?(raw_path) $require_sites ||= {} site, line, info = caller.first.split(':') expanded_site = File.expand_path(site) load_dir = $LOAD_PATH.detect{|dir| File.exist?(File.expand_path(raw_path + ".rb", dir)) } expanded_path = File.expand_path(raw_path, load_dir) if (expanded_path.index(ROOT_PATH) == 0) && $require_sites.key?(expanded_path) && $require_sites[expanded_path][:as] != raw_path && expanded_path !~ /test_helper$/ warn "!" * 80 warn "#{expanded_path} is being reloaded!" warn "It was originally loaded as: #{$require_sites[expanded_path][:as]}" warn "From #{$require_sites[expanded_path][:in]}" warn "But now it is being loaded as: #{raw_path}" warn "In #{expanded_site}" warn "!" * 80 end $require_sites[expanded_path] = { :as => raw_path, :in => expanded_site } end end unless defined?($reload_guard_enabled) alias require_without_reload_check require alias require require_with_reload_check $reload_guard_enabled = true end
This code should be loaded as early as possible in your project. Once loaded, it spits out some a very noisy warning every time a file is re-loaded using a different path:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /home/avdi/articles/double-load-guards/lib/foo is being reloaded! It was originally loaded as: ././foo From /home/avdi/articles/double-load-guards/lib/client.rb But now it is being loaded as: ./../lib/foo In /home/avdi/articles/double-load-guards/lib/client.rb !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
But how do we avoid reloads, once we have located the offenders? The simplest remedy is to always expand relative paths before requiring them. I prefer to use the two-argument form of File.expand()
to construct fully-qualified paths:
# client.rb require File.expand_path('../lib/foo', File.dirname(__FILE__))
Eliminate your double-loads, and your Ruby code will load faster, produce fewer warnings, and be that much less prone to bugs.
I disagree. Doing require File.expand_path is way to ugly. Just setup you $LOAD_PATH correctly and you will not encounter any of such problems. Treat your code as if it was a library.
Konstantin, thanks for your comment. Setting up $LOAD_PATH is essential and I make sure to add the current project to the load path in all but the smallest of projects.
However, there are always a few files where relative requires can't be avoided. For instance, it is conventional to set up unit test/spec files so that they can be run standalone. In order for this to work each test file has to start out by requiring a test_helper.rb or a spec_helper.rb which then sets up $LOAD_PATH, requires additional libraries, etc. Because this test helper is the first thing to be loaded and can't rely on anything else to be configured, it has to be loaded with a relative path.
This is where I see the most double-loads occurring, because when all of the tests are run together they each require the test helper file, often with differing relative paths. Using expand_path is a way to ensure that files that must loaded relatively are required in a consistent way.
I may update the post to make it clear that I'm not suggesting you use relative requires throughout a project.
You're right. But specs/tests is about the only place I do so myself.
Why not use Ruby 1.9, which fixed the problem by storing the expanded path in the require table? http://eigenclass.org/hiki/Changes+in+Ruby+1.9#l25
Another awesome post Avdi! We were having some hard-to-trace double constant definitions and using this code in my Rails pre-initializer TOTALLY surfaced them. Thanks!
I found the File.expand_path two-argument trick useful for writing some code that works both in Ruby 1.8 and 1.9 (where require behavior changes)
I found the File.expand_path two-argument trick useful for writing some code that works both in Ruby 1.8 and 1.9 (where require behavior changes)