Preserving session with Mechanize

Here’s one for the so-I’ll-find-it-next-time-I-google files.

Say you’re automating some website interactions with Mechanize.

require "mechanize"

agent = Mechanize.new
# screen-scrapey stuff...

Say these interactions require a slow login process of fetching a login page, filling in fields, submitting, and waiting for the response.

login_page = agent.get("/login")
form  = login_page.form_with(action: "/login")
form.username = login
form.password = password
home_page = agent.submit

Say you reeeeeeally don’t want to have to wait for this login process every single time the task runs. With Mechanize, you can save and load the agent’s cookies to a YAML file.

if File.exist?("cookies.yaml")
  agent.cookie_jar.load("cookies.yaml")
else
  login_page = agent.get("/login")
  form  = login_page.form_with(action: "/login")
  form.username = login
  form.password = password
  home_page = agent.submit
end

# screen-scrapey stuff

agent.cookie_jar.save("cookies.yaml")

(You can save the cookies in other formats too, but that’s beyond the scope of this article).

If you do this, you may be surprised to find that your logged-in session still isn’t preserved. That’s because by default, Mechanize skips session cookies when saving them.

But you can override this behavior with an option:

agent.cookie_jar.save("cookies.yaml", session: true)

With the session flag set, Mechanize will write out the full contents of its cookie jar for later loading.