cURL works but mechanize doesn't

Question

I am trying to scrape my university web site using ruby mechanize. This my ruby script;

require 'mechanize'
agent = Mechanize.new
agent.get('https://kampus.izu.edu.tr')

This script doesn't return response. I need to see login page but the response is different. I also tried it with cURL like this;

curl https://kampus.izu.edu.tr

This works and return the login page. What am I missing?

Answer 1

Make sure that you are storing the output of agent.get(). From your example, I don't see how you would be using/printing the response of this request.

Try this:

require 'mechanize'
agent = Mechanize.new 
page = agent.get("https://kampus.izu.edu.tr")
puts page.body

The .get() method returns a Mechanize::Page object that you can call other methods on, such as .css(), to select elements by css selectors. Check out the documentation here