Trying to download this file from this website with OpenUri fails with the runtime error "HTTP redirection loop".
Here is how I can reproduce the error:
> require 'open-uri'
=> true
> open('http://apps.london.ca/OpenData/ShapeFiles_Zipped/2010_skateboard_parks_shp.zip')
RuntimeError: HTTP redirection loop: http://apps.london.ca/uniquesig87fdc01fb86ce6f0fd235c713015d7d7/uniquesig0/InternalSite/StartApp.asp?resource_id=837A134B9EC24A2197B6AF5745B6CA55&login_type=0&site_name=appstrunk&secure=0&orig_url=http%3a%2f%2fapps.london.ca%2fOpenData%2fShapeFiles_Zipped%2f2010_skateboard_parks_shp.zip
from /home/john/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/open-uri.rb:232:in `open_loop'
from /home/john/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/open-uri.rb:150:in `open_uri'
from /home/john/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/open-uri.rb:716:in `open'
from /home/john/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/open-uri.rb:34:in `open'
from (irb):2
from /home/john/.rvm/rubies/ruby-2.2.2/bin/irb:11:in `<main>'
$ curl -L http://apps.london.ca/OpenData/ShapeFiles_Zipped/2010_skateboard_parks_shp.zip
curl: (47) Maximum (50) redirects followed
It seems the 302 Object Moved handler on this server has not been properly configured; it expects the previous request to have set a few cookies and simply sends the client back if it doesn't find them.
OpenURI appears to be incapable of handling such a circumstance, but Net::HTTP can and isn't that much more complex. I've presented below a demonstration of how you might go about orchestrating the "handshake" in order to successfully obtain the file.
i've had a similar issue with open('http://www.replayjeans.com/us/shop/product/women/jumpers-knitwear/neoprene-printed-sweatshirt/pc/48/c/61/sc/-1/1962')
which i solved modifying D.E. Akers workaround a bit:
url = 'http://www.replayjeans.com/us/shop/product/women/jumpers-knitwear/neoprene-printed-sweatshirt/pc/48/c/61/sc/-1/1962'
uri = URI(url)
res = Net::HTTP.get_response(uri)
cookie = res['Set-Cookie']
req = Net::HTTP::Get.new(uri)
req['Cookie'] = cookie
res = Net::HTTP.start(uri.hostname, uri.port) { |http| http.request(req) }
However there is another issue that i can't resolve.
Some requests don't end up in a redirection loop, but response has a status 302 Moved Temporarily and open(url) returns not the page i expect.
If I apply redirect loop workaround for this case, i get the correct page.
Tried to google but no avail so far.
I don't think this is a bug. As 0x0dea (D.E. Akers) pointed out, other programs work the same way. However, I think cookie handling in open_uri could be a useful feature. Attached is a patch that implements the necessary support.
I used this Roda app to test the cookie redirection support: