Project

General

Profile

Actions

Bug #19621

closed

Resolv::Hosts uses ineffective File.read, making using big hosts file 'impossible'

Added by felix.wolfsteller@betterplace.org (Felix Wolfsteller) over 1 year ago. Updated over 1 year ago.

Status:
Closed
Assignee:
-
Target version:
-
[ruby-core:113372]

Description

By default on unixoid systems, Resolv will read /etc/hosts once. Privacy- and security aware people might use the file to prevent unwanted traffic, developers use it to quickly manipulate address resolution.

Resolv::Hosts uses IO.read, which seems to be inefficient when dealing with large amounts of data that should be consumed by line.

E.g. if you install the /etc/hosts additions by hblock (https://github.com/hectorm/hblock), the first call to resolve an address will likely take minutes.

Unfortunately, replacing .open ... .each with IO.foreach does not help.

Benchmarking with partial examplary /etc/hosts from above (172751 line) with this

require 'resolv'
require 'benchmark'

Benchmark.measure do
  Resolv::Hosts.new.lazy_initialize
end

yields to

25.622515   8.821095  34.443610 ( 34.495448)

.

Reading in all the lines into memory first and then consuming them (File.readlines) might improve the situation, but is probably not desirable due to memory concerns.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0