Feature #9925: rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume

• Status: Closed
• Assignee: -
• Target version: -

[[e-mail] (Aaron Stone)]

In ext/socket/raddrinfo.c, the function rsock_addrinfo() always uses AF_UNSPEC for DNS queries. This is causing me a very high volume of IPv6 DNS lookups. rsock_addrinfo() is used by TCPSocket (and all other Socket base classes, e.g. Socket and UDPSocket), and TCPSocket is used by Net::HTTP.

Remember that DNS does not do negative caching - if a hostname does not have a AAAA record, then DNS will always try to look up that record again!

I propose that the following code should have some way to force IPv4 or IPv6 lookups:

http://rxr.whitequark.org/mri/source/ext/socket/raddrinfo.c

378 struct addrinfo*
379 rsock_addrinfo(VALUE host, VALUE port, int socktype, int flags)
380 {
381     struct addrinfo hints;
382 
383     MEMZERO(&hints, struct addrinfo, 1);
384     hints.ai_family = AF_UNSPEC;
385     hints.ai_socktype = socktype;
386     hints.ai_flags = flags;
387     return rsock_getaddrinfo(host, port, &hints, 1);
388 }

For example, an environment variable named something like RUBY_GAI could be set to "INET" or "INET6" to switch the hints.ai_family away from AF_UNSPEC.

[[e-mail] (Aaron Stone)]

Suggested code change: https://github.com/ruby/ruby/pull/636

[nobu (Nobuyoshi Nakada)]

• Tracker changed from Bug to Feature
• Description updated (diff)

It doesn't feel a good idea to change a behavior in a library by an environment variable implicitly.

[[e-mail] (Aaron Stone)]

Similar configuration is available in the "/etc/gai.conf" system (it does not apply in this use case, however). It is exactly the global nature of the setting that helps me the most. In many cases in the raddrinfo.c file, Ruby is hardcoded to a particular lookup style and that is causing me significant system load for the nonexistent DNS queries.

There is also a compile-time Ruby flag for LOOKUP_ORDER_HACK_INET and LOOKUP_ORDER_HACK_INET6. Perhaps those could become an environment variable instead of compile-time flags?

Another approach could be a module variable in Socket that I could toggle globally or per-instance. Something like this:

Socket::DEFAULT_DNS_LOOKUP = :UNSPEC

I could either globally set Socket::DEFAULT_DNS_LOOKUP = :INET or on a per-instance basis, e.g.

my_socket = TCPSocket.new()
my_socket.dns_lookup = :INET

[[e-mail] (Aaron Stone)]

Ping. I was hoping someone from the Ruby core would provide feedback and direction regarding my suggestions:

There is also a compile-time Ruby flag for LOOKUP_ORDER_HACK_INET and LOOKUP_ORDER_HACK_INET6. Perhaps those could become an environment variable instead of compile-time flags?

and

Another approach could be a module variable in Socket that I could toggle globally or per-instance. Something like this:

Socket::DEFAULT_DNS_LOOKUP = :UNSPEC

I could either globally set Socket::DEFAULT_DNS_LOOKUP = :INET or on a per-instance basis, e.g.

my_socket = TCPSocket.new()
my_socket.dns_lookup = :INET

Another idea, as I was just looking at this again, is that the purpose of rsock_addrinfo is to wrap rsock_getaddrinfo. What if we simply converted the callers of rsock_addrinfo to call rsock_getaddrinfo directly? The list is pretty short, it's Socket, IPSocket, UDPSocket, TCPSocket: http://rxr.whitequark.org/mri/ident?i=rsock_addrinfo In the initializers to these classes, add an optional address_family parameter:

http://www.ruby-doc.org/stdlib-2.1.0/libdoc/socket/rdoc/TCPSocket.html#method-c-new

TCPSocket.new(remote_host, remote_port, local_host=nil, local_port=nil, address_family=nil)

That way, from Net::HTTP connect, at http://rxr.whitequark.org/mri/source/lib/net/http.rb#867 you'd be able to adjust:

878 TCPSocket.open(conn_address, conn_port, @local_host, @local_port)

to specify whether the conn_address should be looked up as IPv4, IPv6, or both.

[craig65535 (Craig Davison)]

I have a similar issue.

While running strace, I noticed that a ruby script sending a UDP datagram to localhost would call sendto twice. The first sendto used an IPv6 address, and would fail, and the second sendto used an IPv4 address and would succeed.

The issue seems to be that sockets are created with a domain of PF_INET by default, but hostname lookups (getaddrinfo) are done with no address family hint. I have attempted to fix this in https://github.com/ruby/ruby/pull/1052 by reading the socket's address family in rsock_addrinfo(), and passing it as a hint to getaddrinfo().

Here is a short script that illustrates the problem.

require 'socket'
socket = UDPSocket.new
socket.send("123", 0, "localhost", 5556)

Here is the strace output, before my patch:

$ strace ~/ruby-master/bin/ruby ~/udp.rb 2>&1 | grep INET | egrep '(send|socket)'
socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_IP) = 7
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 8
socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 8
sendto(7, "123", 3, 0, {sa_family=AF_INET6, sin6_port=htons(5556), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EAFNOSUPPORT (Address family not supported by protocol)
sendto(7, "123", 3, 0, {sa_family=AF_INET, sin_port=htons(5556), sin_addr=inet_addr("127.0.0.1")}, 16) = 3

And after my patch:

$ strace ~/ruby-fix-getaddrinfo/bin/ruby ~/udp.rb 2>&1 | grep INET | egrep '(send|socket)'
socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_IP) = 7
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 8
sendto(7, "123", 3, 0, {sa_family=AF_INET, sin_port=htons(5556), sin_addr=inet_addr("127.0.0.1")}, 1

[[e-mail] (Aaron Stone)]

Thank you Craig! Since your patch was accepted, this ticket is now resolved for me. I'll also mark my Ruby Github PR as resolved by your PR.

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed