select doesn't handle fd's > FD_SETSIZE very well
The main motivation for reporting this bug is that currently on win32 [any version] if you try to pass more than 64 sockets to select, it ignores the 65th onward. This limit is not obeyed by Ruby currently and therefore some sockets will never select.
I'd first suggest raising an error if more than 64 total sockets are passed [to the most recent thread that called select?]
While examining it, this same aspect [too many or too high of numbered selects] is also seen to cause some problems on other platforms.
In the test cases attached, sometimes Mac OS X consistently crashes, sometimes Linux does. Depends on the test.
I'd bet that these problems also continue for Ruby 1.9 as well.
How to run tests:
first touch file 'abc' # used for some tests
then run ulimit -n 2000 # if this fails, you may need to run "sudo bash" first
then run the files.
To ruby's credit, without having first run the ulimit, above, most platforms don't seg fault [though windows still has the difficulty mentioned], so this isn't life critical.
Note that in Linux/OS X if a file descriptor's "number" is > FD_SETSIZE then it will be ignored by select. In Windows if the "total number of descriptors" past to select is > FD_SETSIZE then those past FD_SETSIZE will be ignored.
Updated by usa (Usaku NAKAMURA) over 10 years ago
- Priority changed from Normal to 3
- ruby -v set to -
When the patch is applied, every select call uses the stack by 112KB (about 1.8KB at present).
And, if you pass 4097 sockets to select, the same problem occurs again.
So this approach is not preferable.
In 1.9, this problem has already been solved, because we didn't have to consider the binary level compatibility.
But in 1.8, we should give priority to maintenance of compatibility more than such a rare case.
Of course, it is necessary to correct this problem if we can.
I leave this problem as Open until another effective ideas arise.
Updated by rogerdpack (Roger Pack) over 10 years ago
Two things come to mind, if 4096 is not an acceptable option :)
1) check for fd number count before doing a select. If it's > FD_SETSIZE raise error.
2) A compromise, ex: setting FD_SETSIZE to 512, since that's the limit of MSVCRT 6, which is used by 99% of 1.8.x compilers. I put 4096 as a conservative number.
Thanks for looking into this.
Updated by rogerdpack (Roger Pack) about 10 years ago
Would it be possible to at least set FD_SETSIZE to 256 so that it works with VC6/mingw (which is 99% of ruby on windows distros)?
That would be good--currently there it is hard for extensions to use select correctly because they overflow the FD set and cannot reset it because it is hard compiled to too small on windows.
Updated by rogerdpack (Roger Pack) over 9 years ago
Another option might be something like the following (any feedback?)
--- win32/win32.h (revision 26621)
+++ win32/win32.h (working copy)
@@ -23,6 +23,9 @@
+#if defined MINGW32
+# define FD_SETSIZE 256 // larger default for msvcrt 6