Project

General

Profile

Actions

Bug #19042

closed

Bug: Dir.glob ignores subdirectories in alternation when alternation is preceded by recursive directory pattern

Added by matthew.kern2 (Matt Kern) 2 months ago. Updated about 1 month ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:110220]

Description

The Dir.glob method omits results from subdirectories listed in an alternation when that alternation is preceded by a recursive directory pattern (**/).

Demonstration here: https://replit.com/@MattKern1/Dirglob-subdirectory-alternation-issue?v=1

I have reproduced this in ruby 2.7.2 and ruby 3.0.0.
Local output:

➜  tree
.
└── tmp
    ├── dir_a
    │   └── file_a1.txt
    └── dir_b
        └── subdir
            └── file_b1.txt

4 directories, 2 files
➜  irb
2.7.2 :001 > Dir.glob('tmp/{dir_a,dir_b/subdir}/*.txt')
 => ["tmp/dir_a/file_a1.txt", "tmp/dir_b/subdir/file_b1.txt"] 
2.7.2 :002 > Dir.glob('**/{dir_a,dir_b/subdir}/*.txt')
 => ["tmp/dir_a/file_a1.txt"] 
2.7.2 :003 > exit

You can see that while the text file in dir_a shows up in both examples, the text file in dir_b/subdir shows up when the preceding path is explicitly defined, but not when tmp/ is replaced with the more general recursive directory expression **/.

Works properly with ruby 2.5.3

2.5.3 :001 > Dir.glob('tmp/{dir_a,dir_b/subdir}/*.txt')
 => ["tmp/dir_a/file_a1.txt", "tmp/dir_b/subdir/file_b1.txt"] 
2.5.3 :002 > Dir.glob('**/{dir_a,dir_b/subdir}/*.txt')
 => ["tmp/dir_a/file_a1.txt", "tmp/dir_b/subdir/file_b1.txt"]

This seems related to this line of code: https://github.com/ruby/ruby/blob/7b413c1db3e65909c6899e1d3be4d16c3e76149c/dir.c#L2140

I'm not sure exactly what changed between version 2.5.3 and 2.7.2, but there do appear to be some significant code changes (and the order of results from Dir.glob is sometimes different). Possibly related to this discussion? https://bugs.ruby-lang.org/issues/17280


Related issues 1 (0 open1 closed)

Related to Ruby master - Bug #13167: Dir.glob is 25x slower since Ruby 2.2Closedh.shirosaki (Hiroshi Shirosaki)Actions
Actions #1

Updated by nobu (Nobuyoshi Nakada) about 2 months ago

  • Related to Bug #13167: Dir.glob is 25x slower since Ruby 2.2 added
Actions #2

Updated by nobu (Nobuyoshi Nakada) about 2 months ago

  • Backport changed from 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN to 2.7: REQUIRED, 3.0: REQUIRED, 3.1: REQUIRED

Updated by h.shirosaki (Hiroshi Shirosaki) about 2 months ago

If brace patterns have '/', the brace would fail to match here.
https://github.com/ruby/ruby/blob/7b413c1db3e65909c6899e1d3be4d16c3e76149c/dir.c#L2327

This may be a fix. Expand brace patterns early if the brace contains '/'.

diff --git a/dir.c b/dir.c
index a3ea5eea50..a35aace6f1 100644
--- a/dir.c
+++ b/dir.c
@@ -2305,7 +2305,7 @@ glob_helper(
 #endif
             break;
           case BRACE:
-            if (!recursive) {
+            if (!recursive || strchr(p->str, '/')) {
                 brace = 1;
             }
             break;

Updated by nobu (Nobuyoshi Nakada) about 2 months ago

Thank you.
I made a test, and that patch seems fine.

diff --git a/test/ruby/test_dir.rb b/test/ruby/test_dir.rb
index 514c6e5921e..34d82076ac5 100644
--- a/test/ruby/test_dir.rb
+++ b/test/ruby/test_dir.rb
@@ -259,6 +259,19 @@
     end
   end
 
+  def test_glob_recursive_with_brace
+    Dir.chdir(@root) do
+      %w"c/dir_a c/dir_b c/dir_b/dir".each do |d|
+        Dir.mkdir(d)
+      end
+      expected = %w"c/dir_a/file c/dir_b/dir/file"
+      expected.each do |f|
+        File.write(f, "")
+      end
+      assert_equal(expected, Dir.glob("**/{dir_a,dir_b/dir}/file"))
+    end
+  end
+
   def test_glob_order
     Dir.chdir(@root) do
       assert_equal(["#{@root}/a", "#{@root}/b"], Dir.glob("#{@root}/[ba]"))
Actions #5

Updated by h.shirosaki (Hiroshi Shirosaki) about 2 months ago

  • Status changed from Open to Closed

Applied in changeset git|329d5424a479bb08e75bd750c51a5382e382731c.


[Bug #19042] Fix Dir.glob brace with '/'

Dir.glob brace pattern with '/' after '**' does not match
paths in recursive expansion process.
We expand braces with '/' before expanding a recursive.

Co-authored-by: Nobuyoshi Nakada

Updated by nagachika (Tomoyuki Chikanaga) about 1 month ago

  • Backport changed from 2.7: REQUIRED, 3.0: REQUIRED, 3.1: REQUIRED to 2.7: REQUIRED, 3.0: REQUIRED, 3.1: DONE

ruby_3_1 ffa439fd29141626878f4c4b56e86b2fee17294e merged revision(s) 329d5424a479bb08e75bd750c51a5382e382731c.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0