Globstar is enabled, and the pattern `**/` returns files only inside sub-directories but `**/*` returns files both in root and in sub-directories

Question

My file structure looks like this

-rw-r--r-- 1 athar 197609 12467 Jun 12 22:33 rootFile1.md
-rw-r--r-- 1 athar 197609   736 Jun  9 17:42 rootFile2.md
-rw-r--r-- 1 athar 197609   662 Jun  9 18:21 rootFile3.md
drwxr-xr-x 1 athar 197609     0 Jun 15 23:58 testfolder/

with the testfolder/ containing

testfolder:
subDirFile1.md  subDirFile2.md

I've enabled globstar using shopt -s globstar. The documentation of globstar states (source)

If set, the pattern ‘**’ used in a filename expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a ‘/’, only directories and subdirectories match.

$ ls **/
subDirFile1.md  subDirFile2.md

This makes sense according to the docs, where it only matches dirs and sub-dirs, not files. However,

$ ls **/*.md
rootFile1.md  rootFile3.md               testfolder/subDirFile2.md
rootFile2.md  testfolder/subDirFile1.md

Shouldn't the above command only output the subDirFiles since ** is followed by / and as per docs, only directories and subdirectories match? Why does it match files in the root as well?

I should also mention that when i disable globstar, the output will only search the sub directory.

$ ls **/*.md
testfolder/subDirFile1.md  testfolder/subDirFile2.md

Answer 1

I think part of the confusion here is that ls itself also implicitly expands directories -- that is, if you pass a directory name as an argument, it lists the contents of the directory rather than the directory itself. This means the output from something like ls **/ is the result of two expansion processes, each of which plays by different rules.

You can see this clearly with set -x, which makes bash print each command after substitutions and expansions have been done:

$ ls **/
+ ls testfolder/
subDirFile1.md  subDirFile2.md

Here, the **/ expands to just the directory that's in the current directory. It does not include the current directory, which is why rootFile1.md etc are not listed.

It would include subdirectories if there were any. Let's create one and see the effect:

$ mkdir testfolder/subdir
+ mkdir testfolder/subdir
$ touch testfolder/subdir/subsubfile{1..2}.md
+ touch testfolder/subdir/subsubfile1.md testfolder/subdir/subsubfile2.md

$ ls **/
+ ls testfolder/ testfolder/subdir/
testfolder/:
subDirFile1.md  subDirFile2.md  subdir

testfolder/subdir/:
subsubfile1.md  subsubfile2.md

Now ls got two directories as arguments, so it breaks the listing into sections by directory, making it clearer what's going on.

Another way to help clarify this is to use ls -d, which tells ls not to implicitly expand directories, so you only see one level of expansion happening.

Answer 2

From man bash:

When the globstar shell option is enabled, [..] two adjacent *s used as a single pattern will match all files and zero or more directories and subdirectories.

Your rootFile1.md does indeed have zero directories in front, so it matches.

Answer 3

To officially expand on the existing answer, when globstar is enabled:

• ** matches "all files and zero or more directories and subdirectories"

• **/ will not match files on its own, but otherwise it behaves the same as ** meaning it CAN match zero directories/subdirectories.

• *.md is a plain old glob and can match on anything.

When you put it all together, **/*.md says "match in zero or more directories/subdirectories something ending in .md." Your root directory is zero directories deep, so it's included in the match.

For completeness, when globstar is disabled ** is equivalent to * so ls **/*.md is the same thing as ls */*.md.

If your goal is to ignore the root directory, you can prepend the pattern with */ to create */**/*.md. The leading */ will only match directories and doesn't have the same gotcha of matching on "zero directories".