Wildcard syntax in organize keyword?

snapcraft
1

I’ve noticed that in the alsa remote part wildcard notation is used in the organize keyword to allow build in different snaps:

    organize:
      snap/*/current/usr/lib/*: usr/lib/
      snap/*/current/usr/share/*: usr/share/

This syntax isn’t documented in snapcraft help plugins and do not conform to the following expected syntax from the current documentation:

    organize:
      snap/*/current/: /

So I play with it a bit, here’s the result:

Without organizing:

parts/alsa-lib/install/
├── snap
│   └── alsa
│       └── current
│           └── usr
│               └── share
│                   └── alsa
│                       ├── alsa.conf
│                       ├── alsa.conf.d
│                       │   └── README
│                       ├── cards

With organize: { snap/alsa/current/: / }:

parts/alsa-lib/install/
├── snap
│   └── alsa
└── usr
<stripped>
    └── share
        ├── aclocal
        │   └── alsa.m4
        └── alsa
            ├── alsa.conf
            ├── alsa.conf.d
            │   └── README
<stripped>
16 directories, 120 files

With organize: { snap/*/current/: / }:

parts/alsa-lib/install/
├── current
│   └── usr
│       └── share
│           └── alsa
│               ├── alsa.conf
│               ├── alsa.conf.d
│               │   └── README
│               ├── cards
│               │   ├── AACI.conf
<stripped>

With organize: { snap/*/current/*: / } the build failed with:

shutil.Error: Destination path '/home/ubuntu/snapcraft-alsa/parts/alsa-lib/install/usr' already exists

With organize: { snap/*/current/usr/*: usr/ } the build failed with:

shutil.Error: Destination path '/home/ubuntu/snapcraft-alsa/parts/alsa-lib/install/usr/share' already exists

With organize: {snap/*/current/usr/share/*: usr/share/}, finally, the expected result:

parts/alsa-lib/install/
├── snap
│   └── alsa
│       └── current
│           └── usr
│               └── share
└── usr
<stripped>
    └── share
        ├── aclocal
        │   └── alsa.m4
        └── alsa
            ├── alsa.conf
            ├── alsa.conf.d
            │   └── README
<stripped>

Is there any proper documentation on how this works, and it is supported or not in the first place?

Also, this could be a lot easier if the organize keyword understands the $SNAPCRAFT_PROJECT_NAME expansion.

2

The logic seems to be located at https://github.com/snapcore/snapcraft/blob/9d1407a852dcf90a40d960757c598ec148ad9abb/snapcraft/internal/pluginhandler/__init__.py#L917

def _organize_filesets(fileset, base_dir):
    for key in sorted(fileset, key=lambda x: ['*' in x, x]):
        src = os.path.join(base_dir, key)
        # Remove the leading slash if there so os.path.join
        # actually joins
        dst = os.path.join(base_dir, fileset[key].lstrip('/'))

        sources = iglob(src, recursive=True)

        for src in sources:
            if os.path.isdir(src) and '*' not in key:
                file_utils.link_or_copy_tree(src, dst)
                # TODO create alternate organization location to avoid
                # deletions.
                shutil.rmtree(src)
            elif os.path.isfile(dst):
                raise errors.SnapcraftEnvironmentError(
                    'Trying to organize file {key!r} to {dst!r}, '
                    'but {dst!r} already exists'.format(
                        key=key, dst=os.path.relpath(dst, base_dir)))
            else:
                os.makedirs(os.path.dirname(dst), exist_ok=True)
                shutil.move(src, dst)

I wonder why the logic treats keys with stars differently?

3

the wildcards are handled by by the iglob() python function on the 8th line in your code dump above. iglob() will expand any wildcards to a list of the files and folders that they match; above it stores the result into the sources variable and then iterates them in the second for loop.

1 Like