Overwriting existing file in organize

snapcraft
1

I tried something like this in snapcraft.yaml

...
organize:
    somefile: /usr/lib/somefile
...

Where /usr/lib/somefile was installed by apt. Snapcraft complains that it cannot overwrite the file because it already exists. The trouble is, “somefile” is a script that I need to adjust to make java correctly deal with fonts (https://askubuntu.com/questions/791853/snap-package-for-java-swing-and-awt-application-crashes-when-it-is-executed).

Probably there is a more elegant way of solving this problem :slight_smile: but is there some way to force snapcraft to do this? I.e. override-build and rm -f or some such thing?

2

I’m not totally sure if this is related to this thread (please excuse my if it is not) but I’m also running into issues with organize currently:

  File "/snap/snapcraft/2722/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/_runner.py", line 193, in _handle_builtin_function
    function(**function_args)
  File "/snap/snapcraft/2722/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 744, in _do_stage
    fixup_func=fixup_func,
  File "/snap/snapcraft/2722/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 1078, in _migrate_files
    snapcraft.file_utils.create_similar_directory(src, dst)
  File "/snap/snapcraft/2722/lib/python3.5/site-packages/snapcraft/file_utils.py", line 265, in create_similar_directory
    stat = os.stat(source, follow_symlinks=follow_symlinks)
FileNotFoundError: [Errno 2] No such file or directory: '/root/parts/desktop-file/install/usr/share/applications'

when simply doing

  desktop-file:
    plugin: nil
    source: .
    organize:
      fluffychat_snap.desktop: usr/share/applications/
    stage: [ usr/share/applications/fluffychat.desktop ]

or even doing it manually:

  desktop-file:
    plugin: nil
    source: .
    override-build: |
      mkdir -p $SNAPCRAFT_PART_INSTALL/usr/share/applications/
      cp fluffychat_snap.desktop $SNAPCRAFT_PART_INSTALL/usr/share/applications/fluffychat.desktop

… doesn’t work:

...
  File "/snap/snapcraft/2650/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/_runner.py", line 193, in _handle_builtin_function
    function(**function_args)
  File "/snap/snapcraft/2650/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 744, in _do_stage
    fixup_func=fixup_func,
  File "/snap/snapcraft/2650/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 1097, in _migrate_files
    os.remove(dst)
IsADirectoryError: [Errno 21] Is a directory: '/root/stage/usr'

(Trying to snap https://github.com/ChristianPauly/fluffychat)

Any ideas what I’m doing wrong here? Thanks a lot in advance :slight_smile:

$ snapcraft --version
snapcraft, version 3.3
3

Make sure you’re building the snap in a disposable environment(the default multipass VM will do), then you can do those things in the override-build scriptlet(with superuser privileges).

The organize keyword are only used to manipulate files meant to be shipped in the snap.

1 Like
4

Thanks, I’m using the default multipass vm.

5

I would suggest running the steps one by one and use the tree command to examine what happened in each Snapcraft lifecycle steps:

  • snapcraft help [pull|build|stage|prime|pack]
  • snapcraft build --shell _part_name_
  • snapcraft build --shell-after _part_name_
2 Likes
6
  • The dump plugin should be used here
  • The stage property is not necessary as Snapcraft by default stage everything in the $SNAPCRAFT_PART_INSTALL directory
  • Also note you’ve specify different filenames in the organize and stage property: fluffychat_snap.desktop versus fluffychat.desktop, if you want to rename by using the organize property use:
    organize:
      fluffychat_snap.desktop: usr/share/applications/fluffychat.desktop
1 Like
7

Thank you very much for your help!

  desktop-file:
    plugin: dump
    source: .
    organize:
      fluffychat_snap.desktop: usr/share/applications/fluffychat.desktop
    stage: [ usr/share/applications/fluffychat.desktop ]

Unfortunately, I still get this exception:

  File "/snap/snapcraft/2722/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/_runner.py", line 193, in _handle_builtin_function
    function(**function_args)
  File "/snap/snapcraft/2722/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 744, in _do_stage
    fixup_func=fixup_func,
  File "/snap/snapcraft/2722/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 1097, in _migrate_files
    os.remove(dst)
IsADirectoryError: [Errno 21] Is a directory: '/root/stage/usr'

Removing the stage property doesn’t seem do fix it either.

snapcraft build --shell-after _part_name_

looks all right (install directory):

└── usr
    └── share
        └── applications
            └── fluffychat.desktop

Looks like it fails when staging?

8

That is indeed wierd, the entire snapcraft yaml source would be helpful.

1 Like
9
1 Like
10

Oh just realized I haven’t pushed the desktop file yet (since it doesn’t work yet). Will push it in a second so you can have a look at it.
EDIT: Pushed the changes with the desktop file

1 Like
11

You should avoid specifying stage and prime keywords for now.

/cc @reviewers @architects Can anyone check out this? The stage rules from one part(fluffychat) interfere with the stage step of another unrelated part(desktop-file).

1 Like
12

Wow, thanks a lot, looks like that’s the issue indeed. Smells like a bug to me. At least that’s not how this used to work unless I’m overseeing a mistake that I have made.

13

Can you come up with a simple reproducer in a bug report please?

2 Likes
14

I’ll try to craft one out of it.

1 Like
15

Heh, and for anyone wondering about my original question . . . I solved my own problem by realizing that I had created a situation where I was running into conflicts because different parts were trying to overwrite each other in stage. I re-crafted one part to handle creating a jar and another to handle putting a wrapper in the right place.

One thing I would love to see in the docs, @degville, (which by my last reading doesn’t seem to be there – please correct me if I’m wrong) is a clear explanation of how snapcraft moves files between pull -> build -> stage -> prime in different circumstances and what snapcrafters can do to affect that (like overriding sections, using the organize, stage and prime keywords, etc). I have a general idea of how this works, but I do often find myself logging into my snapcraft VM to find out where snapcraft put things when my snap doesn’t work.

2 Likes
16

One thing I would love to see in the docs, @degville,

Hello Spence! Thanks for the suggestion, it’s not explicitly covered at the moment and I think you’re right about the importance of making this clear. We should get close while filling in the missing parts for Snapcraft build, debug and publishing docs roadmap, but I’m going to specifically add what you’re suggesting so that we don’t miss it.

17

@degville, yes, that looks very promising. It would have saved me some time! I’d be happy to volunteer to be a beta reader/reviewer, if you need one.

I was also planning on writing a tutorial in my documentation system (github.com/att/mktechdocs) that would live on github. Mostly it would be for me so I wouldn’t forget things, but maybe someone here could use it eventually.

1 Like
18

Not sure if this will be helpful to someone else but I solved my problem like this:

For a given stage, there is a file that I want to replace later with a different stage. Right now I want to replace it wholesale instead of patching it. I can do that at a later point and is probably a better way to do this. However, it’s nice to do this short-term while iterating and also just to know how to do it in general. I override the pull for that stage like this:

override-pull: |
  snapcraftctl pull
  rm $SNAPCRAFT_PART_SRC/path/to/my/file/somefile.txt

Then the later stage can have that file and put it in the same place and there are no issues.

19

And now I’m not actually sure that was working since I am getting weird behavior on successive tries. Perhaps I’ll just make it a patch and move on?

20

I saw it mentioned elsewhere and haven’t tried it myself but a specific example for fileset exclude may yield the desired results: