Snapcraft clean behavior when stage-packages change


#1

When one changes the stage-packages for a part, snapcraft cleans the whole part (re-pulls and rebuilds it):

Cleaning later steps and re-pulling my-part (‘stage-packages’ property changed)

When the part has a long compile or fetch time this can be quite annoying. I work around it by creating new temporary parts for the stage-packages for development which I later merge with the part they actually belong to. However, I think this is not an ideal workflow.

I wonder why the pull and build steps depend on stage-packages in the first place? Isn’t a part supposed to only require build-packages in the build stage - therefore stage-packages would be part of the install stage? I’m probably missing something :slight_smile:


#2

Is there a reason why this isn’t ideal? FWIW, this is what I always do for precisely the reason you explain, I seem to remember there is a reason why snapcraft works this way, but can’t remember why. But I don’t think having stage-packages in a separate part has any real negative benefit to it, perhaps making it unclear which parts need which stage-packages.


#3

Thanks a lot for your answer! Yeah, it’s not such a big deal. I still think it would be a great improvement if it didn’t have to clean pull/build when stage-packages change because a big feature IMHO of staged build processes like snapcraft’s is that things that are not dependent on each other can be cleared independently. It’s also probably not obvious to new users and would improve the overall user experience IMO.

I seem to remember there is a reason why snapcraft works this way

Maybe someone from the snapcraft team can clarify that? It’s perfectly possible that I’m missing some dependencies that make this feature request impossible or unfeasible.


#4

Ooo ooo, I gotcha. It’s because snapcraft fetches stage-packages before it executes the pull step for the part. In other words, it’s architected such that the pull step can depend upon stage-packages (and indeed, some plugins have this dependency, such as the python plugins. Sadly, snapcraft has no way of knowing whether or not the stage-packages are required to pull any given part, so it errs on the safe side and assumes that’s always the case. And indeed, extracting stage-packages to another part means that they’re not needed to build the part either, which means… why are they stage-packages for that part in the first place? Making them their own part is a perfectly reasonable thing to do.

Understandably, you’re ever so slightly misunderstanding those terms. build-packages and stage-packages aren’t named to correspond to lifecycle steps (e.g. pull, build, stage, etc.). Rather:

  • build-packages are installed onto the host building the snap in order to help build the snap (or the part). These are typically the -dev package that includes headers and stuff that aren’t wanted in the snap, but are required to build it in the first place. Everything you declare here will essentially be apt install'd onto the machine building the snap before the snap building process begins
  • stage-packages aren’t installed onto the host, but downloaded and unpacked (i.e. staged) into the snap itself as a component within the part in which it is declared. These might be the runtime version of the -dev package that is a build-package (e.g. perhaps libfoo-dev is the build-package, and libfoo is the stage-package).

Does that clarify things a little?


#5

Thanks a lot for your answer and detailed explanation! Yep, it’s clear to me now, thank you. I knew the basic role of build and stage packages but I did indeed wrongly assume that they correspond to the lifecycle steps due to the naming.

why are they stage-packages for that part in the first place?

Yeah, I think it obviously makes sense to group instructions that logically belong to the same piece of software in the same part. But there should be no problem naming the part so that it’s clear where the stage packages belong to. Maybe this could be documented as a “best practice”?

I wonder if it would be possible to change the system in such a way that they indeed correspond to the lifecycle steps but I’m not sure if it’s worth the effort :slight_smile: Anyway, thanks a lot!