Access remote/local parts

Need some advice on remote parts

I’ve just built a snap on build.snapcraft.io, more specifically its intended to be used as a part.

I’m now building the snap that will use the above part.

So the question is how does snapcraft find the part when its building my snap.

My understanding is that my part will be on the ‘edge’ channel until approved.

Does that mean I need to swich snapcraft to the edge channel in order to find my part on the build server?

Alternatively how do I get snapcraft to find a local copy of the part?

So I got some help from one of the team and ended up embedding the part via:

parts:
  tomcat-with-ssl:
  plugin: python
  source: https://github.com/bsutton/tomcat-with-ssl-snap
  source-type: git

The problem now is that the embedded part exposes a number of apps (daemons) however I’m not certain how to exposed those app entry points in the new parent snap (after installing the snap the daemons don’t run and are exposed as services).

I tried just copying the app entry points from the part to the snap but that didn’t work (the ‘command’ couldn’t be found).

So what is the correct way to do this.

You can see the part here:

The snap unfortunately is private but the full snapcraft.yaml is:

name: orion-monitor 
version: '0.1' 
summary: monitoring of orion vms.
description: |
  OrionMonitor is a webapp designed to monitor Orion Vms and stop/start them
  when they are not required.

grade: devel 
confinement: devmode 

# this is what I tried origional but got an error 'tomcat-launch' can't be found.
#apps:
  #tomcat:
    #command: tomcat-launch
    #daemon: simple
    #plugs: [network, network-bind]

parts:
  # build the web app
  orion-monitor-webapp:
    plugin: maven
    source: git@bitbucket.org:sbsutton/orionmonitor.git
    maven-options:
      [-DskipTests=true]
    organize:
      war/orionmonitor-1.0-SNAPSHOT.war : webapps/orionmonitor.war
    after: [tomcat-with-ssl]

  tomcat-with-ssl:
    plugin: python
    source: https://github.com/bsutton/tomcat-with-ssl-snap
    source-type: git

BTW need to look at the doco on using non-published parts. I would never have guessed that I needed a plugin of ‘python’ to include the part and I spent a long time looking for an answer.
Even after I understood what needed to be done I couldn’t find the doco explaining it!

Brett

what creates “tomcat-launch” where would it come from ? (i dont see it in your remote part).

Its in the remote parts but gets generated dynamically as part of the ‘install:’

  install: |
    cp -r output/build/* $SNAPCRAFT_PART_INSTALL
    cat <<EOF > $SNAPCRAFT_PART_INSTALL/tomcat-launch

ok, i’m officially blind (missing indendation really tricked me there) :slight_smile:

do you actually see the part being run when you build your snap ?
do you see the launcher in the parts/tomcat-with-ssl/install/ dir ?

there is some nice explanation from @kyrofa about sharing files between parts at:

inline.

[ogra] ogra https://forum.snapcraft.io/u/ogra February 21

ok, i’m officially blind (missing indendation really tricked me there) :slight_smile:

I have a spare set of glasses here somewhere… :slight_smile:

do you actually see the part being run when you build your snap ?

Not certain what you mean by ‘part being run’

do you see the launcher in the parts/tomcat-with-ssl/install/ dir ?

No the launcher is missing.

Does the launcher need to be dynamically generated (I copied this from an existing snap part). Can I not just put it into the snap directly?

there is some nice explanation from @kyrofa https://forum.snapcraft.io/u/kyrofa about sharing files between parts at:

I will have a read.

snapcraft tells you what it does, do you see the remote part being pulled/prepared/built/staged/primed ?

$ snapcraft
Preparing to pull the-part 
Pulling the-part 
Preparing to build the-part 
Building the-part 
Staging the-part 
Priming the-part 
Snapping 'foo' |                                                                                
Snapped foo_0.1_amd64.snap

i suspect the remote part isnt doing a lot, you define a python plugin but there is no setup.py or anything in https://github.com/bsutton/tomcat-with-ssl-snap so it is a no-op in the end …

1 Like

OK, I used the python part on the advice of one of you colleagues.
It didn’t make much sense to me.

I’m working from this example:

They just use the ‘after’ clause to pull the part because its hosted on the build servers.

As my part isn’t ready for release its not on the build servers so I started how to include a local part or the part from git.

So I guess the question is how should I actually include this part so it works.

I did just realise that the example does have the ‘apps’ defined in the snap rather than the part so I’m currently moving the apps statements in to the snap.

It would also be handy to know how I can pull the code from the local host rather than git as this would speed up my test cycles.

Yes its building the part.

So I can partially answer this question.

If you have a part stored on your local host you can access the part by specifiing a ‘source:’ with a relative path.

The path MUST be relative to where you run ‘snapcraft’ from.

So you can do something like this:

parts:

  tomcat-with-ssl:
    plugin: dump
    source: ../../../tomcat-with-ssl

I’m still uncertain if the ‘plugin: dump’ is the correct plugin to use.

So I’ve tried something different.

I moved the script out of the ‘install:’ clause and create the script on disk in snap/src/scripts/tomcat-launch

I then added an organize statement to the part:

  tomcat:
    plugin: ant
    source: https://github.com/apache/tomcat85.git
    source-type: git
    prepare: |
      cp build.properties.default build.properties
    organize:
      snap/src/scripts/tomcat-launch : tomcat-launch

Alas the tomcat-launch file is still not be copied anywhere.
I’ve search parts,stage and prime and the file is being copied to none of these.

Note these changes were made in conjuction with the above noted change to using the ‘dump’ plugin from the snap:

parts:
  tomcat-with-ssl:
    plugin: dump
    source: ../../../tomcat-with-ssl

One thing i noticed is that https://github.com/bsutton/tomcat-with-ssl-snap carries its snapcraft.yaml in the snap/ dir, i wonder if remote parts actually respect this.

All other remote parts i have seen yet have their yaml file in the toplevel dir, might be worth a try to move it one level up … i have slight doubts that your remote snapcraft.yaml is even executed.

I can try that in the morning (its 11pm here).

There still is the question of what plugin I should use when including a remote part.
I’ve re-read the doco and can’t see anything that seems to apply.

What I did for a remote part that was on build servers was use a simple ‘after:’. It seems problematic that I have to jump through a whole lot more hoops just because its a local part.

I think we need some more doco around using parts to make it clear what the process is for each method of including a part i.e.
local source
build server
git repo (or similar).

So I’ve move snapcraft.yaml as suggested and its still not working.

It looks to me like its still not actually building the part.

I see:

snapcraft
Preparing to pull tomcat-with-ssl 
Pulling tomcat-with-ssl 
'orion-monitor-webapp' has prerequisites that need to be staged: tomcat-with-ssl
Preparing to build tomcat-with-ssl 
Building tomcat-with-ssl 
Staging tomcat-with-ssl 
Preparing to pull orion-monitor-webapp 
Get:1 http://mirror.orionvm.com/ubuntu xenial InRelease [247 kB]   

The output suggests its not actually building tomcat-with-ssl as I should see it installing a deb repo and installing the certbot deb.

I’m really quite confused why this isn’t working.

Surely other people are including local parts in their snaps and they don’t have this problem. I would have thought this was a common use case?

I really need to get this solved.

So I’ve gone back and re-read the doco and I’m left even more confused.

The question that I’ve not got a clear answer to is, what ‘plugin’ should I use when pulling in a local part.

The doco says this:

plugin:
Specifies the plugin that will manage this part. Snapcraft will pass the plugin all the other user-specified part options, those options defined with the other keys below. There are three ways in which the plugin can be defined: a plugin name to use a built-in plug-in (for details, see the list of built-in plugins), a local path such as parts/plugins/x-plugin_name.py to use a local (custom defined) plugin and if plugin is not defined locally, the plugin defined for the part in the parts repository.

Now the point of interest is this:

and if plugin is not defined locally, the plugin defined for the part in the parts repository.

This seems to imply that for remote parts you don’t specify a plugin as the plugin details are taken from the yaml of the remote part.

Now this makes a lot of sense. A remote part will actually have one or more of its own parts. Each of these parts will in turn have their own plugin that defines how to build them.

So if there are multiple parts, each with their own plugin then it makes NO SENSE to specify the plugin when pulling a remote part.

This is in fact demonstrated by the ‘after’ clause. If you pull in a remote part using the ‘after’ clause then there is no need and in fact no way to specify the plugin for the remote part.

So why would we be specifying a plugin for a ‘local part’. Where a local part means a part with its own snapcraft.yaml which lives in another directory on the host machine.

So why then can’t i do the following:

parts: 
  tomcat-with-ssl:
    source: ../../../tomcat-with-ssl

the path …/…/…/tomcat-with-ssl points to the local path which has its own snapcraft.yaml.

The problem is that when I try to build this I get:

Failed to get part information: Cannot find the definition for part 'tomcat-with-ssl'. If it is a remote part, run `snapcraft update` to refresh the remote parts cache. If it is a local part, make sure that it is defined in the `snapcraft.yaml`.

I’m banging my head against a brick wall on this one.