External repositories


#11

There’s no “Ubuntu Store”. What we have is a Snap Store, which is a cross-distribution platform.

The point I’m trying to convey above is that most users don’t want to install repositories. They want to install software, and they want to have a good experience doing that. This is the focus of our design.

Don’t get me wrong, though. I know there are clearly advantages in having federated repositories, but there are also a large number of problems related to user experience that we’re not willing to just ignore. If and when we introduce such a concept, all of these will have to be nailed down beforehand, and that takes time and energy that we’re consciously choosing to put on the fundamental problems today.

Just walk around the forum for a bit and you’ll see what I mean. The volume of on going activity and work is non-trivial, and some of the topics and features discussed are extremely interesting capabilities that have a direct impact into the way users and developers interact. This is our priority.

That’s a very particular interpretation of something being free, and one you might want to rethink at some point. By the end of the day we both want to have a good platform for distributing software on, and the conversation here says more about the priorities in doing so than the nature of the work.


#12

technically yes, practically, to gain the full feature and security set you will need a lot of manpower to implement all the surroundings from user authentication over “upload security checks of packages”, package signing, assertion management and serial-vault … the above is really only a demo, definitely nothing you want to use in production …


#13

Okay. Snap Store, but what is the governance of the Snap Store? Does Canonical own it, able to change the rules or shut it down at any time if they like? Or is Canonical obligated to always maintain it now?


#14

@G.S.1 Have a look at the #store topic here in the forum. That’s where the conversations about reviews, auto-connections, renames, and anything else involving the metadata maintenance takes place. It’s an open process with documents linked describing the particular exchanges on certain events, and participation on the individual events is open. In fact, participation from people with insight into related topics is very appreciated.

Obviously, this whole project is backed by Canonical, so if you are worried about its continuation, the proper way to help is to join us. Developing alternative stores for very successful projects is pretty fun. :wink:


#15

What I meant by “repositories” is that even if the Snap Store (or a variant of it) be open source, there is no way to actually add other stores. Unless of course you were being sarcastic…


#16

I was being honest rather than sarcastic.

Similar to the points made above, one of the key reasons why the store is not open source is because the source code from it originated from a convoluted evolution of a system that was five (?) years old and had several strict dependencies on the shape of our infrastructure, including things that were completely unrelated to snaps. We have a team working to clean that up now, and while I agree it would be much better to do all this in the open, I pick my battles. The snapd source code which implements the complete in-system logic is entirely open, snapcraft is completely open, the tooling we use to generate Ubuntu Core is also open, the testing infrastructure we developed to test snapd is open and has multiple distributions on it, even the conversations around all of those things are open.

So, as a long term open source developer, I think we’re doing pretty well, and I’m happy that we’re sustaining our focus on solving the actual problems we came here to solve, as better detailed above.


Disabling automatic refresh for snap from store
Disabling automatic refresh for snap from store
#17

Ok, just my 2 cents. I still think that G.S.1 has a valid point and that being able to run independent “stores” in a user-friendly fashion is important in the long run. There are various possible reasons why the current situation is not ideal, for example:

  • someone may want release a “controversial” app that Canonical doesn’t want to be associated with for whatever reason. Fine, but if that means being unable to release it at all through Snap, then Snap is NOT an open source platform, no matter what the licence says; or

  • some apps may be simply humongous, extremely resource-intensive to compile and way bigger than what Canonical’s store is willing/able to provide in terms of resources and hosting. For example, Flightgear with all its data can easily reach many gigabytes. If it’s not practical or economical for Canonical to host such packages, should this mean that they simply can’t be released through Snap?

I fully understand the reasoning behind the current design and policy, but Linux is not and should not try to be iOS with its centralised top-down ecosystem. In all snap vs. flatpak discussions the objection to the single centralised store model keeps coming back again and again and again, and IMHO Canonical should listen to the packagers’ community if it wants its platform to be successful. Frankly, adding an “add store URL” option to snapd can’t be that difficult, and no-one is expecting Canonical to somehow support these alternative stores or provide any resources or manpower to them.


#18

That bit sounds like a reasonable suggestion, but, as ever, this being software development, could be much more difficult in practice than you’d think and not high on snappy developers’ priorities (you could try making your own PR and seeing what they make of it)…


#19

You can release any snap on any website. The user has to just download that snap and run snap install --dangerous foo.snap to install it.


#20

What if you added 3 stores and now they all contain the same package “foo” (or better than that, the core package. Adding URLs is not the problem. Figuring out sane behavior / semantics is the hard part.


#21

Obviously. But the user shouldn’t have to run “–dangerous” just to install an app. Besides any snap installed that way will not get any automatic upgrades. So this is emphatically NOT what we are talking about.


#23

Why would that be a problem?

snap install store1:foo
snap install store2:bar

Autoconnects would obviously happen only between plugs/slots from the same store.


#24

Note that this is exactly why --dangerous is called like that. Installing random software from random places is, dangerous. As is installing from random repositories. Users won’t be usually able to differentiate well-looking software from well-looking malware.


#25

And where do refreshes come from? Does that block refresh if you already have it? If I upload a new revision of a popular bitcoin wallet snap to my flashlight repository will everyone also using my flashlight snap also get the bitcoin wallet snap as a refresh? Will it have access to the data of the real wallet snap? Will it be able to make a transaction to my account from its automatically-fired configure script.

There are many reasons why this hard to get right.


#26

Each snap would always be refreshed from the store it is installed from, unless changed manually:

snap install bitcoin (default Canonical store)
...    
snap refresh flashlight:bitcoin

How to host your private store?
#27

The point is, Snap is designed in a way that adding external Stores will be hard. However, adding external Stores is (in my and jzimm’s opinion) the most important thing snap is lacking for long-term success right now.

I had this idea last night: What if…?

A. Instead of having Canonical controlling all servers that Snap uses, what if every distribution (that was interested) ran their own servers, which then synced to each other to a ‘main’ map of packages. Any distribution in the operation can accept an app to be added to the ‘main’ map. So in other words, Snap would be joint-operated by Canonical, Red Hat, and other big distributions.
B. When an app is purchased on a distribution other than Ubuntu, the company that developed that distribution gets a cut of the purchase, equal to what Ubuntu would take. In other words, whoever runs the server for the paying distribution gets the 30% cut. Users could also use to pay up to a different distribution (whoever runs the payments server the user wants to use.) If there is no payment server for the distro, they can choose who they want to give the 30% to for the payments.
C. Every distribution has a separate ‘channel’ which isn’t included in the ‘main’ packages. This is where distribution-specific snaps get shared. So, Ubuntu would ship with the ‘main’ snap repository that everyone jointly shares and a ‘ubuntu’ repository for Ubuntu-specific snaps. Red Hat would ship with ‘main’ and ‘redhat’ repositories.
D. Every distribution has their own version of build.snapcraft.io, except they all contribute (by default) to the ‘main’ pool of apps which all of the distributions mirror.


#28

You can very easily just host your additional snaps on a webserver and simply have a wrapper script caring for the “–dangerous” bits as well as upgradeability …

Since that topic got me curious i took some time in my lunchbreak today to hack up http://paste.ubuntu.com/25362069/

copy the script to ~/bin, make it executable and you can call snap-get upgrade from a crontab entry for snaps you installed with it…

Thats a simple way have additional snaps even with upgrades (without any signature or assertion checking though and the upgradeability is based on the verion field) and might help for in-house CI or whatnot, in any case it shows that there is nothing blocking to even just use a remote webserver as your snap provider if you want …

Apart from the core snap (which you could as well just mirror to that webserver via snap download) there is no need at all to use the Canonical snap store if you dont want to …

(NOTE: make sure to have htm2text and wget installed if you actually want to use it)


#29

Interesting, thank you.

" there is no need at all to use the Canonical snap store if you dont want to …"

Well, then, why doesn’t Snap support adding repositories (again)?


#30

did you read the script ? it will install everything in --dangerous mode, snaps are more than just the app content, the store gives a guarantee that they follow a certain standard (security checks on upload), the store signs them with an archive key so they can be verified (assertions) etc etc … my script can be used for a trusted entity (as i said for in-house CI where you trust the publisher) but not as an actual replacement for a fully fledged store …

if you want an actual store you need to implement the surroundings …

all i wanted was to show that you can use snaps also differently and nothing blocks you from that different thing to be just a standard webserver …


#31

Oh that, that is OK. I just meant Snap out of the box. Thank you very much for the script though! However, I will still continue pushing for Snap supporting external stores out-of-the-box, as that is still important. However, this isn’t a bad workaround. I actually had been trying to write something like this myself on GitHub.

What about the joint-operation proposal I showed a few levels up?