Please address "store is not open-source" again

21

I started a series of blogposts explaining varying questions I’ve seen come up around the internet.

Let me know if I’m wrong about something or missed something. Ideas for future blogposts are also welcome.

14 Likes
22

Excellent blog posts, thanks very much !

1 Like
23

I work at a company that is likely about to embark on an Ubuntu Core deployment.

I am conflicted about this as a “person”. From a business perspective I think this is definitely the right thing for us and in general I like snaps and think they are a net-positive.

I love Ubuntu. I’ve been with it a long time. I think at times Canonical has been treated unfairly.

Having said all of that, the single-source of snaps is a major bummer for me and it keeps me, the person, from going “all-in” on them. I keep eyeing flatpaks.

Like; if I’m going to do something in my free-time (and I do a lot of that) I’m not going to invest in snap generation.

I don’t think Canonical’s past reasoning of “no contributions, etc.” is valid here. Even if you got none it’s about good faith and principles.

Plus- people run gigantic Kubernetes clusters in their houses so the notion that it’s too burdensome strikes me as insincere.

Ultimately, it’s a real identity crisis for me. Having just installed 20.04 on several of my machines it makes me question that decision.

Take all of this for what it’s worth, not much.

[my opinions are my own and do not represent anyone except me, the private person]

2 Likes
24

Having said all of that, the single-source of snaps is a major bummer for me and it keeps me, the person, from going “all-in” on them. I keep eyeing flatpaks.

…

I don’t think Canonical’s past reasoning of “no contributions, etc.” is valid here. Even if you got none it’s about good faith and principles.

I think it’s important not to conflate the “proprietary back-end” with the “single-source” complaints. In my opinion, there are good reasons to have only a single source, but the snap store should be open source.

The brand store mechanism removes most of the downsides of having a single-source design and if Canonical’s store is open source, it will be a lot easier for other people to build their own, if only by looking at how everything is implemented.

Having a proprietary back-end however, is an issue, plain and simple. I also think the reasoning behind not open sourcing it is off. I do think the effort required to open source it will pay off. Even without any contributions, having Launchpad open source was, and still is, a huge boost to Ubuntu.

As an example: given that so many Ubuntu derivatives started out on Launchpad, would the same have happened if it was closed source? Would a company like Elementary inc. bet their entire business on using a platform completely owned by a competitor?

Open Sourcing Launchpad was like adding airbags to a car. Even when they’re never used, they’re still useful. They sell more cars, they give people trust, and in the unlikely event that something bad happens, they prevent a lot of suffering.

This doesn’t stop me from contributing to the Snap ecosystem though. GitHub has significantly improved the Open Source community, even though the service is completely proprietary. Snap is already significantly improving Linux, even though it has a proprietary back-end. Flatpak is a cool project but it its scope and potential is limited. I’m convinced we need the Linux printing stack in a container and Flatpak isn’t going to give us that. I’m convinced that almost any game should not have access to my webcam, and Flatpak isn’t going to give us that either.

6 Likes
25

Sorry- I shouldn’t have referenced Flatpak specifically as I’m not at all interested in contrasting or comparing them or debating about the merits of one or the other.

What I should have just said was that my potential, personal investment of time and energy in the Snap ecosystem will be pretty limited; whereas if the source was open and one had the ability to have their own source of snaps I would have zero reason to reserve.

I run on open source systems for a reason. I learn open source systems for a reason. I refuse to put my personal time and energy into proprietary systems (to the extent I’m able now and moreso going forward).

And further, those points above make me increasingly call into question all of my future support for Ubuntu and Canonical as my home in Linux. If that could truly sink in with someone above it might mean something. But probably not.

1 Like
26

whereas if the source was open

github.com/snapcore github.com/canonical-web-and-design/snapcraft.io

Only part that is missing is server code, Amazon S3 buckets, snap signing (assertions) and database APIs. You won’t find these things open sourced in any good store, for a reason. Everything else is open source.

one had the ability to have their own source of snaps

Repositories are overrated. Even Flatpak devs have realized this and created Flathub, which is a single source of flatpaks with extra steps. :smiley: You can make your own snaps and “sideload” them (local/private/branded snaps). I would advise against installing from untrusted sources.

2 Likes
27

Please, do tell.

Even if you have a centralized location for distributing most/all apps there are plenty of use cases where that is not what you want.

I shouldn’t have gotten in this thread/debate. I was really just trying to communicate that the feel is real and it’s real from long-time lovers of Ubuntu.

28

Please don’t speculate about reasons why the store is not open source. Canonical’s official position is that the store is currently woven into their own internal infrastructure. Open Sourcing it would require a massive effort to untangle this and they don’t think it’s worth the effort.

There is already enough FUD about Canonical spreading around the internet, we don’t need to add to it.

I am not trying to debate the merits, I’m sorry if it seems that way. I’m also not trying to argue against you. I’m simply explaining why I personally still contribute to this ecosystem, even though I am uncomfortable with the proprietary back-end.

I’m sorry you feel like my comments are debating you, this is not my intention. I want to thank you for voicing your opinions here.

It might be hidden deep inside my post, but the main thing I’m trying to ask you is “do you only have an issue with the proprietary back-end or do you also have an issue with the centralized design?”. If the latter, I’m interested in knowing why this would still be an issue for you, even if the entire snap store was open source.

2 Likes
29

I think two things would satisfy me:

  1. Snapd without modification allowed you, the user/owner of the system, to provide another source of snaps, prioritizing them as providers when a conflict exists.
  2. Either the “store” source code was open source OR full documentation was released that made it a “low-effort” for someone else to implement an alternative. Low-effort in this context means no reverse-engineering required.

We know #1 is possible because alternative stores exist. My suspicion is, and they’ve more or less said as much, that there is economic incentive for them NOT to do these things.

This, for me, goes back to part of my crisis. It makes Ubuntu feel less like it felt in the past. Free forever but with all of our development effort and the availablility of the apps you’ll use on our platform coming from us.

This is also sort of an innovation barrier as well because it’s a single source of control.

The more I dwell on it, the less I become enamored with the future of Ubuntu. Which I cannot stress enough is very, very sad. I’m not a troll and I’m not a hater. Quite the contrary so it’s upsetting.

We all have to make hard choices in life; perhaps this is one. We shall see.

2 Likes
30

I think this is a serious trivialization and grouping that does you and the community a disservice. There are passionate Ubuntu users that care deeply about these things.

Maybe I’m the only one? Seems unlikely. But way to dismiss ALL of the people that love Ubuntu and deeply care about these things as well.

Wow. Just wow.

31

Please consider these thoughts:

  • What will you, personally, gain from the Store being open source or documentation made available for someone else to reimplement?
  • What will you, personally, do with that code or documentation?
3 Likes
32

As I said previously. I shouldn’t have jumped in this thread. I’m not really interested in debating my position or externally validating it, I’m not.

I have my position, what I want from a platform and if that platform doesn’t give me those things then I have to look elsewhere.

It seems like that’s the resounding, overwhelming drumbeat. Fine. I’ll deal with it and move on.

But, again – long, long time Ubuntu user and lover. It makes me sad. Cheers.

1 Like
33

I am not trying to say your opinion is wrong. Nor am I trying to argue. My aim is for those thoughts to be considered - you don’t have to tell me or the rest of the internet your reasoning. I wanted for you to introspect your own opinions and try to appreciate what it is that you personally are sad about. For example, more thoughts: are you concerned by the potential for the store to be shut down with no replacement? Or are you concerned that there is an ethical GPL/FSF-style problem? Or are you concerned on behalf of other people but don’t actually have a problem with things yourself? etc…

3 Likes
34

It’s as simple as this; if I want to distribute software (as a snap) that is signed by me and asserted to be valid by me there is currently no mechanism for someone to trust me and take the snaps.

Sure, I can give them a download and tell them to use “–dangerous”. (If I’m mistaken, please correct me).

This is creating an Android / side-loading style setup for a platform. Technically other ways to get software on your computer but made to be as unattractive as possible as to scare away 90% of the users.

I don’t think there are ethical issues. Perhaps philosophical ones. I would argue it violates the spirit of the early days of what made Ubuntu my choice of platforms.

I am sad because it no longer seems that way to me.

I think the news with Epic, Apple and Google and stores is very poignant in this discussion. Take that for what you will.

6 Likes
35

Trust. Some kind of airbag, as @futuretim said, if Canonical someday decide to (hypothetically speaking, sure) shutdown the service, for example, taking with it a lot of community effort.

Just an example.

2 Likes
36

if your snap is on ubuntu store to install on offline machine, just:

Sudo snap download yoursnap

And on your target machine
sudo snap ack yoursnap.assert
sudo snap install yoursnap.snap

No --dangerous needed

1 Like
37

@futuretim let me acknowledge the feeling you have; I think it’s completely understandable, and also quite rational. Yes, there is a risk that something wipes out Canonical, and with it perhaps your investment of time in snaps. I don’t think you’re crazy to have that feeling, in your shoes I might have the same feeling too. I want to acknowledge that this is an understandable feeling that doesn’t have anything to do with animosity towards Canonical, it’s normal even for a person who is a supporter in general of Ubuntu.

From where I am, with many years of dedication to getting things right in open source, I think we are doing the right thing in the way we are investing in snaps (despite quite a lot of social pressure to stop) and that includes the approach we have taken with the store. It’s a complicated set of trades, and I don’t have a crystal ball either, but I feel that we are on the right track. It’s a complicated decision in part because I acknowledge the uncertainty it creates in people like you, who are part of what makes Ubuntu special. Nevertheless, I know what happens if I, and my colleagues, are afraid of complicated and conflicted decisions, which is that we then fail to keep Ubuntu at the front of what’s possible, and that hurts our users more.

So when I respond to these sorts of questions, it’s not that I am dismissive of concerns, it’s rather that I have a view that on balance this is the right way forward.

I’m not going to restate all of the pros and cons here, they have mostly been covered a thousand times. Perhaps its worth stating, though, that I don’t think the snap store is any more a critical vulnerability than Canonical itself. Fact is we are a tiny company, 1/20th the size of Red Hat by revenue. I deliberately chose to build on Debian not simply because I was a DD but because I recognise the resiliency of a pure-community effort. I also recognise it’s limitations, which is why I thought the combination of Debian and Ubuntu could be a winner that moved the state of the social-technical-OS-business art forward beyond where ‘enterprise linux’ had got to in 2004.

The fact that Ubuntu has been SO widely adopted is wonderful and a validation, and also of course keeps me awake at night because it’s a big responsibility. If we make a bad mistake, or get upended, then a lot of people who have embraced Ubuntu, even if they don’t contribute with time or money, will lose a great deal even if they don’t use the snap store at all. Most of what makes Ubuntu special depends in some way on Canonical, otherwise it would have happened organically before Canonical :). That doesn’t give us the right to ‘trap’ anybody in some sort of proprietary infra, btw, and that’s not the argument I find convincing for the snap store approach we have followed, but it recognises that taking some risks of the sort that make you sincerely uncomfortable is warranted, because I know that those risks are no different than the wider story, that Canonical’s continued success, and ability to be successful with out me or its current generation of leaders, is important to the future of Ubuntu’s community and users.

I can tell you we have spent much time investing in ways to de-SPOF snaps from Canonical. Many of the capabilities for offline use of snaps are there because we share the concerns you outline, of wanting the ability to distribute software without the store. I’m OK that we haven’t gone and built ANOTHER store codebase for people who want to do that, because I believe that in open source environments genuine gaps get filled organically. I took the view that we were way ahead in seeing the need for this kind of high-security, high-reliability, high-integration app distribution capability, and we should keep going even if others would prefer we stop to neutralise a perceived competitive advantage.

Anyhow, I’ll stop there, I just wanted to acknowledge your feelings on the topic, as a supporter of Ubuntu.

14 Likes
38

I think that the combination of a pure-community upstream with a commercial downstream is also really useful in alleviating the fears of the uncertain future for contributors. You feel like the long-term value of your contribution is not completely tied to the fate of the commercial entity.

I think this is where Snap hits a sore spot and why the removal of the Chromium package sparked so many emotions. It hints at a new era where the Snap Store becomes the upstream, and thus where Canonical themselves become the upstream. The long-term value of contributions to the Chromium Snap, for example, are in question when the upstream doesn’t have the same resiliency as a massive community effort like Debian.

Yes, the process of building the snap package itself reuses large parts of the Debian effort, but I think we can all agree that Snap is the next step in an evolution that started with copying software from a magazine. Then came the tarballs, the MakeFiles, CMake, APT packages and now finally Snaps. Even though Debian packages are built with CMake and Makefiles, we still see Debian as the upstream for these packages instead of the repositories containing the MakeFiles. For this same reason, the Snap Store is the upstream for this next generation of packages. The centralized design only solidifies the idea that the Snap Store is becoming a new kind of distribution.

Of course, it’s not as simple as applying the “upstream distribution” model to the Snap store. Given that it’s designed for developers publishing their apps, we can’t apply our way of thinking about distributions one-to-one to the snap store. But I think it does indicate that this is different from other Canonical project like LXD and Mir. This is not “simply” a piece of software created by Canonical.

All this just to say; I think some serious thought should go into how to replicate the “community upstream with a strong commercial downstream” idea in Snap and the Snap Store.

1 Like
39

Yes. Our design goal was to keep the protocol between snapd and the snap store as simple as possible, which would allow for alternative stores to be written as needed. Since snapd is open source, one can decide which keys one trusts and which stores one trusts. I don’t know, over the years, how that protocol has evolved; it might be much more sophisticated now, but I suspect it’s still basically “hello, I have these versions of these snaps from these channels, what should I do”.

Most of the hard work that goes into the store is actually to support what publishers want; progressive releases so you can carefully roll out new versions, epochs so you can manage upgrades across major versions of things like application database schemas and formats, variable update rates so you can have billions of devices that don’t need hourly updates, but also have critical infrastructure that does get hourly updates, enterprise gateways that provide differential policies behind the firewall than in the world at large, etc. It’s certainly the case that a baseline ‘community upstream’ store would not depend on those.

The Debian archive format has many advantages for distributed contributors and consumers, which is why we continue to believe in it and invest in it. It also has disadvantages - just look at the pace at which deb updates actually get applied if you want to see something heartbreaking for a security patch maintainer.

The only way I can see to offer the best of both worlds is, well, to invest in both worlds :slight_smile:

4 Likes
40

Mark, thanks for the response. I must say that it’s fairly amazing that you are as active on the forums as you are. I can’t imagine what your time constraints are like.

I will be forever grateful for you and Ubuntu. I’ve been with Ubuntu as long as I can remember, certainly earlier than 8.04, so well over a decade. I was here through the haters, through Amazon on my start menu, through Mir (which I didn’t mind) and I desperately hoped the Ubuntu phone would become a thing (I’m waiting on my Librem 5 in fact).

I’m a software engineer. I love Linux and Ubuntu has always been my choice. I’ve pushed it (gently) every chance I can get. I fought our IT guy at my start-up (with the help of our Director) to insist that it made no sense to develop for Linux on Windows.

Years ago I tried to pay for Ubuntu when I downloaded it but calls to contacts provided on the form for a method other than PayPal were met with, “meh”. So I never had that chance but tried.

I have a Dell XPS 13 that shipped (2x in fact) with Ubuntu.

And as I mentioned earlier I am championing a switch from generic Ubuntu Server on our devices to Ubuntu Core.

I say all of that for the context of what I’m about to say: It is clear to me that Canonical has chosen the path of thinking they know what’s better for my computer than I do by removing some very vital choice from me (either by de facto measures or through a “not-supported by us” mechanism).

One of the things I always loved about Ubuntu (and why the haters never bothered me) is that while at times you made choices about things I disagreed with “the knobs were always there”. That’s liberating and satisfying.

By giving a user no options (other than to uninstall it? maybe?) snapd from Ubuntu I must either fork snapd to do what I want (I’m chewing on this) or find a new home.

I would do neither lightly.

And, in any event, whether I choose a new home or fork snapd it wouldn’t mean I have any ill will toward Canonical or Ubuntu. Both can exist.

Regards,

futuretim

2 Likes