Please address "store is not open-source" again

Please don’t speculate about reasons why the store is not open source. Canonical’s official position is that the store is currently woven into their own internal infrastructure. Open Sourcing it would require a massive effort to untangle this and they don’t think it’s worth the effort.

There is already enough FUD about Canonical spreading around the internet, we don’t need to add to it.

I am not trying to debate the merits, I’m sorry if it seems that way. I’m also not trying to argue against you. I’m simply explaining why I personally still contribute to this ecosystem, even though I am uncomfortable with the proprietary back-end.

I’m sorry you feel like my comments are debating you, this is not my intention. I want to thank you for voicing your opinions here.

It might be hidden deep inside my post, but the main thing I’m trying to ask you is “do you only have an issue with the proprietary back-end or do you also have an issue with the centralized design?”. If the latter, I’m interested in knowing why this would still be an issue for you, even if the entire snap store was open source.

2 Likes

I think two things would satisfy me:

  1. Snapd without modification allowed you, the user/owner of the system, to provide another source of snaps, prioritizing them as providers when a conflict exists.
  2. Either the “store” source code was open source OR full documentation was released that made it a “low-effort” for someone else to implement an alternative. Low-effort in this context means no reverse-engineering required.

We know #1 is possible because alternative stores exist. My suspicion is, and they’ve more or less said as much, that there is economic incentive for them NOT to do these things.

This, for me, goes back to part of my crisis. It makes Ubuntu feel less like it felt in the past. Free forever but with all of our development effort and the availablility of the apps you’ll use on our platform coming from us.

This is also sort of an innovation barrier as well because it’s a single source of control.

The more I dwell on it, the less I become enamored with the future of Ubuntu. Which I cannot stress enough is very, very sad. I’m not a troll and I’m not a hater. Quite the contrary so it’s upsetting.

We all have to make hard choices in life; perhaps this is one. We shall see.

2 Likes

I think this is a serious trivialization and grouping that does you and the community a disservice. There are passionate Ubuntu users that care deeply about these things.

Maybe I’m the only one? Seems unlikely. But way to dismiss ALL of the people that love Ubuntu and deeply care about these things as well.

Wow. Just wow.

Please consider these thoughts:

  • What will you, personally, gain from the Store being open source or documentation made available for someone else to reimplement?
  • What will you, personally, do with that code or documentation?
3 Likes

As I said previously. I shouldn’t have jumped in this thread. I’m not really interested in debating my position or externally validating it, I’m not.

I have my position, what I want from a platform and if that platform doesn’t give me those things then I have to look elsewhere.

It seems like that’s the resounding, overwhelming drumbeat. Fine. I’ll deal with it and move on.

But, again – long, long time Ubuntu user and lover. It makes me sad. Cheers.

1 Like

I am not trying to say your opinion is wrong. Nor am I trying to argue. My aim is for those thoughts to be considered - you don’t have to tell me or the rest of the internet your reasoning. I wanted for you to introspect your own opinions and try to appreciate what it is that you personally are sad about. For example, more thoughts: are you concerned by the potential for the store to be shut down with no replacement? Or are you concerned that there is an ethical GPL/FSF-style problem? Or are you concerned on behalf of other people but don’t actually have a problem with things yourself? etc…

3 Likes

It’s as simple as this; if I want to distribute software (as a snap) that is signed by me and asserted to be valid by me there is currently no mechanism for someone to trust me and take the snaps.

Sure, I can give them a download and tell them to use “–dangerous”. (If I’m mistaken, please correct me).

This is creating an Android / side-loading style setup for a platform. Technically other ways to get software on your computer but made to be as unattractive as possible as to scare away 90% of the users.

I don’t think there are ethical issues. Perhaps philosophical ones. I would argue it violates the spirit of the early days of what made Ubuntu my choice of platforms.

I am sad because it no longer seems that way to me.

I think the news with Epic, Apple and Google and stores is very poignant in this discussion. Take that for what you will.

6 Likes

Trust. Some kind of airbag, as @futuretim said, if Canonical someday decide to (hypothetically speaking, sure) shutdown the service, for example, taking with it a lot of community effort.

Just an example.

2 Likes

if your snap is on ubuntu store to install on offline machine, just:

Sudo snap download yoursnap

And on your target machine
sudo snap ack yoursnap.assert
sudo snap install yoursnap.snap

No --dangerous needed

1 Like

@futuretim let me acknowledge the feeling you have; I think it’s completely understandable, and also quite rational. Yes, there is a risk that something wipes out Canonical, and with it perhaps your investment of time in snaps. I don’t think you’re crazy to have that feeling, in your shoes I might have the same feeling too. I want to acknowledge that this is an understandable feeling that doesn’t have anything to do with animosity towards Canonical, it’s normal even for a person who is a supporter in general of Ubuntu.

From where I am, with many years of dedication to getting things right in open source, I think we are doing the right thing in the way we are investing in snaps (despite quite a lot of social pressure to stop) and that includes the approach we have taken with the store. It’s a complicated set of trades, and I don’t have a crystal ball either, but I feel that we are on the right track. It’s a complicated decision in part because I acknowledge the uncertainty it creates in people like you, who are part of what makes Ubuntu special. Nevertheless, I know what happens if I, and my colleagues, are afraid of complicated and conflicted decisions, which is that we then fail to keep Ubuntu at the front of what’s possible, and that hurts our users more.

So when I respond to these sorts of questions, it’s not that I am dismissive of concerns, it’s rather that I have a view that on balance this is the right way forward.

I’m not going to restate all of the pros and cons here, they have mostly been covered a thousand times. Perhaps its worth stating, though, that I don’t think the snap store is any more a critical vulnerability than Canonical itself. Fact is we are a tiny company, 1/20th the size of Red Hat by revenue. I deliberately chose to build on Debian not simply because I was a DD but because I recognise the resiliency of a pure-community effort. I also recognise it’s limitations, which is why I thought the combination of Debian and Ubuntu could be a winner that moved the state of the social-technical-OS-business art forward beyond where ‘enterprise linux’ had got to in 2004.

The fact that Ubuntu has been SO widely adopted is wonderful and a validation, and also of course keeps me awake at night because it’s a big responsibility. If we make a bad mistake, or get upended, then a lot of people who have embraced Ubuntu, even if they don’t contribute with time or money, will lose a great deal even if they don’t use the snap store at all. Most of what makes Ubuntu special depends in some way on Canonical, otherwise it would have happened organically before Canonical :). That doesn’t give us the right to ‘trap’ anybody in some sort of proprietary infra, btw, and that’s not the argument I find convincing for the snap store approach we have followed, but it recognises that taking some risks of the sort that make you sincerely uncomfortable is warranted, because I know that those risks are no different than the wider story, that Canonical’s continued success, and ability to be successful with out me or its current generation of leaders, is important to the future of Ubuntu’s community and users.

I can tell you we have spent much time investing in ways to de-SPOF snaps from Canonical. Many of the capabilities for offline use of snaps are there because we share the concerns you outline, of wanting the ability to distribute software without the store. I’m OK that we haven’t gone and built ANOTHER store codebase for people who want to do that, because I believe that in open source environments genuine gaps get filled organically. I took the view that we were way ahead in seeing the need for this kind of high-security, high-reliability, high-integration app distribution capability, and we should keep going even if others would prefer we stop to neutralise a perceived competitive advantage.

Anyhow, I’ll stop there, I just wanted to acknowledge your feelings on the topic, as a supporter of Ubuntu.

14 Likes

I think that the combination of a pure-community upstream with a commercial downstream is also really useful in alleviating the fears of the uncertain future for contributors. You feel like the long-term value of your contribution is not completely tied to the fate of the commercial entity.

I think this is where Snap hits a sore spot and why the removal of the Chromium package sparked so many emotions. It hints at a new era where the Snap Store becomes the upstream, and thus where Canonical themselves become the upstream. The long-term value of contributions to the Chromium Snap, for example, are in question when the upstream doesn’t have the same resiliency as a massive community effort like Debian.

Yes, the process of building the snap package itself reuses large parts of the Debian effort, but I think we can all agree that Snap is the next step in an evolution that started with copying software from a magazine. Then came the tarballs, the MakeFiles, CMake, APT packages and now finally Snaps. Even though Debian packages are built with CMake and Makefiles, we still see Debian as the upstream for these packages instead of the repositories containing the MakeFiles. For this same reason, the Snap Store is the upstream for this next generation of packages. The centralized design only solidifies the idea that the Snap Store is becoming a new kind of distribution.

Of course, it’s not as simple as applying the “upstream distribution” model to the Snap store. Given that it’s designed for developers publishing their apps, we can’t apply our way of thinking about distributions one-to-one to the snap store. But I think it does indicate that this is different from other Canonical project like LXD and Mir. This is not “simply” a piece of software created by Canonical.

All this just to say; I think some serious thought should go into how to replicate the “community upstream with a strong commercial downstream” idea in Snap and the Snap Store.

1 Like

Yes. Our design goal was to keep the protocol between snapd and the snap store as simple as possible, which would allow for alternative stores to be written as needed. Since snapd is open source, one can decide which keys one trusts and which stores one trusts. I don’t know, over the years, how that protocol has evolved; it might be much more sophisticated now, but I suspect it’s still basically “hello, I have these versions of these snaps from these channels, what should I do”.

Most of the hard work that goes into the store is actually to support what publishers want; progressive releases so you can carefully roll out new versions, epochs so you can manage upgrades across major versions of things like application database schemas and formats, variable update rates so you can have billions of devices that don’t need hourly updates, but also have critical infrastructure that does get hourly updates, enterprise gateways that provide differential policies behind the firewall than in the world at large, etc. It’s certainly the case that a baseline ‘community upstream’ store would not depend on those.

The Debian archive format has many advantages for distributed contributors and consumers, which is why we continue to believe in it and invest in it. It also has disadvantages - just look at the pace at which deb updates actually get applied if you want to see something heartbreaking for a security patch maintainer.

The only way I can see to offer the best of both worlds is, well, to invest in both worlds :slight_smile:

4 Likes

Mark, thanks for the response. I must say that it’s fairly amazing that you are as active on the forums as you are. I can’t imagine what your time constraints are like.

I will be forever grateful for you and Ubuntu. I’ve been with Ubuntu as long as I can remember, certainly earlier than 8.04, so well over a decade. I was here through the haters, through Amazon on my start menu, through Mir (which I didn’t mind) and I desperately hoped the Ubuntu phone would become a thing (I’m waiting on my Librem 5 in fact).

I’m a software engineer. I love Linux and Ubuntu has always been my choice. I’ve pushed it (gently) every chance I can get. I fought our IT guy at my start-up (with the help of our Director) to insist that it made no sense to develop for Linux on Windows.

Years ago I tried to pay for Ubuntu when I downloaded it but calls to contacts provided on the form for a method other than PayPal were met with, “meh”. So I never had that chance but tried.

I have a Dell XPS 13 that shipped (2x in fact) with Ubuntu.

And as I mentioned earlier I am championing a switch from generic Ubuntu Server on our devices to Ubuntu Core.

I say all of that for the context of what I’m about to say: It is clear to me that Canonical has chosen the path of thinking they know what’s better for my computer than I do by removing some very vital choice from me (either by de facto measures or through a “not-supported by us” mechanism).

One of the things I always loved about Ubuntu (and why the haters never bothered me) is that while at times you made choices about things I disagreed with “the knobs were always there”. That’s liberating and satisfying.

By giving a user no options (other than to uninstall it? maybe?) snapd from Ubuntu I must either fork snapd to do what I want (I’m chewing on this) or find a new home.

I would do neither lightly.

And, in any event, whether I choose a new home or fork snapd it wouldn’t mean I have any ill will toward Canonical or Ubuntu. Both can exist.

Regards,

futuretim

2 Likes

Sorry, this discussion is basically about not using the Snap Store or having an alternative to. So the setup of the discussion would include the snap you are trying to distribute is not on the Snap Store.

1 Like

It’s very difficult to respond to the ‘know whats better on my machine than I do’ argument, because making choices is basically why we exist.

One of the hallmarks of the move to free software is precisely the ability to do whatever you want on your own system. Linux from scratch is a wonderful exercise in exactly that. Arch, and before it Gentoo, are very compelling variants on that theme. I’m glad all of that exists.

A long time ago I was able to use Linux that way to do some pretty good stuff in the world. I was grateful. But I was also mindful of the fact that, bluntly, it was too hard. I wanted more entrepreneurs to be able to build things on open source without having to figure out as much of the detail. And the way to achieve that was to take responsibility for the detail so that they don’t have to. I’m always entertained by the ‘Ubuntu is for newbies’ slur, because it sort of misses the beauty of the work we do. We’re there for newbies, and we’re also there for people who have figured out what they don’t need to figure out. Want to recompile the kernel? Sure you can, but I can tell you that 99.9999% of people don’t after a while, because it’s not much value add.

This is no different. Really, it isn’t.

Think about it. Say you sit down to create a new store. As stated above, that isn’t a huge amount of work. But now you have an interesting dilemma for your users. That store is full of binary software that you didn’t compile. Are you going to take responsibility for that? Are you going to scan it and commit to removing anything that causes a problem? Or are you going to say to your users ‘OK, I’m giving you the ability to stick binaries on your system but you need to decide if you can trust them’.

We know the latter doesn’t work. If you say to people ‘hey now there are hundreds of repos out there you should decide which ones you trust’ you are walking them into a cesspit of obscure and impossible decisions. And if you step up to do the work of taking responsibility, then you are asking people to trust you. Which makes you someone who ‘knows more than they do about the trust on their computer’.

It’s quite the dilemma.

I have opinions about lots of things I would like to see done differently in the world. But I’ve learned that the only things that matter are the ones I am willing to commit to doing something about, otherwise its hot air and trolling. Actually doing something about free software is a life commitment and there are quite a few people who have done that, and taken different positions in doing that, which is great. Naturally, perspectives will vary. I’m nevertheless proud to be focused on enabling publishers to reach Linux users, with a minimum of friction and security risks in bringing those two together, and I simply don’t know how to do that without taking some responsibility for the bit in the middle.

It isn’t interesting to recreate debs. We already have debs, with their strengths and weaknesses. Just recreating that is not something I would spend time on, or ask anybody else to spend time on.

It is interesting to try to build something new, which is complimentary to debs. This is our attempt to do that. I’m glad we have competition because we might not build the best widget. I doubt that competition would be around if we hadn’t set out to build this widget. So let’s see :slight_smile:

16 Likes

Thank you for sharing your valuable thoughts on the issue Mark.

:clap:t4:

1 Like

They say, “Turn the blind eye, and people will shutoff about it eventually.” but when it comes to opensource community-driven by Linux, people won’t shut off about it. NEVER.

They will have to move to another nice option. :wink: And by that time, it will be late to ‘open up’ for some people & realize.

After all, this is all driven by what discussed in forums, social platforms, community pages, wiki, etc.

An open store exists on GitHub but doesn’t yet work, if anyone really believes we need an open snap store please contribute to it (I would need to learn to program first and am clearly not motivated enough!): https://github.com/gjsman/opensnap (thanks @G.S.1)

Thanks Mark for addressing some of these controversial topics even though I know it takes time.

(Apologies for bumping this thread)

2 Likes

A separate open-source implementation of a snap store has been started. You can read more details about it in this post:

https://forum.snapcraft.io/t/announcing-project-kebe-open-source-snap-store-start/25088/3

4 Likes

Any updates on this? I really want to see this being open sourced