Package Calibre - EPUB books reader

solat · October 7, 2020, 4:22pm

Can someone please package Calibre program.

Just a short reason:
I have installed Calibre from official Ubuntu Store, but today program suddenly stopped working, it crashes at startup (I reported bug and I am waiting for bug reply). That is exactly why I hate apt packages, because some day when the software is most needed it suddenly gets broken (probably some upgrade was performed, because I regularly install Ubuntu updates). Snaps should fix this kind of dependency hell problems and I hope someone is capable of package Calibre.

popey · October 7, 2020, 6:59pm

I have looked at snapping calibre in the past. The upstream developers are very much against 3rd party packages of their software. This discourages me somewhat from publishing a snap of it.

James-Carroll · October 7, 2020, 8:06pm

I’ve had the same response in the past, but I think it’s worth mentioning just for sake of discussion that there’s some justification for going against upstreams wishes, though I’m not specifically saying do or don’t.

Calibre specifically is a GPL3 application, and that’s a choice made by the developers themselves. It seems conflicting to me they would make it to be GPL3 whilst also basically encouraging people to revoke the rights they grant for it being GPL3. Regardless, it’s legally viable.

At a guess, they don’t want people making third party packages because bug reports could end up being sent back to upstream which are introduced by the third party packages. I think this is a valid excuse ethically, it’s understandable to ask people not to give you more work to do that they’re not responsible for. I think this is a strong arguement for respecting upstreams wishes, because I’ve seen it in action, it essentially adding cost to upstream unfairly, given they ask not to.

But it’s something that needs to be weighed against the current scenario, that third party packages do exist already, including in Ubuntu’s own repositories. The question then could be viewed as, would the Snap be better than the other options available, and if so, by having more people use the Snap does it decrease the amount of issues third party packages generate overall.

If the answer is yes, then honestly I’d say that despite the upstreams wishes, it’s still a good move overall for both users and the upstream to snap it. But if the answer is no, for example, if snap creates new categories of issues that innately unfixable or this might tarnish their brand or their bug tracker, I’d be inclined to hold off snapping it too.

I think basically it’s a nuanced decision that’s worth making on an per app basis, there’s some valid justification for respecting upstreams wishes and some valid justification to ignore it, and either option is valid depending on perspectives.

Just my 0.02$

popey · October 7, 2020, 8:10pm

I basically agree with you. But just because something is technically possible to distribute, sometimes you have to take into consideration the wishes of the upstream. JWZ has had long & repeated complaints about his software (xscreensaver) being packaged in distros, and being outdated. The elementary developers have requested that their software isn’t distributed for other packaging formats or distributions for exactly the reason you specify, they don’t support it, and it places a cognitive load on them to manage reports they can’t reproduce or don’t want to fix.

solat · October 8, 2020, 1:00pm

The upstream developers are very much against 3rd party packages of their software.

I don’t know if this is really a good argument. Every Linux distribution’s package maintainer package Calibre in each of Linux distribution. All this official distribution packages are basically third party packages.

I understand that it is annoying to deal with 3rd false package creation problems. But this kind of problems could be avoided if good how to build instructions are written.

The next thing I don’t like is there concept of installing software from “sh” script from official calibre web site. It is hard to control what installation really is performing and can be dangerous to install from sh script (and how can I really be sure this is really official project web site). Users should be absolutely discouraged to install software from sh scripts. Snaps are way better in this way they are sand-boxed and installed from store and not from web site.

JWZ has had long & repeated complaints about his software (xscreensaver) being packaged in distros, and being outdated.

Isn’t there a perfect solution for this problem? Snaps updates automatically when new version is ready. Problem is immediately gone.

it places a cognitive load on them to manage reports they can’t reproduce or don’t want to fix

Isn’t this really easy? Require from bug reports to properly report a bug. Like “ubuntu-bug <program_name>” and from report should be clear if it is official software or from third-party. If it is from 3rd party, just close bug report with one sentence: “Please report bug on Ubuntu Launchpad bug repository.” or “Please report bug to package maintainer that has packaged the program.”

And the last thing is. Calibre is already available as flatpack image. We are in the market if snaps are not created and flatpacks are and users will install what is available. I prefer to have snap packages, that is why I made request for snap package.

popey · October 8, 2020, 1:20pm

Again, I agree with you. I just don’t like ruffling feathers in upstream projects.

See https://bugs.launchpad.net/calibre/+bug/1707517 and https://bugs.launchpad.net/calibre/+bug/1699620

JWZ’s complaints pre-date snaps by some considerable margin, and yes, this is partly why snaps exist.

The flatpak of Calibre is relatively popular, with around 5K users. I imagine a well maintained snap of that would be similarly appreciated. So if someone is willing to do the work to make one, and maintain it, that’d be great. I’d even help maintain it over on snapcrafters repo.

solat · October 8, 2020, 4:28pm

Those bug reports discussions are from 2017 and snap has much improved. The problem I see they are not interested in repackaging because according to message in bug report, Linux users are only 2% of Calibre. Probably this is a count downloaded from Calibre repository and not the ones downloaded from Linux distribution stores (because stores do not exchange this kind of info with developers).

Like I understand (from bug report) current tarbar file contains everything that is needed on Linux distributions. Looks similar to AppImage to me. I don’t know how difficult is it to put it inside snap container.

popey · October 8, 2020, 4:38pm

I made a snap a while back of Calibre, but never published it. It’s certainly possible. The thing about Calibre is it’s a one-of-a-kind “best in class” cross-platform application, like Google Chrome (don’t hate me), or OBS Studio. These applications get tons of installs on Windows, because they’re the best/only gig in town. As such, the Linux market share for that application is proportionally very low, because the Windows user base is so very high. It’s an application which transcends nerds (unlike OBS) to normals (like Chrome). I know very un-technical people who use Calibre, because it’s the thing everyone recommends when you have an ebook reader. Less so these days where most people just bought a kindle and don’t care about freedom though.

So yeah, if someone wants to work on it, go for it.

Reynolds5 · October 8, 2020, 7:10pm

keep in mind that you don’t have to build calibre from source to make snap - you can repackage official binaries (which is what flatpak does). That makes maintenance effort minimal and the result will have upstream support (they help fixing flatpak related issues in source code)

popey · October 15, 2020, 8:48am

Pertinent conversation:

solat · October 15, 2020, 9:19am

@popey, thanks for this info.

Just to be clear for anyone else on this forum. @popey post above is not related to any snap package, but Calibre package from official Ubuntu apt repository.

For Calibre Ubuntu 20.04 official apt repository there is currently opened a bug report and 11 duplicates, so many users are affected. Bug report is at:

The beauty of Calibre is, it is written in Python code, so very simple to fix this problem. In bug report someone prepared the Python code patch (one line of code actually is only needed) and another users created bash script to apply the fix on Ubuntu 20.04. I tested this solution and problem is fixed on my computer. But I hope the proper official fix is released for non-nerd users.

It looks to me there are several problems:

Calibre 4.99 shipped with Ubuntu 20.04 was Calibre Python3’s test bed version. This was never meant to be official version. When proper Calibre’s Pyhon3 version was released upstream it was named with version 5, but it was never imported into Ubuntu 20.04.
Ubuntu 20.04 LTS version ships beta type code.
In this case it is lack of control that upstream has changed something and didn’t fix it downstream.
One Python3 package updated in Ubuntu 20.04 and Calibre was broken.
Just guessing (speculation), the package maintainer of Calibre on Ubuntu 20.04 does not use Calibre day to day, because if he/she is, I would expect to fix this one line problem and release update very quickly.
Fix is known for few days and update was still not prepared to be shipped to Ubuntu 20.04. Lack of maintenance etc.

In general having centralized software like installing from apt can have drastic consequences if something changes. That is the reason I requested Calibre snap package, so I can have a program that is independent of host operating system changes.

ogra · October 15, 2020, 9:30am

please see:

it involves re-building the Qt toolchain for Sip5 to get the proper fix and the timing of the post 4.99 release was simply after focal, which is why 4.99 went into focal … there is obviously work going on in the above thread to fix the situation now.

memeplex · October 28, 2020, 7:33pm

I would like to package Calibre because I’ve run into many of the above problems and also knew about upstream attitude towards downstreams. It will be my first Snap package so if you can share anything I may start from, it would probably be helpful.

memeplex · October 28, 2020, 7:41pm

This exactly describes my experience and my own inferences after looking at the relevant reports and asking here and there.

gabops · October 31, 2020, 2:37am

Hello there,

After looking for the snap version of Calibre and finding out that it does not exist, I decided to take the challenge and snap it myself. I have a functional version that can be accessed here → https://github.com/gabops/calibre-snap/blob/master/snap/snapcraft.yaml. I’ve tried it locally and it seems to work perfectly fine. I am in the process of accomplishing the check list in the readme file, some of the steps are things I need to learn as is the first time I build and publish a snap so any help would be appreciated

Regards

gabops · October 31, 2020, 4:51pm

It turns out that is not gonna be easy as I though. I am struggling a little bit now working with strict mode. I am getting a ‘permission denied’ when running the app produced by a Unix socket the application tries to create when starting. it is using ‘@calibre-singleinstance-1000-GUI’ as socket name and Apparmor blocks it by default.

Oct 31 13:25:51 gpl-01 kernel: [ 5535.875825] audit: type=1400 audit(1604150751.198:5152): apparmor=“DENIED” operation=“bind” profile=“snap.calibre.calibre” pid=13630 comm=“calibre” family=“unix” sock_type=“stream” protocol=0 requested_mask=“bind” denied_mask=“bind” addr="@calibre-singleinstance-1000-GUI"

For debugging purposes I tried to create the socket by hand from a Python interpreter inside the snap with the same name and, as expected, it failed. However, If I use ‘snap.calibre.calibre’ as name it works. I’ve been digging into the Calibre code and I found where is that socket name being defined (https://github.com/kovidgoyal/calibre/blob/master/src/calibre/utils/lock.py#L142). I was hopping to find the possibility of overwritting the value via env var, for example, but sadly it is hard coded. The possibilities now are 3:

Research on how to configure the Snap plugs for allowing to bind a socket with that name. For now I’ve tried the plugs ‘network’, ‘network-bind’ and ‘network-observe’ + some hours of research on the internet with no luck.
Install Calibre using the source code (not the ‘compiled’ version I am installing) and then use gnu/patch or git to modify the function that is setting the socket name for matching with what Apparmor expects from snap. I don’t like this option as it adds complexity to the snapcraft.yml file and more stuff to maintain between versions.
Prepare a PR on the Calibre repo with a change that allows to overwrite that value. The codebase of Calibre is quite complex and doing this will take some time apart from the fact that your PR would require to be approved (which I doubt it would happen in a reasonable amount of time).

For now I am focusing on the point 1 but I ran out of ideas for the moment.

eschwartz · November 30, 2020, 4:59am

I’m very concerned by your illegitimate, inaccurate portrayal of things.

For context, I’m the person you’re quoting in those links. The links aren’t saying that calibre should not be packaged by third parties – they’re saying it’s not the job of the calibre developer to maintain a snap, and the calibre developer will never do so. This was a direct response to people informing the calibre developer that not only should he switch from a universal tarball binary installer to snap, but he should also join up as a Ubuntu / Mint developer in order to maintain the .deb, which is… a bit ridiculous.

For additional context: I’m not the calibre developer. I’m a third-party package distributor.

To be perfectly clear: third-party packages are fine, but bug reports for them should not be directed to calibre, but to Ubuntu, or snapcraft, etc. – it’s the responsibility of the third-party package distributor to determine if the bug is valid, check if newer releases fix the bug or else reproduce it on git master, and interact with upstream if necessary.

That inevitably happens. A lot. The calibre developer has a saved email template for it.

Not everyone follows the Debian/Ubuntu guidance to report bugs to the distro.

eschwartz · November 30, 2020, 5:09am

There is not, to my knowledge, any maintainer at all. There is no one to use it. It’s autosynced with tons of other applications from an unstable testing version of Debian, because that is how Ubuntu works. Hence it inherits all the bugs of an unstable testing version of Debian. Welcome to the “universe” repository.

If Ubuntu supported it, it would be in “main” and have an actual maintainer.

popey · November 30, 2020, 10:37am

The bugs I linked to didn’t ask for the Calibre developers to switch. The way I read it they asked for additional packages. In the past building packages for each distro was hard and did indeed mean someone getting involved in each distro. That could be the upstream themselves, but often could be the responsibility of enthusiastic community contributors.

The newer packaging systems also lend themselves to 3rd party community maintained packages. That way the upstream developer doesn’t have to get into the community for every minor distro.

Someone asked for a snap. You said “Also IMHO snap is still a disaster and I’m very grateful Kovid doesn’t use it.”. Kovid said “I’m afraid I have no interest in getting into the Linux packaging game.
And I dont see what’s so complex about copy-pasting a single line of code into the terminal and executing it. It’s not like the user has to type it themselves, or understand it, or anything… status wontfix”

So I stand by my assertion that the upstream developers are against 3rd party packages of their software. If that has changed, great. If not, so be it.

eschwartz · November 30, 2020, 12:39pm

Please don’t assume for a moment I’m here to be proselytized to about packaging systems. The upstream developer doesn’t have to get into the community for every minor package format like snap.

Taballs are more compatible than snap.

It is not in the best interests of the upstream developer to reduce compatibility by adding snap and dropping tarballs.
It is not in the best interests of the upstream developer to do a lot more work by adding snap and providing two formats.

Just like it is with distro packages, it is the job of Ubuntu to package things for snap, and it’s the job of Flathub to package things for flatpak.

it is the job of people who use these systems to maintain these systems.

Correct… because in my biased opinion, snap sucked then and still does now. Don’t worry – I’m also one of the lead developers of pacman, a dpkg competitor, and I think dpkg sucks too. I don’t want Kovid to switch to that either.

I’m allowed to have opinions about personal likes and dislikes.

And it is highly convenient to me and many others, that calibre does not require snap or dpkg to install a universal, distro-independent prebuilt binary.

None of this means that upstream is “hostile” to other people providing dpkg or snap third-party packages. Merely that upstream is completely uninterested in picking up that job in person.

You may do whatever you like with this information. I’m not here to be convinced by anything you have to say, whether about snap’s virtues or your analysis of upstream I’m here exclusively to correct your mistake, and, if you continue to be stubborn, to ensure other people have access to the information they need in order to realize you’re creating and perpetuating a falsehood.

I’ve said my piece.