Post install python bytecompile?

snapcraft
1

Currently snapcraft tries to bytecompile some of the files during build (which is insufficient, because the whole dependency tree needs to be bytecompiled for things to be actually fast), this pollutes the snap and increases the size (depending on the size of the project). For crossbar snap we bytecompile the whole prime directory, which balloons the output size by 33% at least. With Python3.8 that could be avoided all together, which allows for custom location of pycache https://bugs.python.org/issue33499

I wanted to know and (potentially trigger the conversation) if there are plans to achieve something in the future i.e. bytecompile python files Post install and save the output to a writable location.

Also if there are workaround(s) today that I could use to reduce the size of the snap, that would be of great help

NOTE: bytecompile of the whole prime tree was important in our case because the software is also supposed to run on a RaspberryPi and on that environment, without bytecompile(ing) the complete tree, the startup time was 30+ seconds.

1 Like
2

you could perhaps simply remove all byte-compiled files at the prime step and do it from an install hook in SNAP_DATA if you use a python version that allows custom pycache…

note that the raspberry pi is pretty fast for an arm device, on a 400MHz single core ARM like an imx6ULL or a beaglebone this goes from 30+ seconds you mention above into the area of several minutes, which is why the change in snapcraft was made in the first place …

snapcraft should perhaps grow a snapcraft.yaml option for the python plugin that allows you to steer what it does regarding byte-compilation.

2 Likes
3

I was able to (finally) achieve that and the results are pretty.

The snap size reduced from 49MBs to 32MBs, the startup time has also reduced which could be because there is less unpacking to do on runtime and probably python 3.8 is slightly faster.

Here are the interesting snapcraft bits and the post-refresh hook.

1 Like