In snapcraft we have a test suite that builds and runs snaps. This is one of the older test suite, and it has survived many refactors and changes in focus.
What we currently have in there is a group of demo snaps, and the abandoned tour. The suite is super slow, and it hasn’t found any useful bugs in a long time, so I want to get rid of it.
The tour was replaced by the tutorials, which should have their own tests closer to where the code for the tutorials is stored, not in snapcraft. And most of the snaps we are using as demos are not actually demonstrating useful things. If we want to demonstrate something, a better way to do it now would be to write a tutorial.
I spent an entire saturday suffering because of one of this tests, so it is now time to do something about it. I discussed a few ideas with the snapcraft team, and we agreed on having instead a suite that tests the integration between snapcraft and snapd.
The snapd integration suite will check the functioning of things that we define in the snapcraft.yaml but that are only exercised when the snap is installed or executed, like hooks, daemons, aliases. We also want to have a test that builds, installs and executes a simple hello world for each of the snapcraft plugins.
One additional detail is that we have some tests that are too slow, and will cause Travis to timeout. We want those tests to run only on autopkgtest, so we will add a flag for slow tests, to skip them in travis. They will run only once per day, when we trigger our autopkgtests.
It will take some time to fully get rid of the suite, because we have to make sure that we don’t lose coverage in tests and documentation, but we’ll get there. I have started implementing this plan, but there’s always time to make adjustments. Any comment is welcome.
pura vida