Snaps download should be resumable

For big snaps it often becomes a challenge for me, especially when the download is interrupted due to some reason. In this specific case, I had to re-download Android Studio because at 90% it failed with

error: cannot perform the following tasks:
- Download snap "android-studio" (28) from channel "candidate" (received an unexpected http response code (504) when trying to download https://068ed04f23.site.inte
rnapcdn.net/download-snap/VNTeIi51l6cIZ1gaWHwx0PoP7JaCb61c_28.snap?t=2018-01-17T23:00:00Z&h=898a8cc26b1fca0688a07ce34e39989f3c417314)

So I had to redownload the whole thing. I just got an email from one of my friends who had that issue twice, so he will be downloading the snap the third time now.

  • Please fix the downloading, that timeout feels like a bug in the store to me.
  • Snaps download should be resumable.
2 Likes

They should resume, and if they don’t then it’s a bug in snapd or the store (or CDN).

I just started installing a big snap such as yours and then tried to simulate all kinds of network issues like pulling the cable out and CTRL+Cing the snap process (and killing it). I still end up with a .partial file in /var/lib/snapd/snaps

-rw-r--r--  1 root root 382281818 Jan 17 23:10 android-studio_28.snap.partial

As soon as I reconnected to the network, snapd moved forwards with the download and finished it off without any interaction from me.

I assume that’s the way it’s designed to work, and if it doesn’t then I’d file a bug.

3 Likes

It could be that the specific case (see log) I mentioned above is not correctly handled by snapd, so it ends up discarding the already downloaded data.

Will raise a bug on snapd.

When downloads fails I’ve never seen my downloads resume it always starts from the beginning

Hello everyone,

I’m also having the same problem. My system is Debian Buster.

Today I installed snapd using terminal.

sudo apt install snapd

Then I tried to install blender --classic.

sudo snap install blender --classic

After some time, it stopped downloading. So then I again tried to install it. It completely restarts the download. And it stopped the download again after some time.

I thought this maybe a problem with snapd on CLI. So I tried to install,

sudo snap install snap-store

And same thing happen with this. Download stopped after some time and when I try to reinstall it, it starts from the beginning.

As @popey said, I looked into the /var/lib/snapd/snaps

When downloading, there’s a file, after I abort it or automatically canceled, that fill be deleted automatically.

This may sound crazy, but I really did this… :smiley: I copied that file to a different location while installing, after installation canceled(around 2% and 3%), I copy it back to that (/var/lib/snapd/snaps) folder and try to download it again. File size is increasing, but… downloading start from 0%… I did this several times and results are always the same. Final file size was about 4MB. But every time I copy it to that folder and restart the download, it starts from 0%…

Blender --classic file size is about 123MB

If anyone can help me through this, I really appreciate it.

Thank you.

It would be good to know why your download is halting like that.
snapd does retry several times, if left alone, and will not remove the file until it gives up (if you enable debug logging you can see that going on). And it is possible to ‘seed’ the partial files for the downloading to continue, we do this in some integration tests, but it can be finicky.
The easiest way for you to move ahead without having to worry about this is to do a snap download, and then snap ack and snap install (the output of snap download will tell you what to do exactly). snap download will leave a partial file behind when interrupted with Ctrl+C, and should be able to resume things just fine.

Thanks for your reply @chipaca

But it didn’t worked for me. Download also canceled as I’m having problem with installing. Download file isn’t available and download restarted.

could you explain that one to me? show me the logs?

Hello @chipaca

Can you please guide me to get a log file? I don’t know how can I do that.

Show me the terminal output when you do snap download blender, and how you determine the download file isn’t available and download restarted.

We can make it more verbose later.

Thanks @chipaca

This is what happen when I try to install,

joseph@joseph:~$ sudo snap install blender --classic
error: cannot perform the following tasks:
- Download snap "core" (7396) from channel "stable" (read tcp 10.102.16.91:46902->151.101.26.217:443: read: connection reset by peer)

Installation automatically canceled at 2%

This is what happen when I try to download,

joseph@joseph:~$ sudo snap download blender
Fetching snap "blender"
error: read tcp 10.102.16.91:54256->151.101.42.217:443: read: connection reset by peer

Download automatically canceled at 2%

Every time I try to reinstall or re-download the file, it start at 0%…

After the download bails, what’s the output of ls -l blender*?

I’m sorry, but how can I type that command? I typed it as you said, But I can’t find any results from that…

joseph@joseph:~$ ls -l blender*
ls: cannot access 'blender*': No such file or directory

That’s fine.

What happens if you do snap download without adding a sudo in front of it?

In any case, the sudo shouldn’t make a difference (beyond leaving the file with the wrong owner). But, the fact that you don’t get a .partial file means the download isn’t even able to start. Let’s turn on debug logs now. Try this:

SNAPD_DEBUG=1 SNAPD_DEBUG_HTTP=7 snap download blender
joseph@joseph:~$ snap download blender
Fetching snap "blender"
error: read tcp 10.102.16.91:60812->151.101.54.217:443: read: connection reset by peer

Result was the same…

I’ll try this now… :slight_smile:

joseph@joseph:~$ SNAPD_DEBUG=1 SNAPD_DEBUG_HTTP=7 snap download blender
Fetching snap "blender"
2019/08/21 19:06:13.294698 helpers.go:224: DEBUG: Going to download snap "blender" (unset) from channel "" to "".
2019/08/21 19:06:13.295022 retry.go:41: DEBUG: Retrying https://api.snapcraft.io/v2/snaps/refresh, attempt 1, elapsed time=6.43µs
2019/08/21 19:06:13.295413 logger.go:67: DEBUG: > "POST /v2/snaps/refresh HTTP/1.1\r\nHost: api.snapcraft.io\r\nUser-Agent: snapd/2.37.4-1+b1 (series 16; classic; devmode) snap debian/10 (amd64) linux/4.19.0-5-amd64\r\nContent-Length: 347\r\nAccept: application/json\r\nContent-Type: application/json\r\nSnap-Classic: true\r\nSnap-Device-Architecture: amd64\r\nSnap-Device-Series: 16\r\nAccept-Encoding: gzip\r\n\r\n{\"context\":[],\"actions\":[{\"action\":\"download\",\"instance-key\":\"download-1\",\"name\":\"blender\",\"epoch\":null}],\"fields\":[\"architectures\",\"base\",\"confinement\",\"contact\",\"created-at\",\"description\",\"download\",\"epoch\",\"license\",\"name\",\"prices\",\"private\",\"publisher\",\"revision\",\"snap-id\",\"snap-yaml\",\"summary\",\"title\",\"type\",\"version\",\"media\",\"common-ids\"]}"
2019/08/21 19:06:14.857278 logger.go:74: DEBUG: < "HTTP/1.1 200 OK\r\nContent-Length: 3169\r\nContent-Type: application/json\r\nDate: Wed, 21 Aug 2019 13:36:14 GMT\r\nServer: gunicorn/19.7.1\r\nSnap-Store-Version: 15\r\nX-Request-Id: 6715A552C2700A325C8501BB5D5D48CD151B8B5B\r\nX-Vcs-Revision: 57828e7\r\nX-View-Name: snapdevicegw.webapi_refresh.snap_refresh\r\n\r\n{\"error-list\":[],\"results\":[{\"effective-channel\":\"stable\",\"instance-key\":\"download-1\",\"name\":\"blender\",\"released-at\":\"2019-07-30T14:28:55.511384+00:00\",\"result\":\"download\",\"snap\":{\"architectures\":[\"amd64\"],\"base\":null,\"common-ids\":[],\"confinement\":\"classic\",\"contact\":\"\",\"created-at\":\"2019-07-30T14:22:35.038569+00:00\",\"description\":\"Blender is the free and open source 3D creation suite. It supports the\\nentirety of the 3D pipeline\\u2014modeling, rigging, animation, simulation,\\nrendering, compositing and motion tracking, even video editing and game\\ncreation.\\n\\nBlender is a public project, made by hundreds of people from around the\\nworld; by studios and individual artists, professionals and hobbyists,\\nscientists, students, VFX experts, animators, game artists, modders, and\\nthe list goes on.\",\"download\":{\"deltas\":[],\"sha3-384\":\"9c4bee58188d00f9443aee893dad6d669c44c89431683c880b6ed6195e784f7013c0fc696c63f8b2366814f13ad0c621\",\"size\":123510784,\"url\":\"https://api.snapcraft.io/api/v1/snaps/download/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap\"},\"epoch\":{\"read\":[0],\"write\":[0]},\"license\":\"GPL-3.0\",\"media\":[{\"height\":null,\"type\":\"icon\",\"url\":\"https://dashboard.snapcraft.io/site_media/appmedia/2019/07/blender.png\",\"width\":null},{\"height\":null,\"type\":\"screenshot\",\"url\":\"https://dashboard.snapcraft.io/site_media/appmedia/2019/07/blender_screenshot_1.jpg\",\"width\":null},{\"height\":null,\"type\":\"screenshot\",\"url\":\"https://dashboard.snapcraft.io/site_media/appmedia/2019/07/blender_screenshot_2.jpg\",\"width\":null},{\"height\":null,\"type\":\"screenshot\",\"url\":\"https://dashboard.snapcraft.io/site_media/appmedia/2019/07/blender_screenshot_3.jpg\",\"width\":null},{\"height\":null,\"type\":\"screenshot\",\"url\":\"https://dashboard.snapcraft.io/site_media/appmedia/2019/07/blender_screenshot_4.jpg\",\"width\":null}],\"name\":\"blender\",\"prices\":{},\"private\":false,\"publisher\":{\"display-name\":\"Blender Foundation\",\"id\":\"IEzHQPsJJcDJXVbfCdg5mh4cfFs7Xmhq\",\"username\":\"blenderfoundation\",\"validation\":\"verified\"},\"revision\":33,\"snap-id\":\"ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz\",\"snap-yaml\":\"name: blender\\nversion: '2.80'\\nsummary: Blender is the free and open source 3D creation suite.\\ndescription: \\\"Blender is the free and open source 3D creation suite. It supports the\\\\n\\\\\\n  entirety of the 3D pipeline\\\\u2014modeling, rigging, animation, simulation,\\\\nrendering,\\\\\\n  \\\\ compositing and motion tracking, and video editing.\\\\n\\\\nBlender is a public project,\\\\\\n  \\\\ made by hundreds of people from around the\\\\nworld; by studios and individual artists,\\\\\\n  \\\\ professionals and hobbyists,\\\\nscientists, students, VFX experts, animators, game\\\\\\n  \\\\ artists, modders, and\\\\nthe list goes on.\\\\n\\\\nThe standard snap channels are used\\\\\\n  \\\\ in the following way:\\\\n\\\\n       stable - Latest stable release.\\\\n    candidate\\\\\\n  \\\\ - Test builds for the upcoming stable release.\\\\n\\\"\\narchitectures:\\n- amd64\\nconfinement: classic\\ngrade: stable\\napps:\\n  blender:\\n    command: command-blender.wrapper\\nlicense: GPL-3.0\\n\",\"summary\":\"Blender is the free and open source 3D creation suite.\",\"title\":\"Blender\",\"type\":\"app\",\"version\":\"2.80\"},\"snap-id\":\"ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz\"}]}\n"
2019/08/21 19:06:14.857622 retry.go:53: DEBUG: The retry loop for https://api.snapcraft.io/v2/snaps/refresh finished after 1 retries, elapsed time=1.56260957s, status: 200
2019/08/21 19:06:14.858057 store.go:1352: DEBUG: Starting download of "/home/joseph/blender_33.snap.partial".
2019/08/21 19:06:14.858757 retry.go:41: DEBUG: Retrying https://api.snapcraft.io/api/v1/snaps/download/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap, attempt 1, elapsed time=17.679µs
2019/08/21 19:06:14.859114 logger.go:67: DEBUG: > "GET /api/v1/snaps/download/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap HTTP/1.1\r\nHost: api.snapcraft.io\r\nUser-Agent: snapd/2.37.4-1+b1 (series 16; classic; devmode) snap debian/10 (amd64) linux/4.19.0-5-amd64\r\nAccept: \r\nX-Ubuntu-Architecture: amd64\r\nX-Ubuntu-Classic: true\r\nX-Ubuntu-Series: 16\r\nX-Ubuntu-Wire-Protocol: 1\r\nAccept-Encoding: gzip\r\n\r\n"
2019/08/21 19:06:15.756700 logger.go:74: DEBUG: < "HTTP/1.1 302 FOUND\r\nContent-Length: 513\r\nContent-Type: text/html; charset=utf-8\r\nDate: Wed, 21 Aug 2019 13:36:15 GMT\r\nLocation: https://fastly.cdn.snapcraft.io/download-origin/fastly/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap?token=1566406800_06c56fca6b5f0648018fb312bdf89bb473568c27\r\nServer: gunicorn/19.7.1\r\nSnap-Store-Version: 15\r\nX-Request-Id: 6715A548C6B90A325C9601BB5D5D48CF14FCA6B9\r\nX-Vcs-Revision: 57828e7\r\nX-View-Name: snapdevicegw.webapi_download.snap_download\r\n\r\n"
2019/08/21 19:06:15.757073 logger.go:67: DEBUG: > "GET /download-origin/fastly/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap?token=1566406800_06c56fca6b5f0648018fb312bdf89bb473568c27 HTTP/1.1\r\nHost: fastly.cdn.snapcraft.io\r\nUser-Agent: snapd/2.37.4-1+b1 (series 16; classic; devmode) snap debian/10 (amd64) linux/4.19.0-5-amd64\r\nAccept: \r\nReferer: https://api.snapcraft.io/api/v1/snaps/download/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap\r\nX-Ubuntu-Architecture: amd64\r\nX-Ubuntu-Classic: true\r\nX-Ubuntu-Series: 16\r\nX-Ubuntu-Wire-Protocol: 1\r\nAccept-Encoding: gzip\r\n\r\n"
2019/08/21 19:06:17.467005 logger.go:74: DEBUG: < "HTTP/2.0 200 OK\r\nContent-Length: 123510784\r\nAccept-Ranges: bytes\r\nAge: 603\r\nCache-Control: max-age=86400\r\nContent-Disposition: attachment; filename=ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap\r\nContent-Type: application/octet-stream\r\nDate: Wed, 21 Aug 2019 13:36:17 GMT\r\nEtag: f4e2fefe-dc63-4e52-85a5-3daa5eca1289\r\nLast-Modified: Wed, 21 Aug 2019 12:57:57 GMT\r\nServer: TwistedWeb/14.0.2\r\nStrict-Transport-Security: max-age=2592000\r\nVary: Accept-Encoding\r\nVia: 1.1 varnish\r\nVia: 1.1 varnish\r\nX-Bzr-Revision-Number: 7464\r\nX-Cache: HIT, HIT\r\nX-Cache-Hits: 0, 1\r\nX-Request-Id: XV0-1X8AAQEAAHhvAeQAAAAW1\r\nX-Served-By: cache-lcy19235-LCY, cache-sea1034-SEA\r\nX-Timer: S1566394577.335737,VS0,VE1\r\n\r\n"
2019/08/21 19:08:45.706319 retry.go:83: DEBUG: Retrying because of: read tcp 10.102.16.91:60852->151.101.54.217:443: read: connection reset by peer
2019/08/21 19:08:45.706378 retry.go:41: DEBUG: Retrying https://api.snapcraft.io/api/v1/snaps/download/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap, attempt 2, elapsed time=2m30.847641533s
2019/08/21 19:08:45.715890 logger.go:67: DEBUG: > "GET /api/v1/snaps/download/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap HTTP/1.1\r\nHost: api.snapcraft.io\r\nUser-Agent: snapd/2.37.4-1+b1 (series 16; classic; devmode) snap debian/10 (amd64) linux/4.19.0-5-amd64\r\nAccept: \r\nRange: bytes=819393-\r\nX-Ubuntu-Architecture: amd64\r\nX-Ubuntu-Classic: true\r\nX-Ubuntu-Series: 16\r\nX-Ubuntu-Wire-Protocol: 1\r\n\r\n"
2019/08/21 19:08:46.856673 logger.go:74: DEBUG: < "HTTP/1.1 302 FOUND\r\nContent-Length: 513\r\nContent-Type: text/html; charset=utf-8\r\nDate: Wed, 21 Aug 2019 13:38:46 GMT\r\nLocation: https://fastly.cdn.snapcraft.io/download-origin/fastly/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap?token=1566406800_06c56fca6b5f0648018fb312bdf89bb473568c27\r\nServer: gunicorn/19.7.1\r\nSnap-Store-Version: 15\r\nX-Request-Id: 6715A557BB390A325CD101BB5D5D496615288E69\r\nX-Vcs-Revision: 57828e7\r\nX-View-Name: snapdevicegw.webapi_download.snap_download\r\n\r\n"
2019/08/21 19:08:46.857046 logger.go:67: DEBUG: > "GET /download-origin/fastly/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap?token=1566406800_06c56fca6b5f0648018fb312bdf89bb473568c27 HTTP/1.1\r\nHost: fastly.cdn.snapcraft.io\r\nUser-Agent: snapd/2.37.4-1+b1 (series 16; classic; devmode) snap debian/10 (amd64) linux/4.19.0-5-amd64\r\nAccept: \r\nRange: bytes=819393-\r\nReferer: https://api.snapcraft.io/api/v1/snaps/download/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap\r\nX-Ubuntu-Architecture: amd64\r\nX-Ubuntu-Classic: true\r\nX-Ubuntu-Series: 16\r\nX-Ubuntu-Wire-Protocol: 1\r\n\r\n"
2019/08/21 19:08:52.736703 logger.go:74: DEBUG: < "HTTP/2.0 206 Partial Content\r\nContent-Length: 122691391\r\nAccept-Ranges: bytes\r\nAccept-Ranges: bytes\r\nAge: 0\r\nCache-Control: max-age=86400\r\nContent-Disposition: attachment; filename=ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap\r\nContent-Range: bytes 819393-123510783/123510784\r\nContent-Type: application/octet-stream\r\nDate: Wed, 21 Aug 2019 13:38:48 GMT\r\nEtag: f4e2fefe-dc63-4e52-85a5-3daa5eca1289\r\nLast-Modified: Wed, 21 Aug 2019 13:11:39 GMT\r\nServer: TwistedWeb/14.0.2\r\nStrict-Transport-Security: max-age=2592000\r\nVary: Accept-Encoding\r\nVia: 1.1 varnish\r\nVia: 1.1 varnish\r\nX-Bzr-Revision-Number: 7464\r\nX-Cache: MISS, MISS\r\nX-Cache-Hits: 0, 0\r\nX-Request-Id: XV1DC38AAQEAAHohEwMAAABE1\r\nX-Served-By: cache-lcy19244-LCY, cache-lax8639-LAX\r\nX-Timer: S1566394728.047012,VS0,VE329\r\n\r\n"
2019/08/21 19:11:01.227418 store.go:1368: DEBUG: download of "https://api.snapcraft.io/api/v1/snaps/download/ZWyHJJiPBBfMKpaaKKrpSgAcupWkHWPz_33.snap" failed: &net.OpError{Op:"read", Net:"tcp", Source:(*net.TCPAddr)(0xc000234ba0), Addr:(*net.TCPAddr)(0xc000234c00), Err:(*os.SyscallError)(0xc00045e080)}
error: read tcp 10.102.16.91:47204->151.101.26.217:443: read: connection reset by peer

I looked at the download file… It disappeared right after download stopped.

I see the bug now, and yes it’s a bug; I’ll fix it. I’d only tested this with errors that also killed the download op… :expressionless:

In the meantime, you could use curl or wget to download that URL (the one that starts https://api.snapcraft.io/api/v1/snaps/download/), rename it blender_33.snap, and run snap download blender (so it checks the hash, and it gets the .assert file). Sorry for the inconvenience.

Do note, however, that there’s something wonky with your network :slight_smile:

1 Like

I feel sorry for the bug… But I’m happy you found it… :smiley:

I’ll do this… :slight_smile:

Yeah I know that… That’s why this resuming is so much important to me… :wink:

Thank you so much for helping me @chipaca

1 Like

Hello @chipaca Problem continues… :open_mouth: I did a little research before ask this and didn’t found any solution.

Today I downloaded blender_33.snap with wget. Then,

  joseph@joseph:~$ snap download blender
Fetching snap "blender"
Fetching assertions for "blender"
Install the snap with:
   snap ack blender_33.assert
   snap install blender_33.snap

then,

joseph@joseph:~$ sudo snap ack blender_33.assert

and then,

joseph@joseph:~$ sudo snap install blender_33.snap
error: This revision of snap "blender_33.snap" was published using classic confinement and thus may
       perform arbitrary system changes outside of the security sandbox that snaps are usually
       confined to, which may put your system at risk.

       If you understand and want to proceed repeat the command including --classic.

so then,

joseph@joseph:~$ sudo snap install blender_33.snap --classic
error: cannot perform the following tasks:
- Download snap "core" (7396) from channel "stable" (read tcp 10.102.132.145:39778->151.101.42.217:443: read: connection reset by peer)

I’m stuck again… :sob: