Extremely slow snap downloads

I’ve just updated to ubuntu 20.04 and while installing software snaps noticed how much slower it is than the old apt-get way.

Getting about 100Kb/s.

Downloading “Geany” text editor for example:
Using apt-get: 3 seconds.
Using snap: 20 Minutes.

What can I do about this?

Is there a way to point my computer to a more local snap CDN?

Any help appreciated.
J

I am here to confirm the same scenario. Pretty slow in 20.04 even Shows it will take 20mins download VLC :-1: and 165Kbps is the maximum it went regarding download speed :confused:

Screenshot%20from%202020-04-26%2002-45-17

Downloads have been slower than usual over the last couple of days due to additional load from the release of Ubuntu 20.04 LTS. We’ve made some tweaks this morning that should have sped things up.

Downloading is very slow and I keep getting Download snap "jenkins" (1058) from channel "stable" (unexpected EOF)

today the speed is back to normal, thank you

i was installing snapd and testing using hello-world,

Download snap “core” (10126) from channel “stable”

the download speed is 680B/s -_-
the ETA is 1 day and 19 hours -__-

and I got 0 answers for

dig fastly.cdn.snapcraft.io

I understand that these are not “your” servers but this problem exists in almost every region on the planet and exists only with downloading your packages. Which means that chance of a fault in servers is a billion times the chance of a problem in our connections. Pleas rectify the error.

  • Linux Mint 19.3 From Srinagar, Kashmir.

Try switching the wifi powersave off.

Idk why it affected the wired speed too in my case but it did.

I edited /etc/NetworkManager/conf.d/default-wifi-powersave-on.conf and changed the number from 3 to 2 to disable it.

That fixed the issue.

I’m having the same issue on 150 MB ethernet, Ubuntu 20.04.02.

If it’s the same issue perhaps try applying the suggested solution. :wink:

Can you try disabling powersave? It fixed my problem even with ethernet.

image It is not my first time with this problem. I just restarted downloading and speed returned back to normal. And i saw this problem on my vds with certbot snap. Servers of course don’t use wifi and powersaving can’t harm it?

image And again, same symptoms

Same issue, very unstable download speed. The same snap can take anywhere from 3 to 15 minutes. Network and everything else is exactly the same. The only difference is maybe time of day?

Any official recommendation on how to point to a faster CDN?

1 Like

Same slow download with Ubuntu 20.04 LTS on jetson-tx1 dev board, after enable logging, I’ve got following debug log.

3月 21 15:57:21 py-jetson snapd[56365]: autorefresh.go:281: DEBUG: Next refresh scheduled for 2021-03-21T23:38:55+08:00.
3月 21 15:57:37 py-jetson snapd[56365]: daemon.go:215: polkit error: Authorization requires interaction
3月 21 15:57:37 py-jetson snapd[56365]: daemon.go:317: DEBUG: pid=56500;uid=1000;socket=/run/snapd.socket; POST /v2/snaps/cmake 11.709726ms 401
3月 21 15:57:43 py-jetson snapd[56365]: api.go:383: Installing snap "cmake" revision unset
3月 21 15:57:43 py-jetson snapd[56365]: store_action.go:441: DEBUG: Deltas enabled. Adding header Snap-Accept-Delta-Format: xdelta3
3月 21 15:57:43 py-jetson snapd[56365]: retry.go:49: DEBUG: Retrying https://api.snapcraft.io/v2/snaps/refresh, attempt 1, elapsed time=11.51µs
3月 21 15:57:43 py-jetson snapd[56365]: logger.go:67: DEBUG: > "POST /v2/snaps/refresh HTTP/1.1\r\nHost: api.snapcraft.io\r\nUser-Agent: snapd/2.49 (series 16; classic; devmode) ubuntu/20.04 (arm64) linux/4.9.201-tegra\r\nContent>
3月 21 15:57:44 py-jetson snapd[56365]: logger.go:74: DEBUG: < "HTTP/1.1 200 OK\r\nContent-Length: 3859\r\nContent-Type: application/json\r\nDate: Sun, 21 Mar 2021 07:57:44 GMT\r\nServer: gunicorn/19.7.1\r\nSnap-Store-Version: 34>
3月 21 15:57:44 py-jetson snapd[56365]: retry.go:61: DEBUG: The retry loop for https://api.snapcraft.io/v2/snaps/refresh finished after 1 retries, elapsed time=1.296890804s, status: 200
3月 21 15:57:44 py-jetson snapd[56365]: daemon.go:317: DEBUG: pid=56513;uid=0;socket=/run/snapd.socket; POST /v2/snaps/cmake 1.304347339s 400
3月 21 15:57:51 py-jetson snapd[56365]: api.go:383: Installing snap "cmake" revision unset
3月 21 15:57:51 py-jetson snapd[56365]: store_action.go:441: DEBUG: Deltas enabled. Adding header Snap-Accept-Delta-Format: xdelta3
3月 21 15:57:51 py-jetson snapd[56365]: retry.go:49: DEBUG: Retrying https://api.snapcraft.io/v2/snaps/refresh, attempt 1, elapsed time=12.813µs
3月 21 15:57:51 py-jetson snapd[56365]: logger.go:67: DEBUG: > "POST /v2/snaps/refresh HTTP/1.1\r\nHost: api.snapcraft.io\r\nUser-Agent: snapd/2.49 (series 16; classic; devmode) ubuntu/20.04 (arm64) linux/4.9.201-tegra\r\nContent>
3月 21 15:57:52 py-jetson snapd[56365]: logger.go:74: DEBUG: < "HTTP/1.1 200 OK\r\nContent-Length: 3859\r\nContent-Type: application/json\r\nDate: Sun, 21 Mar 2021 07:57:52 GMT\r\nServer: gunicorn/19.7.1\r\nSnap-Store-Version: 34>
3月 21 15:57:52 py-jetson snapd[56365]: retry.go:61: DEBUG: The retry loop for https://api.snapcraft.io/v2/snaps/refresh finished after 1 retries, elapsed time=1.164744659s, status: 200
3月 21 15:57:52 py-jetson snapd[56365]: daemon.go:317: DEBUG: pid=56584;uid=0;socket=/run/snapd.socket; POST /v2/snaps/cmake 1.205098385s 202
3月 21 15:57:52 py-jetson snapd[56365]: taskrunner.go:439: DEBUG: Running task 247 on Do: Ensure prerequisites for "cmake" are available
3月 21 15:57:52 py-jetson snapd[56365]: taskrunner.go:439: DEBUG: Running task 248 on Do: Download snap "cmake" (835) from channel "stable"
3月 21 15:57:52 py-jetson snapd[56365]: store_download.go:161: DEBUG: Available deltas returned by store: []
3月 21 15:57:52 py-jetson snapd[56365]: store_download.go:197: DEBUG: Starting download of "/var/lib/snapd/snaps/cmake_835.snap.partial".
3月 21 15:57:52 py-jetson snapd[56365]: retry.go:49: DEBUG: Retrying https://api.snapcraft.io/api/v1/snaps/download/lxhU87e4ZIqx4FU6vYgqY0oRWJ4OnxVD_835.snap, attempt 1, elapsed time=24.323µs
3月 21 15:57:52 py-jetson snapd[56365]: logger.go:67: DEBUG: > "GET /api/v1/snaps/download/lxhU87e4ZIqx4FU6vYgqY0oRWJ4OnxVD_835.snap HTTP/1.1\r\nHost: api.snapcraft.io\r\nUser-Agent: snapd/2.49 (series 16; classic; devmode) ubunt>
3月 21 15:57:54 py-jetson snapd[56365]: logger.go:74: DEBUG: < "HTTP/1.1 302 FOUND\r\nContent-Length: 587\r\nContent-Type: text/html; charset=utf-8\r\nDate: Sun, 21 Mar 2021 07:57:54 GMT\r\nLocation: https://canonical-bos01.cdn.s>
3月 21 15:57:54 py-jetson snapd[56365]: logger.go:67: DEBUG: > "GET /download-origin/canonical-lgw01/lxhU87e4ZIqx4FU6vYgqY0oRWJ4OnxVD_835.snap?interactive=1&token=1616324400_6c7f4219e83868be5f7901ea5a0e5bdad793b8f3 HTTP/1.1\r\nHo>
3月 21 15:57:55 py-jetson snapd[56365]: logger.go:74: DEBUG: < "HTTP/1.1 200 OK\r\nContent-Length: 102739968\r\nAccept-Ranges: bytes\r\nCache-Control: max-age=86400\r\nContent-Disposition: attachment; filename=\"lxhU87e4ZIqx4FU6v

Average download speed using snap

sudo snap install cmake --classic

40kbps download speed.

Normal using wget to download the direct link wget https://api.snapcraft.io/api/v1/snaps/download/lxhU87e4ZIqx4FU6vYgqY0oRWJ4OnxVD_835.snap

lxhU87e4ZIqx4FU6vYgqY0oRWJ4OnxVD_835.snap 100%[===================================================================================================================================>] 97.98M 2.76MB/s in 49s

For me the same issues. This is not a 1 time thing.
When 1 boodstrap 3 servers 1 is always fast. other 2 are slow(maybe just accidentally).
This is defently not a internet traffic thing. Cluster have 10GB WAN connections. Also snap is slow, curl is fast on the same machine.

It feels like throttling… so please canonical fix it. it must be on the client or backend site of snap. And if you are throttling let us setup mirrors. We love to host them.

Update: I found https://docs.ubuntu.com/snap-store-proxy/ so i hope this will solve my issues.

Ok snap-store-proxy is a solution, but only limeted to 5 devices :frowning: Also found the issue

canonical-bos01.cdn.snapcraftcontent.com is really slow for me… i don’t know way…

Is there a way to force only to use the other 2 cdns?

canonical-lgw01.cdn.snapcraftcontent.com canonical-lcy01.cdn.snapcraftcontent.com

I’m also getting ridiculously slow speeds over both WiFi and Ethernet (I have 100mbps fibre to my ISP). I’m seeing around 150kb/sec, sometimes dropping as low as 70kb/sec.

traceroutes:

$ traceroute canonical-lcy01.cdn.snapcraftcontent.com
traceroute: Warning: canonical-lcy01.cdn.snapcraftcontent.com has multiple addresses; using 91.189.88.179
traceroute to canonical-lcy01.cdn.snapcraftcontent.com (91.189.88.179), 64 hops max, 52 byte packets
 1  fritz.box (192.168.178.1)  2.212 ms  1.060 ms  0.956 ms
 2  * * *
 3  * * 122.56.113.5 (122.56.113.5)  15.702 ms
 4  122.56.113.4 (122.56.113.4)  15.643 ms  15.979 ms  15.970 ms
 5  ae7-2.akbr7.global-gateway.net.nz (122.56.119.53)  16.498 ms  13.488 ms  15.912 ms
 6  et1-0-0.sgbr3.global-gateway.net.nz (122.56.119.30)  39.610 ms  40.892 ms
    et8-0-0.sgbr3.global-gateway.net.nz (122.56.119.26)  39.667 ms
 7  ae2-10.sgbr4.global-gateway.net.nz (202.50.232.246)  36.587 ms  36.558 ms  40.596 ms
 8  6939.syd.equinix.com (45.127.173.24)  39.801 ms  37.976 ms  39.728 ms
 9  100ge0-36.core1.syd3.he.net (184.104.194.146)  40.526 ms  42.399 ms  38.538 ms
10  100ge0-35.core1.per1.he.net (184.104.194.142)  85.815 ms  86.491 ms  88.384 ms
11  100ge12-1.core1.sin1.he.net (184.104.194.137)  128.871 ms  133.576 ms  129.730 ms
12  100ge11-1.core1.mrs1.he.net (184.105.65.14)  278.939 ms  277.209 ms  278.056 ms
13  100ge5-2.core1.par2.he.net (184.105.81.29)  271.989 ms  276.461 ms  275.429 ms
14  100ge11-1.core1.lon2.he.net (184.105.223.253)  273.361 ms  272.965 ms  387.305 ms
15  100ge0-36.core1.lon7.he.net (184.104.196.6)  272.995 ms  277.918 ms  337.809 ms
16  canonical-group-limited.e0-53.switch1.lon7.he.net (184.104.203.50)  306.490 ms  278.417 ms  276.468 ms
17  bahamutt.canonical.com (91.189.88.179)  278.592 ms  280.050 ms  279.827 ms

and

$ traceroute canonical-bos01.cdn.snapcraftcontent.com
traceroute: Warning: canonical-bos01.cdn.snapcraftcontent.com has multiple addresses; using 91.189.91.43
traceroute to canonical-bos01.cdn.snapcraftcontent.com (91.189.91.43), 64 hops max, 52 byte packets
 1  fritz.box (192.168.178.1)  1.658 ms  1.040 ms  3.093 ms
 2  * * *
 3  * 122.56.113.5 (122.56.113.5)  13.526 ms  14.889 ms
 4  122.56.113.4 (122.56.113.4)  17.489 ms  14.439 ms  16.002 ms
 5  ae2-6.tkbr12.global-gateway.net.nz (122.56.127.17)  16.625 ms  13.609 ms
    ae7-2.akbr7.global-gateway.net.nz (122.56.119.53)  16.184 ms
 6  ae10-10.tkbr12.global-gateway.net.nz (202.50.232.29)  15.503 ms
    xe8-0-9.lebr7.global-gateway.net.nz (122.56.127.22)  138.206 ms
    ae10-10.tkbr12.global-gateway.net.nz (202.50.232.29)  13.596 ms
 7  xe-0-0-6-10.sjbr3.global-gateway.net.nz (203.96.120.194)  139.005 ms
    ae0-10.lebr8.global-gateway.net.nz (202.50.232.42)  143.736 ms
    203.96.120.142 (203.96.120.142)  139.128 ms
 8  ae3-10.lebr7.global-gateway.net.nz (122.56.127.26)  141.992 ms
    ae0-10.lebr8.global-gateway.net.nz (202.50.232.42)  139.603 ms  139.671 ms
 9  lag-14.ear2.losangeles1.level3.net (4.68.37.89)  139.297 ms  138.458 ms *
10  lag-14.ear2.losangeles1.level3.net (4.68.37.89)  171.908 ms
    ae-2-3.bear1.boston1.level3.net (4.69.159.249)  281.012 ms *
11  twdx-level3-100g.boston1.level3.net (4.53.60.66)  209.058 ms
    bbr02-et-0-0-7.bos01.twdx.net (198.160.62.0)  213.606 ms
    twdx-level3-100g.boston1.level3.net (4.53.60.66)  210.319 ms
12  bbr02-et-0-0-29.bos01.twdx.net (198.160.62.2)  215.282 ms
    bbr02-et-0-0-7.bos01.twdx.net (198.160.62.0)  241.898 ms
    bbr02-et-0-0-29.bos01.twdx.net (198.160.62.2)  213.864 ms
13  dcr03-hu-0-8-0-0.bsn04.twdx.net (198.160.62.201)  295.189 ms  209.045 ms
    bbr02-et-0-0-29.bos01.twdx.net (198.160.62.2)  215.818 ms
14  dcr03-hu-0-8-0-0.bsn04.twdx.net (198.160.62.201)  291.438 ms
    swp25.viviani.canonical.com (185.134.181.46)  306.392 ms
    swp25.lebedev.canonical.com (185.134.181.50)  306.805 ms
15  darkbowser.canonical.com (91.189.91.43)  216.258 ms  211.979 ms  212.420 ms

wget speeds for the two requested files are:

$ wget https://api.snapcraft.io/api/v1/snaps/download/99T7MUlRhtI3U0QFgl5mXXESAiSwt776_8268.snap?cdn=canonical-bos01
...
Resolving api.snapcraft.io (api.snapcraft.io)... 91.189.92.38, 91.189.92.20, 91.189.92.19, ...
...
48.2KB/s

and

wget https://api.snapcraft.io/api/v1/snaps/download/99T7MUlRhtI3U0QFgl5mXXESAiSwt776_8268.snap?cdn=canonical-lcy01
...
Resolving api.snapcraft.io (api.snapcraft.io)... 91.189.92.38, 91.189.92.20, 91.189.92.19, ...
...
7.06MB/s

That second one was much faster.

60+ minutes to install a single application like CLion is just crazy. I have three more apps like that to install - that’s 4+ hours of my time sitting around waiting.

EDIT: CLion snap has been downloading for 30+ minutes now and has an ETA of another 92 minutes…

EDIT2: Never mind, I ditched snap and used Jetbrains Toolbox to install my tools - took a total of about 4 minutes to download and install all 4 IDEs. If snap downloads remain several orders of magnitude slower than direct download how does anyone expect it to become a standard package format?

There is defiantly a problem connecting to bos01(from europe for me).
Is there a way to force to use only the 2 other cdns?

1 Like

I’m also seeing consistently slow downloads and it’s been like that for weeks, probably months now. That’s across machines, across router restarts, I think different Ubuntu versions too.

I’m now getting a steady 175kB/s - at this pace downloading Opera takes 15 minutes.

It’s gotten so bad that even though I have premade and working Ansible scripts to bootstrap and update my machines, I started rewriting them to remove all uses of snap - installing two applications with snap takes more time than the rest of the machine setup combined (!).

My ISP is UPC in Warsaw Poland and speedtest.net is reporting 400 Mbps download for me.

For anyone having experienced slow downloads in the past, our team completed a set of improvements on April 8th 2021 (particularly with the Boston endpoint) and if you are still experiencing more than very occasional poor download performance we’d like to hear the details.