Problems pushing large snap to store

Finding a few different problems trying to push a large snap to the store. There are likely issues to work through on both ends, and at least on the client end for sure since it’s blowing up instead of politely describing the error.

Some data:

  • The snap is 550MB large.
  • Network connectivity is excellent (from a DC, not home)
  • In all errors, the upload reached 100%

First try:

$ snapcraft push large.snap
Pushing 'large.snap' to the store.
Uploading large.snap [====================================================] 100%
There was an error uploading the package.
Reason: 'Proxy Error'
Text: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>502 Proxy Error</title>\n</head><body>\n<h1>Proxy Error</h1>\n<p>The proxy server receive
d an invalid\r\nresponse from an upstream server.<br />\r\nThe proxy server could not handle the request <em><a href="/unscanned-upload/">POST&nbsp;/unscanned-upload/</a
></em>.<p>\nReason: <strong>Error reading from remote server</strong></p></p>\n<hr>\n<address>Apache/2.4.7 (Ubuntu) Server at upload.apps.ubuntu.com Port 443</address>\n
</body></html>\n'

Second try, with nested exceptions for extra fun:

$ snapcraft push large.snap
Pushing 'large.snap' to the store.
Uploading large.snap [==================================================] 100%
Error while processing...|                                                                                                                                               
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/snapcraft/main.py", line 253, in main
    return run(args, project_options)
  File "/usr/lib/python3/dist-packages/snapcraft/main.py", line 297, in run
    _run_store_command(args)
  File "/usr/lib/python3/dist-packages/snapcraft/main.py", line 359, in _run_store_command
    snapcraft.push(args['<snap-file>'], release_channels)
  File "/usr/lib/python3/dist-packages/snapcraft/_store.py", line 455, in push
    result = _push_snap(snap_name, snap_filename)
  File "/usr/lib/python3/dist-packages/snapcraft/_store.py", line 477, in _push_snap
    tracker.raise_for_code()
  File "/usr/lib/python3/dist-packages/snapcraft/storeapi/__init__.py", line 761, in raise_for_code
    raise errors.StoreReviewError(self.__content)
snapcraft.storeapi.errors.StoreReviewError: <exception str() failed>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/snapcraft", line 31, in <module>
    snapcraft.main.main()
  File "/usr/lib/python3/dist-packages/snapcraft/main.py", line 258, in main
    logger.error(str(e))
  File "/usr/lib/python3/dist-packages/snapcraft/internal/errors.py", line 42, in __str__
    return self.fmt.format([], **self.__dict__)
IndexError: tuple index out of range

Third try:

$ snapcraft push large.snap
Pushing 'large.snap' to the store.
Uploading large.snap [====================================================] 100%
Error while processing...|                                                                                                                                               
The store was unable to accept this snap.
  - binary_sha3_384: A file with this exact same content has already been uploaded

Looks like the second one worked after all.

Ok, so the push went fine, but the review or addition to whatever backend system blew up in a way we couldn’t parse. I’ll replicate and fix the client.

Thanks! Note that the push went fine the second time only, apparently. First one was a different error.

The first type of the problem, the Proxy Error should be fixed now. I was able to reproduce the first problem in staging and the issue was due to a mismatch in timeouts in apache for CUD and swift.

I haven’t been able to recreate any of the problems lately in staging. (after the fix landed)

No luck in reproducing the second traceback, neither before and nor after the above fix. The search on ’ <exception str() failed> type ’ does not give me much clues either. Probably a knock on one from the first error, but that’s a guess.

Thanks for addressing the first issue.

The second problem actually has two tracebacks, which most probably need addressing. If nothing else, the error handling is broken in snapcraft itself. It should never show a traceback to the user after a store review issue, and the compounded error means that logic itself is also problematic at the moment.

Thanks again for looking into this.