Help packaging swift compiler in a snap

vinay · September 15, 2017, 8:06pm

I’m attempting to create a snap for Swift. Here is my snapcraft.yaml:

name: swift # you probably want to 'snapcraft register <name>'
version: '3.1.1-1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Swift install for development
description: |
  All of the binaries necessary to use and develop with swift on Ubuntu Linux

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: devmode # use 'strict' once you have the right plugs and slots

apps:
  swift:
    command: bin/swift

parts:
  swift:
    source: https://swift.org/builds/swift-3.1.1-release/ubuntu1610/swift-3.1.1-RELEASE/swift-3.1.1-RELEASE-ubuntu16.10.tar.gz

    stage-packages:
      - clang
    plugin: dump

When I install the snap with sudo snap try --devmode prime, I get a swift command that gives me the following error:

vinay@host:~/swift$ swift -v
Swift version 3.1.1 (swift-3.1.1-RELEASE)
Target: x86_64-unknown-linux-gnu
/snap/swift/x1/bin/lldb "--repl=-disable-objc-interop -color-diagnostics"
error: failed to stop process at REPL breakpoint

However, if I run the swift executable directly, it works just fine:

vinay@host:~/swift$ ./prime/bin/swift
Welcome to Swift version 3.1.1 (swift-3.1.1-RELEASE). Type :help for assistance.
  1>

Also, if I run the command listed in the verbose output, it works:

vinay@host:~/swift$ /snap/swift/x1/bin/lldb "--repl=disable-objc-interop -color-diagnostics"
Welcome to Swift version 3.1.1 (swift-3.1.1-RELEASE). Type :help for assistance.
  1>

Wimpress · September 18, 2017, 2:28pm

Hi,

Thanks for making a start on this!

I took a quick look at made the following changes:

name: swift
version: '3.1.1'
summary: The Swift Programming Language
description: |
  Swift is a high-performance system programming language. It has a clean and
  modern syntax, offers seamless access to existing C and Objective-C code and
  frameworks, and is memory safe by default.

# TODO:
#  - Build from source http://paste.ubuntu.com/25564789/

confinement: devmode

parts:
  swift:
    plugin: dump
    source: https://swift.org/builds/swift-3.1.1-release/ubuntu1604/swift-3.1.1-RELEASE/swift-3.1.1-RELEASE-ubuntu16.04.tar.gz
    stage-packages:
      - clang
      - libblocksruntime0
      - libbsd0
      - libedit2
      - libicu-dev          # Yes, the -dev package is deliberately staged.
      - libncursesw5
      - libpython2.7
      - libsqlite3-0
      - libuutil1linux
      - libxml2
      - python2.7

apps:
  lldb:
    command: bin/lldb
    aliases: [ lldb ]
  lldb-4-0-0:
    command: bin/lldb-4.0.0
    aliases: [ lldb-4.0.0 ]
  lldb-argdumper:
    command: bin/lldb-argdumper
    aliases: [ lldb-argdumper ]
  lldb-mi:
    command: bin/lldb-mi
    aliases: [ lldb-mi ]
  lldb-mi-4-0-0:
    command: bin/lldb-mi-4.0.0
    aliases: [ lldb-mi-4.0.0 ]
  lldb-server:
    command: bin/lldb-server
    aliases: [ lldb-server ]
  lldb-server-4-0-0:
    command: bin/lldb-server-4.0.0
    aliases: [ lldb-server-4.0.0 ]
  repl-swift:
    command: bin/repl_swift
    aliases: [ repl_swift ]
  swift:
    command: bin/swift
  swift-autolink-extract:
    command: bin/swift-autolink-extract
    aliases: [ swift-autolink-extract ]
  swift-build:
    command: bin/swift-build
    aliases: [ swift-build ]
  swift-build-tool:
    command: bin/swift-build-tool
    aliases: [ swift-build-tool ]
  swift-demangle:
    command: bin/swift-demangle
    aliases: [ swift-demangle ]
  swift-package:
    command: bin/swift-package
    aliases: [ swift-package ]
  swift-test:
    command: bin/swift-test
    aliases: [ swift-test ]
  swiftc:
    command: bin/swiftc
    aliases: [ swiftc ]

However, I still see the issue you encountered and I’m unable to compile anything, when I try I get the following:

<unknown>:0: error: cannot open file '/snap/swift/x1/lib/swift/CoreFoundation/module.modulemap': Permission denied

vinay · September 21, 2017, 12:03pm

I was looking for other language compilers on Github and found a snap for the D language. It looks like they compile the compiler & frameworks from source. I wonder if that might be necessary here.

I think I might try that next.

lf-araujo · January 4, 2018, 6:23pm

This error happens whenever if you install these binaries as they recommend. One has to change the permission for the files within the lib/swift directory to be modifiable by swiftc… Hope this helps, a swift snap is sorely needed, please continue with the good work…

Thanks

vinay · January 4, 2018, 6:38pm

Thanks — I dropped this for a while, but would like to pick it back up in the new year

lf-araujo · January 9, 2018, 1:32am

Apparently I succeeded:

Find it here

I need testing before pushing to the snap store.

Thanks

lf-araujo · February 19, 2018, 9:30am

Hi,

the snap is ready and can be installed with:

sudo snap install swift-language
snap alias swift-language.swift swift

Now I noticed that the snap is not accessing system libraries. So for example, if I try to get the swift version with swift --version it works ok, but when I try to compile something it complains about not finding glibc. Take this very simple example:

git clone https://github.com/JohnSundell/Marathon.git
cd Marathon
swift build -c release

Will result in:

/snap/swift-language/1/bin/swift-build: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /snap/swift-language/1/lib/x86_64-linux-gnu/libcrypt.so.1)

But libc.so.6 is installed:

└─λ ls /lib/x86_64-linux-gnu/libc.so*
/lib/x86_64-linux-gnu/libc.so.6

How can I allow the snap to get access to the system? Should I create it with the classic flag?

@chipaca @lucyllewy

Thank you,

lf

ogra · February 19, 2018, 10:57am

you rather want ls -l /snap/core/current/lib/x86_64-linux-gnu/libc.so* the host libc is not involved at all here …

did you use snapcraft cleanbuild to create this snap ? it looks like it uses libs linked against a too new libc (2.25) … while snaps need to use 2.23 (the version from xenial) which cleanbuild ensures.

lf-araujo · February 19, 2018, 11:06am

Thanks,

└─λ ls -l /snap/core/current/lib/x86_64-linux-gnu/libc.so*
lrwxrwxrwx 1 root root 12 jan 14 23:51 /snap/core/current/lib/x86_64-linux-gnu/libc.so.6 -> libc-2.23.so

It seems I packaged an older version of libc (2.23), so perhaps I should change something in my yaml?

name: swift-language
version: '4.0.3'
summary: The Swift Programming Language
description: |
  Swift is a high-performance system programming language. It has a clean and
  modern syntax, offers seamless access to existing C and Objective-C code and
  frameworks, and is memory safe by default.

# TODO:
#  - Build from source http://paste.ubuntu.com/25564789/

confinement: strict
grade: stable

parts:
  swift:
    plugin: dump
    source: https://swift.org/builds/swift-4.0.3-release/ubuntu1610/swift-4.0.3-RELEASE/swift-4.0.3-RELEASE-ubuntu16.10.tar.gz
    stage-packages:
      - clang
      - libblocksruntime0
      - libbsd0
      - libedit2
      - libicu-dev          # Yes, the -dev package is deliberately staged.
      - libncursesw5
      - libpython2.7
      - libsqlite3-0
      - libuutil1linux
      - libxml2
      - python2.7

apps:
  lldb:
    command: bin/lldb
    aliases: [ lldb ]
  lldb-4-0-0:
    command: bin/lldb-4.0.0
    aliases: [ lldb-4.0.0 ]
  lldb-argdumper:
    command: bin/lldb-argdumper
    aliases: [ lldb-argdumper ]
  lldb-mi:
    command: bin/lldb-mi
    aliases: [ lldb-mi ]
  lldb-mi-4-0-0:
    command: bin/lldb-mi-4.0.0
    aliases: [ lldb-mi-4.0.0 ]
  lldb-server:
    command: bin/lldb-server
    aliases: [ lldb-server ]
  lldb-server-4-0-0:
    command: bin/lldb-server-4.0.0
    aliases: [ lldb-server-4.0.0 ]
  repl-swift:
    command: bin/repl_swift
    aliases: [ repl_swift ]
  swift:
    command: bin/swift
  swift-autolink-extract:
    command: bin/swift-autolink-extract
    aliases: [ swift-autolink-extract ]
  swift-build:
    command: bin/swift-build
    aliases: [ swift-build ]
  swift-build-tool:
    command: bin/swift-build-tool
    aliases: [ swift-build-tool ]
  swift-demangle:
    command: bin/swift-demangle
    aliases: [ swift-demangle ]
  swift-package:
    command: bin/swift-package
    aliases: [ swift-package ]
  swift-test:
    command: bin/swift-test
    aliases: [ swift-test ]
  swiftc:
    command: bin/swiftc
    aliases: [ swiftc ]

Best,

lf

lf-araujo · February 19, 2018, 11:07am

Or the need for 2.23 comes from snapcraft, meaning that I will not be able to package swift in a snap?

ogra · February 19, 2018, 11:18am

the need of 2.23 comes from the fact that all snaps run on top of the core snap by default, to stay binary compatible they need to use the libc shipped in core … the core snap in turn is based on 16.04, your latest source: entry in your snapcraft.yaml points to a binary built for 16.10 … try the following:

    source: https://swift.org/builds/swift-4.0.3-release/ubuntu1604/swift-4.0.3-RELEASE/swift-4.0.3-RELEASE-ubuntu16.04.tar.gz

that should have the right binaries.

lf-araujo · February 19, 2018, 12:28pm

I expected this to work, but I get a similar error. Is there a way of cleaning existing files, should I be worried that the previous revision interferes with this one?

Snapcraft run smoothly, but upon trying to run swift I get:

└─λ swift-language.swift --version
/snap/swift-language/x1/bin/swift: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /snap/swift-language/x1/lib/x86_64-linux-gnu/libuuid.so.1)

ogra · February 19, 2018, 12:30pm

did you use snapcraft cleanbuild (or build it on https://build.snapcraft.io ) ?

lf-araujo · February 19, 2018, 12:31pm

Nope, will first have to look up the cleanbuild option… (am a noob), will report back.

ogra · February 19, 2018, 12:32pm

Well, if you do not build on a 16.04 system you need to use cleanbuild, else the library versions will be wrong …

if your source is on github anyway, just letting build.snapcraft.io do its thing is quick and easy (and you can just test from the edge channel)

lf-araujo · February 19, 2018, 12:37pm

Thank you so much for letting me know about build.snapcraft.io, that is amazing!

popey · February 19, 2018, 1:30pm

Would it be worth closing the stable channel until you have this working? Given the build in stable doesn’t work.

lf-araujo · February 19, 2018, 3:00pm

yep, will do that.

best,
lf

lf-araujo · February 19, 2018, 5:17pm

It is now working in the beta channel with:

snap install swift-language --beta --devmode

I couldn’t release in stable channel due to executable stack, below is the error:

Found files with executable stack. This adds PROT_EXEC to mmap(2) during mediation which may cause security denials. Either adjust your program to not require an executable stack, strip it with 'execstack --clear-execstack ...' or remove the affected file from your snap. Affected files: usr/lib/arm-linux-gnueabihf/libBlocksRuntime.so.0.0.0 functional-snap-v2_execstack

Can someone suggest the best course of action in this case? Can I disable this file during the snap creation process?
Is it possible to run a clear-execstack within build.snapcraft.io?

ogra · February 19, 2018, 6:32pm

You can surely just add execstack to your build-packages: and call it from some scriptlet like:

execstack -c $SNAPCRAFT_PART_INSTALL/usr/lib/arm-linux-gnueabihf/libBlocksRuntime.so.0.0.0

But there might be a reason why this lib was built that way and it might cause breakage in your app in case the executable stack is really required …