Interface: unnamed sockets

tobuh · August 24, 2020, 10:54am

Hi

I have following setup in mind:
A providing and a consuming snap talking to each other via unnamed unix sockets (using “socketpair()” system call). To my understanding, these are basically file descriptors residing in /proc/$pid/fd/$fd. These file descriptors are promoted to each other by a dbus interface.

This is the setup for a TPM2 software architecture:

Provider-Snap: https://github.com/tpm2-software/tpm2-abrmd
Consumer-Snap: https://github.com/tpm2-software/tpm2-tss + https://github.com/tpm2-software/tpm2-tools

My tests are not successful. Dbus is working properly, but the socketpair is not. I tried network interfaces etc. Content interface is not possible as I don’t have socket-files available. In devmode everything is working fine.

Here is the dmesg:

[ 6384.291733] audit: type=1400 audit(1597118414.330:341): apparmor=“DENIED” operation=“file_receive” profile=“snap.tpm2-tss-engine-snap.getrandom” pid=3391 comm=“gdbus” family=“unix” sock_type=“stream” protocol=0 requested_mask=“send receive” denied_mask=“send receive” addr=none peer_addr=none peer=“snap.tpm2-abrmd-snap.tpm2-abrmd”
[ 6384.291754] audit: type=1400 audit(1597118414.330:342): apparmor=“DENIED” operation=“file_receive” profile=“snap.tpm2-abrmd-snap.tpm2-abrmd” pid=3391 comm=“gdbus” family=“unix” sock_type=“stream” protocol=0 requested_mask=“send receive” denied_mask=“send receive” addr=none peer_addr=none peer=“snap.tpm2-tss-engine-snap.getrandom”

I found this: Tpm2 deprecation

Also due to the original design of the snap, it’s not easy to use the tools and/or libraries it contains

Does that mean, this setup is generally not working? tpm2-abrmd is still recommending userspace ressource manager, though.

Main question: Are there appropriate interfaces available for unnamed unix sockets / file descriptors? Or any other means?

Feel free to ask questions if things are unclear. Thanks for your assistance!

jamesh · August 25, 2020, 2:09am

Can you post some of the code you’re using? It’s not at all clear from the links in your post.

I wouldn’t have expected any problems passing sockets as arguments in D-Bus messages.

tobuh · August 26, 2020, 6:29am

Thanks jamesh for your reply

I don’t think, it is a problem with dbus. I can see the dbus working in the logs. According to my understanding, it is a problem with the socketpair() (unnamed sockets) that is the main part of communication between “abrmd” and the consumer. This socketpair is promoted to the consumer via dbus.

This is the error I receive in the consumer:
** (process:5017): CRITICAL **: 02:42:48.649: call to CreateConnection returned a NULL GUnixFDList WARNING:tcti:src/tss2-tcti/tctildr.c:79:tcti_from_init() TCTI init for function 0x7fbe05fc98 failed with a0008

As it is working in devmode, I wonder which interfaces I have to use.

As code, I only have these snapcraft.yamls:

name: tpm2-abrmd-snap # you probably want to 'snapcraft register <name>'
base: core18 # the base snap is the execution environment for this snap
version: '0.0.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: A snap containing the tpm2-abrmd service
description: |
  TODO

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots

slots:
  tpm2-abrmd-dbus: # name that is used with 'snap connect' on slots side
    interface: dbus
    bus: system
    name: com.intel.tss2.Tabrmd

parts:
  tpm2-tools:
    plugin: autotools
    build-environment:
      - LDFLAGS: "$LDFLAGS -Wl,-rpath-link,$SNAPCRAFT_STAGE/lib"
    source: https://github.com/tpm2-software/tpm2-tools
    source-type: git
    source-branch: "master"
    build-packages:
    - pandoc
    - pkg-config
    - to arm64:
      - autoconf
      - autoconf-archive
      - libcmocka-dev
      - libcurl4-openssl-dev
      - libssl-dev
      - libtool
      - uuid-dev
    stage-packages:
    - to arm64:
      - libasn1-8-heimdal
      - libcurl4
      - libgssapi3-heimdal
      - libhcrypto4-heimdal
      - libheimbase1-heimdal
      - libheimntlm0-heimdal
      - libhx509-5-heimdal
      - libkrb5-26-heimdal
      - libldap-2.4-2
      - libnghttp2-14
      - libpsl5
      - libroken18-heimdal
      - librtmp1
      - libsasl2-2
      - libwind0-heimdal
    prime:
      - -include
    after:
      - tpm2-tss

  tpm2-tss:
    plugin: autotools
    source: https://github.com/tpm2-software/tpm2-tss
    source-type: git
    source-tag: 3.0.0
    build-packages:
    - gcc
    - g++
    - doxygen
    - pkg-config
    - to arm64:
      - autoconf
      - autoconf-archive
      - libtool
      - libltdl-dev
      - libjson-c-dev
      - libgcrypt20-dev
      - liburiparser-dev
      - libcurl4-openssl-dev
      - libssl-dev
    stage-packages:
      - to arm64:
        - libgcrypt20
        - libssl1.1
        - libgpg-error0
    prime:
      - -include

  tpm2-abrmd:
    plugin: autotools
    source: https://github.com/tpm2-software/tpm2-abrmd.git
    source-type: git
    source-tag: 2.3.3-rc0
    configflags:
      - --with-dbuspolicydir=/etc/dbus-1/system.d
    stage-packages:
      - dbus
      - libblkid1
      - libmount1
      - libpcre3
      - libselinux1
      - libuuid1
      - libglib2.0-0
      - libapparmor1
      - libaudit1
      - libgcrypt20
      - libgpg-error0
      - liblz4-1
      - liblzma5
      - libsystemd0
      - zlib1g
    build-packages:
      - to arm64:
        - dbus-x11
        - libdbus-1-dev
        - libglib2.0-dev
    after:
      - tpm2-tss

apps:
  tpm2-abrmd:
    environment:
      G_MESSAGES_DEBUG: all
    command: sbin/tpm2-abrmd --allow-root
    daemon: simple
    restart-condition: always
    passthrough:
      restart-delay: 5s
    plugs: [tpm, network-bind]
    slots: [tpm2-abrmd-dbus]

name: tpm2-tss-engine-snap # you probably want to 'snapcraft register <name>'
base: core18 # the base snap is the execution environment for this snap
version: '0.0.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: A snap containing an openssl sserver
description: |
  TODO

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots

plugs:
  tpm2-abrmd-dbus:
    interface: dbus
    bus: system
    name: com.intel.tss2.Tabrmd

parts:
  tpm2-tools:
    plugin: autotools
    build-environment:
      - LDFLAGS: "$LDFLAGS -Wl,-rpath-link,$SNAPCRAFT_STAGE/lib"
    source: https://github.com/tpm2-software/tpm2-tools
    source-type: git
    source-branch: "master"
    build-packages:
    - pandoc
    - pkg-config
    - to arm64:
      - autoconf
      - autoconf-archive
      - libcmocka-dev
      - libcurl4-openssl-dev
      - libssl-dev
      - libtool
      - uuid-dev
    stage-packages:
    - to arm64:
      - libasn1-8-heimdal
      - libcurl4
      - libgssapi3-heimdal
      - libhcrypto4-heimdal
      - libheimbase1-heimdal
      - libheimntlm0-heimdal
      - libhx509-5-heimdal
      - libkrb5-26-heimdal
      - libldap-2.4-2
      - libnghttp2-14
      - libpsl5
      - libroken18-heimdal
      - librtmp1
      - libsasl2-2
      - libwind0-heimdal
    prime:
      - -include
    after:
      - tpm2-tss

  tpm2-tss:
    plugin: autotools
    source: https://github.com/tpm2-software/tpm2-tss
    source-type: git
    source-tag: 3.0.0
    build-packages:
    - gcc
    - g++
    - doxygen
    - pkg-config
    - to arm64:
      - autoconf
      - autoconf-archive
      - libtool
      - libltdl-dev
      - libjson-c-dev
      - libgcrypt20-dev
      - liburiparser-dev
      - libcurl4-openssl-dev
      - libssl-dev
    stage-packages:
      - to arm64:
        - libgcrypt20
        - libssl1.1
        - libgpg-error0
    prime:
      - -include

  tpm2-tss-engine:
    plugin: autotools
    build-environment:
      - LDFLAGS: "$LDFLAGS -Wl,-rpath-link,$SNAPCRAFT_STAGE/lib"
    source: https://github.com/tpm2-software/tpm2-tss-engine
    source-type: git
    source-tag: "v1.1.0-rc1"
    build-packages:
      - pandoc
      - pkg-config
      - gcc
      - to arm64:
        - autoconf
        - autoconf-archive
        - libtool
        - libssl-dev
    after:
      - tpm2-tss

#only using the tcti from here...
  tpm2-abrmd:
    plugin: autotools
    source: https://github.com/tpm2-software/tpm2-abrmd.git
    source-type: git
    source-tag: 2.3.3-rc0
    configflags:
      - --with-dbuspolicydir=/etc/dbus-1/system.d
    stage-packages:
      - dbus
      - libblkid1
      - libmount1
      - libpcre3
      - libselinux1
      - libuuid1
      - libglib2.0-0
      - libapparmor1
      - libaudit1
      - libgcrypt20
      - libgpg-error0
      - liblz4-1
      - liblzma5
      - libsystemd0
      - zlib1g
    build-packages:
      - to arm64:
        - dbus-x11
        - libdbus-1-dev
        - libglib2.0-dev
    after:
      - tpm2-tss

apps:
  getrandom:
    command: bin/tpm2_getrandom
    plugs: [tpm2-abrmd-dbus, network]

jamesh · August 26, 2020, 6:43am

Can you point to the specific code where you’re attempting to pass the socket via D-Bus? I’m not going to wade through multiple unfamiliar repositories looking for it. The error message makes it seem like a possible misuse of the GLib API, but it is hard to tell without reading the code.

Also: does your code work correctly when you’re not also dealing with snap confinement?

tobuh · August 26, 2020, 8:15am

Sorry, sure …

As far as I understand the code (I’m not a dbus or gio-lib expert), these are the important lines:

Creation of socketpair:

github.com

tpm2-software/tpm2-abrmd/blob/4cdda466010a3699ebe967d990ac715ae3de7d35/src/ipc-frontend-dbus.c#L385


    g_dbus_method_invocation_return_error (
        invocation,
        TABRMD_ERROR,
        TABRMD_ERROR_ID_GENERATION,
        "Failed to allocate connection ID. Try again later.");
    return TRUE;
}
handle_map = handle_map_new (TPM2_HT_TRANSIENT, self->max_transient_objects);
if (handle_map == NULL)
    g_error ("Failed to allocate new HandleMap");
iostream = create_connection_iostream (&client_fd);
connection = connection_new (iostream, id_pid_mix, handle_map);
g_object_unref (handle_map);
g_object_unref (iostream);
if (connection == NULL)
    g_error ("Failed to allocate new connection.");
g_debug ("Created connection with client FD: %d and id: 0x%" PRIx64,
         client_fd, id_pid_mix);
/* prepare tuple variant for response message */
fd_list = g_unix_fd_list_new_from_array (&client_fd, 1);
response_variants[0] = handle_array_variant_from_fdlist (fd_list);

Few lines later, promoting the fd might be done by g_dbus_method_invocation_return_value_with_unix_fd_list()

The actual streaming seems to happen here:

github.com

tpm2-software/tpm2-abrmd/blob/4cdda466010a3699ebe967d990ac715ae3de7d35/src/util.c#L134


           size_t         count)
{
    ssize_t num_read = 0;
    size_t bytes_left = count;
    gint error_code;
    GError *error = NULL;
    g_assert (index != NULL);
    do {
        g_debug ("%s: reading %zu bytes from istream", __func__,  bytes_left);
        num_read = g_input_stream_read (istream,
                                        (gchar*)&buf [*index],
                                        bytes_left,
                                        NULL,
                                        &error);
        if (num_read > 0) {
            g_debug ("successfully read %zd bytes", num_read);
            g_debug_bytes ((uint8_t*)&buf [*index], num_read, 16, 4);
            /* Advance index by the number of bytes read. */
            *index += num_read;
            bytes_left -= num_read;

The error in the consumer happens here:

github.com

tpm2-software/tpm2-abrmd/blob/4cdda466010a3699ebe967d990ac715ae3de7d35/src/tcti-tabrmd.c#L444


static gboolean
tcti_tabrmd_call_create_connection_sync_fdlist (TctiTabrmd     *proxy,
                                                GVariant      **out_fds,
                                                guint64        *out_id,
                                                GUnixFDList   **out_fd_list,
                                                GCancellable   *cancellable,
                                                GError        **error)
{
    GVariant *_ret;
    _ret = g_dbus_proxy_call_with_unix_fd_list_sync (G_DBUS_PROXY (proxy),
        "CreateConnection",
        g_variant_new ("()"),
        G_DBUS_CALL_FLAGS_NONE,
        -1,
        NULL,
        out_fd_list,
        cancellable,
        error);
    if (_ret == NULL) {
        goto _out;

If you find it useful, here are some logs from the producer:

systemd[1]: Started Service for snap application tpm2-abrmd-snap.tpm2-abrmd.
tpm2-abrmd[4691]: tabrmd startup
tpm2-abrmd[4691]: tcti_conf before: "device:/dev/tpm0"
tpm2-abrmd[4691]: logging to stdout
tpm2-abrmd[4691]: tcti_conf after: "device:/dev/tpm0"
tpm2-abrmd[4691]: entering g_main_loop
tpm2-abrmd[4691]: init_thread_func start
tpm2-abrmd[4691]: random_class_init
tpm2-abrmd[4691]: opening entropy source: /dev/urandom
tpm2-abrmd[4691]: reading from entropy source: /dev/urandom
tpm2-abrmd[4691]: connection_manager_set_property
tpm2-abrmd[4691]:   max_connections: 27
tpm2-abrmd[4691]: IpcFrontendDbus set bus_name: com.intel.tss2.Tabrmd
tpm2-abrmd[4691]: ipc_frontend_connect
tpm2-abrmd[4691]: tcti_set_property
tpm2-abrmd[4691]: Allocating 0x1050 bytes for SAPI context
tpm2-abrmd[4691]: tpm2_set_property
tpm2-abrmd[4691]: tpm2_set_property
tpm2-abrmd[4691]: tpm2_init_tpm
tpm2-abrmd[4691]: tpm2_get_tpm_properties_fixed
tpm2-abrmd[4691]: Got proxy object for DBus daemon.
tpm2-abrmd[4691]: on_bus_acquired: com.intel.tss2.Tabrmd
tpm2-abrmd[4691]: on_name_acquired: com.intel.tss2.Tabrmd
tpm2-abrmd[4691]: command_attrs_class_init
tpm2-abrmd[4691]: command_source_class_init
tpm2-abrmd[4691]: command_source_set_property
tpm2-abrmd[4691]: command_source_set_property
tpm2-abrmd[4691]: session_list_new with max-per-connection: 0x4
tpm2-abrmd[4691]: session_list_init
tpm2-abrmd[4691]: resource_manager_set_property
tpm2-abrmd[4691]: resource_manager_set_property
tpm2-abrmd[4691]: resource_manager_set_property
tpm2-abrmd[4691]: response_sink_set_property
tpm2-abrmd[4691]:   setting PROP_IN_QUEUE
tpm2-abrmd[4691]: source_add_sink
tpm2-abrmd[4691]: command_source_add_sink
tpm2-abrmd[4691]: command_source_set_property
tpm2-abrmd[4691]: source_add_sink
tpm2-abrmd[4691]: resource_manager_add_sink
tpm2-abrmd[4691]: resource_manager_set_property
tpm2-abrmd[4691]: init_thread_func done
tpm2-abrmd[4691]: resource_manager_thread start
tpm2-abrmd[4691]: message_queue_dequeue
tpm2-abrmd[4691]: response_sink_thread: blocking on input queue
tpm2-abrmd[4691]: message_queue_dequeue
tpm2-abrmd[4691]: Creating connection with id: 0x4e364ec8bda7c030
tpm2-abrmd[4691]: handle_map_new with handle_type 0x80, max_entries: 0x1b
tpm2-abrmd[4691]: handle_map_init
tpm2-abrmd[4691]: handle_map_set_property: max-entries: 27
tpm2-abrmd[4691]: connection_set_property
tpm2-abrmd[4691]: connection_set_property: set id to 0x4e364ec8bda7c030
tpm2-abrmd[4691]: connection_set_property
tpm2-abrmd[4691]: connection_set_property: set socket
tpm2-abrmd[4691]: connection_set_property
tpm2-abrmd[4691]: connection_set_property: set transient_handle_map
tpm2-abrmd[4691]: Created connection with client FD: 10 and id: 0x4e364ec8bda7c030
tpm2-abrmd[4691]: command_source_on_new_connection: adding new connection
tpm2-abrmd[4691]: command_source_on_input_ready
tpm2-abrmd[4691]: read_data: reading 10 bytes from istream
tpm2-abrmd[4691]: read produced EOF
tpm2-abrmd[4691]: read_tpm_buffer_alloc: err_out freeing buffer
tpm2-abrmd[4691]: command_source_on_input_ready: removing connection from connection_manager
tpm2-abrmd[4691]: connection_manager_remove: removing Connection
tpm2-abrmd[4691]: sink_enqueue
tpm2-abrmd[4691]: resource_manager_enqueue
tpm2-abrmd[4691]: message_queue_enqueue
tpm2-abrmd[4691]: command_source_on_input_ready: removing GCancellable
tpm2-abrmd[4691]: resource_manager_thread: message_queue_dequeue got obj
tpm2-abrmd[4691]: resource_manager_process_control
tpm2-abrmd[4691]: resource_manager_process_control: received CONNECTION_REMOVED message for connection
tpm2-abrmd[4691]: resource_manager_remove_connection: flushing session contexts
tpm2-abrmd[4691]: resource_manager_remove_connection: done
tpm2-abrmd[4691]: sink_enqueue
tpm2-abrmd[4691]: response_sink_enqueue:
tpm2-abrmd[4691]: message_queue_enqueue
tpm2-abrmd[4691]: message_queue_dequeue
tpm2-abrmd[4691]: response_sink_process_control
tpm2-abrmd[4691]: response_sink_process_control: Received CONNECTION_REMOVED message, nothing to do.
tpm2-abrmd[4691]: handle_map_finalize
tpm2-abrmd[4691]: response_sink_thread: blocking on input queue
tpm2-abrmd[4691]: message_queue_dequeue

As already mentioned, in devmode the setup is working fine. There isalso SELinux configuration, so on other LSMs it is working properly.

github.com

tpm2-software/tpm2-abrmd/blob/master/selinux/tabrmd.te

policy_module(tabrmd, 0.0.2)

########################################
#
# Declarations
#

gen_tunable(`tabrmd_connect_all_unreserved', false)

type tabrmd_t;
type tabrmd_exec_t;
init_daemon_domain(tabrmd_t, tabrmd_exec_t)

allow tabrmd_t self:unix_dgram_socket { create_socket_perms };

dev_rw_tpm(tabrmd_t)
logging_send_syslog_msg(tabrmd_t)
sysnet_dns_name_resolve(tabrmd_t)

optional_policy(`

This file has been truncated. show original

I hope I picked the right lines. Again, I am no glib expert. I am not the creator of this code. If in doubt, tell me what you are looking for, and I can try to find it.

Thanks for your help. Very appreciated!