Classic confinement for fluent-bit

store-requests
#1

Fluent Bit is a log processor that can be configured to retrieve messages following content of log files, through TCP, UDP, Systemd, and many other sources.

Classic confinement is required to access arbitrary file system paths and any TCP port for data collection.

How to configure fluent-bit on ubuntu-core
#2

the log-observe interface gives you access to everything in /var/log as well as journald, the network, network-bind and network-control interfaces allow retrieving/sending logs via TCP and UDP … here is an example of a log processing tool:

if you need access to specific logs that are in non-standard locations you can use the system-files interface to allow access to them …

1 Like
#3

Hey @edsiper,

If you read our Process for reviewing classic confinement snaps, access to arbitrary files on the system isn’t typically a justification for classic.

The interfaces suggested by @ogra should help with your needs. Otherwise, could you please post here the locations of the files Fluent bit needs to access? If you still experience issues, you can try snappy-debug while troubleshooting. It will alert you of any denials and recommend missing interfaces. Feel free to post any further question and we are happy to help.

#4

Hi,

thanks for the update.

I gave it a try to log-observe and the other network-* stuff but still I find it’s not a straightforward solution for the end-user, we allow to process many sources of data and paths per user’s need.

If someone want’s to use Fluent Bit to receive, process and ship logs, having to connect an interface manually adds an extra unnecessary step. Fluent Bit is a tool that can run as a service and also as a standalone utility, our goal as maintainers to provide a Snap for it is to make easier it deployment and usage, otherwise using .deb package will continue being the preferred way.

Having Fluent Bit as a classic snap will help users to rapidly setup a data pipeline without any operational concerns. A bit of background about the tool:

  • Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
  • License: Apache v2.0
  • 80% of our users are in the containers space: we see > 600k pulls “per day”
  • we see a trend for it use in non-containerized environments, for hence we need a easy to deploy “Snap”

thanks again for your help,

#5

why would a user have to connect an interface manually ? as you can ask for classic confinement, you can ask the same way for auto-connection of interfaces at install time :wink:

#6

@edsiper As ogra says above, you can request that interfaces for a given snap are automatically connected as per Process for aliases, auto-connections and tracks - so this should allow you to provide a seamless experience for users when installing fluent-bit.

#7

I think there is some lack of understanding of all interfaces and proper setup from my side.

Please advise what setup is required to accomplish the following:

  • open files (read/write) from “random” fs locations: ~home, /opt, /tmp, mounted volumes etc
  • the ability to listen on TCP/UDP ports, as well connect to external services through TCP/UDP
  • create and connect to unix sockets
  • read /dev/kmsg
  • read serial interfaces /dev/ttySx
  • read /proc/* info
  • read from docker unix socket
  • read thermal info
  • spawn an external process: exec(3)

note: we would like to try to avoid setting up a new interface connection for every single use case for our tool, if classic can be considered would be appreciated.

#8

additional information:

respecting the logs (file system-based), they are not usually consumed from /var/log (for most service applications yes…), but we see many use cases where they are pre-parsed logs and placed in a random locations for certain production environments. We need to have the flexibility to read/write where the user decides. If that is a restriction they will need to stick with normal .deb,

#9

The review process and categories of applications / use-cases for classic confinement are well documented - Process for reviewing classic confinement snaps - and reading files from random locations is not a supported use-case for the granting of classic confinement. For most of the other use-cases you raise there are existing interfaces available - and tools like snappy-debug can help elucidate which ones are most appropriate for the various accesses which a snap requires - the use of snappy-debug is also well documented at https://snapcraft.io/docs/debug-snaps

Off the top of my head, I think the following interfaces should allow most of what you need (but again snappy-debug would be a good way to figure this our for cetain):

  • open files (read/write) from “random” fs locations: ~home, /opt, /tmp, mounted volumes etc
    • home and removable-media should cover most locations that users desire
  • the ability to listen on TCP/UDP ports, as well connect to external services through TCP/UDP
    • network-bind and network
  • create and connect to unix sockets
    • this is covered by network[-bind] above
  • read /dev/kmsg
    • this is not covered by current interfaces - could log-observe be used instead?
  • read serial interfaces /dev/ttySx
    • serial-port
  • read /proc/* info
    • system-observe may provide this (it depends on what exactly you are looking for)
  • read from docker unix socket
    • docker
  • read thermal info
    • hardware-observe
  • spawn an external process: exec(3)
    • strictly confined snaps can freely spawn external processes that are shipped within their snap itself - but there is no way to spawn processes outside of the snap as this would break confinement. Can you elaborate more on this?
#10

@edsiper ping, could you analyze the information provided by @alexmurray?

#11

@edsiper - ping, this request cannot proceed without the requested information?

#12

@edsiper - since we’ve not heard back from you, we are removing this request from our review queue. When you have more time to respond, simply do so here and we can add the request back to the queue. Thanks