Hello snappers. We have a tool called hotos that we use to perform analysis of both sosreports and live hosts. It comprises of a set of “plugins” that extract application-specific information for applications such as Openstack, Kubernetes and Ceph and presents findings that might help in the debug/analysis of problems. The snap been strictly confined from the start and this works well for the scenario where you want to analyse a sosreport but less well when wanting run against an actual host. This is basically because, like the sosreport tool itself, it needs access to basically everything on the system in order to collect information relevant to the application that is running including network information, logs, etc configs, /proc info as well as locations specific to the application itself e.g. /var/lib, /run etc etc. Of course some of these locations can be accessed with the use of existing interfaces but some are not and I believe that in this instance requiring interface for all these areas will be counterproductive. As such I believe the appropriate way to allow users to run this on hosts is to allow classic mode. It is probably worth mentioning that the target audience is likely the same as those who already use tools like sosreport and as such expect them to be able access any data necessary. Currently to achieve this users have the install the snap and then run the script like /snap/hotsos/current/hotsos.sh which i would really like to avoid. I look forward to hearing your thoughts. Thanks.
Could you please describe the locations that you consider are not accessible with the use of interfaces? And also why you consider requiring such interfaces will be counterproductive?
Please remember classic snaps are not installable on Ubuntu Core devices and also run in the global mount namespace, which means great care must be taken for the snap to work reliably across distributions. Ideally we try snaps to keep under strict confinement, since classic effectively grants device ownership to the snap.
Hi @emitorino , thanks for taking the time to review my request. This application is used by support teams and operators to analyse and debug applications like Openstack and Kubernetes. Much like the tool sosreport it assumes it has unrestricted access to a host in order to be able to collect information and make decisions on the health of an application based on the state that it observes. Hotsos is read-only and does not modify the state of a host, it merely gathers information, presents it to the user and makes decisions about whether the host/application is affected by any known bugs or issues.
An example of what happens when I run the confined snap on a host is https://pastebin.ubuntu.com/p/Chwpt7D4qk/ i.e. can’t access locations such as /var/log/, /etc/, /var/lib/* and /proc/* and cannot observe ps which, combined with other information is how it detects which applications/binaries are running/not running on host.
I have used interfaces in my other snaps and both understand and appreciate their worth. They work well in scenarios where you have specific requirements and can allow access to those things and nothing else. This application is somewhat the opposite which is why i think it will be very hard to make it depend on interfaces for access while at the same time avoiding having false reports simply because something that it would otherwise be able to observe/access is now restricted due to insufficient privileges. If you have suggestions on interfaces or appoaches I should try I am more than happy to give them a go but as I state I beleive that in this case a restrictive approach will introduce risk to my application that could render it unrelialble and unsusable. Thanks. Ed.
Hi sorry i’ve been very busy lately but ive finally found some time to try this out. It still won’t give me access to everything i need though e.g. ps, ip, /var/lib, df, /proc and so even with those interfaces you mention added and connected i still have some things i cannot access - https://pastebin.ubuntu.com/p/DsCzQJZdWs/. But my point is more that for example if i add some code that depends on the existence of some file under /var/lib/foo or ps entry and my code is set to skip the check if the entry does not exist yet it does exist but rather just cant access it, then that is a false negative and renders my checks somewhat untrustworthy. So yes i can keep adding more interfaces to ensure that more is accessible but I still have this risk that the tool will think something does not exist when actually it is just that it doesn’t have access. I look forward to hearing your thoughts on this.
It’s a pretty long list of connections to have to add each time someone installs the snap, is there a way to have them automatically connected? I’m not sure why i’m still getting an EPERM from ip and that proc entry.
and you can indeed turn this request for classic confinement into a request for auto-connecting these interfaces, just change the topic to reflect this …
It fixed the ip issue but there are still others e.g.
INFO: analysing localhost since no sosreport path provided
ls: cannot open directory ‘/sys/block/’: Permission denied
Traceback (most recent call last):
File “/snap/hotsos/x1/plugins/storage/parts/main.py”, line 5, in
PluginRunner()()
File “/snap/hotsos/x1/common/plugintools.py”, line 154, in call
inst()
File “/snap/hotsos/x1/plugins/storage/parts/pyparts/bcache.py”, line 39, in call
self.get_device_info()
File “/snap/hotsos/x1/plugins/storage/parts/pyparts/bcache.py”, line 20, in get_device_info
for line in cli_helpers.get_ls_lanR_sys_block():
File “/snap/hotsos/x1/common/cli_helpers.py”, line 29, in catch_exceptions_inner2
return f(*args, **kwargs)
File “/snap/hotsos/x1/common/cli_helpers.py”, line 309, in get_ls_lanR_sys_block
output = subprocess.check_output([‘ls’, ‘-lanR’, ‘/sys/block/’])
File “/usr/lib/python3.6/subprocess.py”, line 356, in check_output
**kwargs).stdout
File “/usr/lib/python3.6/subprocess.py”, line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘ls’, ‘-lanR’, ‘/sys/block/’]’ returned non-zero exit status 2.
-Traceback (most recent call last):
File “/snap/hotsos/x1/plugins/kernel/parts/main.py”, line 5, in
PluginRunner()()
File “/snap/hotsos/x1/common/plugintools.py”, line 154, in call
inst()
File “/snap/hotsos/x1/plugins/kernel/parts/pyparts/memory.py”, line 139, in call
self.get_memory_info()
File “/snap/hotsos/x1/plugins/kernel/parts/pyparts/memory.py”, line 113, in get_memory_info
self.check_nodes_memory(“Normal”)
File “/snap/hotsos/x1/plugins/kernel/parts/pyparts/memory.py”, line 93, in check_nodes_memory
nodes = self.numa_nodes
File “/snap/hotsos/x1/plugins/kernel/kernel_common.py”, line 129, in numa_nodes
for line in open(BUDDY_INFO):
PermissionError: [Errno 13] Permission denied: ‘/proc/buddyinfo’
I am also still uncomfortable with the fact that if we run in confined mode we run the risk of false negatives which would render the tool unreliable so would prefer to pursue supporting classic mode. If that is simply not possible then ill just have to build a debian package instead which would be a shame.
as @emitorino suggested before, you should really use snappy-debug from the snappy-debug snap and run it in parallel to your app, there are interfaces for accessing block devices as well as for checking memory and the tool should suggest the correct interfaces to use … if there are still missing bits we might need to extend an existing interface or add a new one …
@dosaboy, there is a process and criteria to grant classic confinement to snaps. Your request is not directly falling under the supported categories so we are trying to help you snap your app while keeping under strict confinement (because of the reasons already explained in the previous comments).
Can you already confirm the snap is providing false positives after fixing the denials?
