Hi all, I have been working on the soscleaner snap for a while, and now pushed it to candidate. Could I get log-observe on the snap automcatically connected please
Can you please describe why soscleaner would require log-observe?
Hi Alex,
thanks for looking into this for me
soscleaner is an application that obfuscates various items from a sosreport. A sosreport will have many configurations and log files that are similar permissions of logs from the system. So in order for soscleaner to work effectively, it needs to be able to view these files, so that it can successfully obfuscate sensitive data
Below is an output of the command not having the log-observe connection
root@sosclean-test:~# soscleaner sosreport-sosclean-test-2020-05-04-nwjjljw.tar.xz
05-04 11:23:58 soscleaner CONSOLE: Log File Created at /tmp/soscleaner-3235800356563888/soscleaner-3235800356563888.log
05-04 11:23:58 soscleaner CONSOLE: SOSCleaner version: 0.4.4
05-04 11:23:58 soscleaner CONSOLE: soscleaner is a tool to help obfuscate sensitive information from an existing sosreport.
05-04 11:23:58 soscleaner CONSOLE: Please review the content before passing it along to any third party.
05-04 11:23:58 soscleaner CONSOLE: Creating Loopback Network Entry
05-04 11:23:58 soscleaner CONSOLE: Beginning SOSReport Extraction
05-04 11:25:20 soscleaner CONSOLE: Created New Obfuscated Network - 129.0.0.0/24
05-04 11:25:20 soscleaner CONSOLE: Created New Obfuscated Network - 130.0.0.0/32
05-04 11:25:20 soscleaner CONSOLE: Adding new obfuscated domain - redhat.com > ofuscateddomain1.com
05-04 11:25:20 soscleaner CONSOLE: Adding new obfuscated domain - localhost.localdomain > ofuscateddomain2.com
05-04 11:25:20 soscleaner CONSOLE: Adding new obfuscated domain - localdomain > ofuscateddomain3.com
05-04 11:25:20 soscleaner CONSOLE: Adding new obfuscated domain - localhost > ofuscateddomain4.com
05-04 11:25:20 soscleaner CONSOLE: Unable to locate user file - /tmp/soscleaner-3235800356563888/soscleaner-3235800356563888/sos_commands/last/lastlog_-u_1000-60000
05-04 11:25:20 soscleaner CONSOLE: Continuing without processing users file
05-04 11:25:20 soscleaner CONSOLE: IP Obfuscation Network Created - 128.0.0.0/8
05-04 11:25:20 soscleaner CONSOLE: *** SOSCleaner Processing ***
05-04 11:25:20 soscleaner ERROR: [Errno 13] Permission denied: u'/tmp/soscleaner-3235800356563888/soscleaner-3235800356563888/sys/fs/cgroup/devices/system.slice/snapd.service/devices.list'
Traceback (most recent call last):
File "/snap/soscleaner/23/lib/python2.7/site-packages/soscleaner.py", line 1056, in _clean_file
new_fh = open(f, 'w')
IOError: [Errno 13] Permission denied: u'/tmp/soscleaner-3235800356563888/soscleaner-3235800356563888/sys/fs/cgroup/devices/system.slice/snapd.service/devices.list'
Traceback (most recent call last):
File "/snap/soscleaner/23/bin/soscleaner", line 71, in <module>
main()
File "/snap/soscleaner/23/bin/soscleaner", line 67, in main
cleaner.clean_report(options, sosreport)
File "/snap/soscleaner/23/lib/python2.7/site-packages/soscleaner.py", line 1642, in clean_report
self._clean_file(f)
File "/snap/soscleaner/23/lib/python2.7/site-packages/soscleaner.py", line 1074, in _clean_file
"CLEAN_FILE_ERROR: Unable to write obfuscated file - %s" % f)
Exception: CLEAN_FILE_ERROR: Unable to write obfuscated file - /tmp/soscleaner-3235800356563888/soscleaner-3235800356563888/sys/fs/cgroup/devices/system.slice/snapd.service/devices.list
When I then run snap connect soscleaner:log-observer, we then have a successful run of soscleaner as shown below
root@sosclean-test:~# soscleaner sosreport-sosclean-test-2020-05-04-nwjjljw.tar.xz -l DEBUG
05-04 11:34:27 soscleaner CONSOLE: Log File Created at /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135.log
05-04 11:34:27 soscleaner CONSOLE: SOSCleaner version: 0.4.4
05-04 11:34:27 soscleaner CONSOLE: soscleaner is a tool to help obfuscate sensitive information from an existing sosreport.
05-04 11:34:27 soscleaner CONSOLE: Please review the content before passing it along to any third party.
05-04 11:34:27 soscleaner CONSOLE: Creating Loopback Network Entry
05-04 11:34:27 soscleaner CONSOLE: Beginning SOSReport Extraction
05-04 11:35:50 soscleaner CONSOLE: Created New Obfuscated Network - 129.0.0.0/24
05-04 11:35:50 soscleaner CONSOLE: Created New Obfuscated Network - 130.0.0.0/32
05-04 11:35:50 soscleaner CONSOLE: Adding new obfuscated domain - redhat.com > ofuscateddomain1.com
05-04 11:35:50 soscleaner CONSOLE: Adding new obfuscated domain - localhost.localdomain > ofuscateddomain2.com
05-04 11:35:50 soscleaner CONSOLE: Adding new obfuscated domain - localdomain > ofuscateddomain3.com
05-04 11:35:50 soscleaner CONSOLE: Adding new obfuscated domain - localhost > ofuscateddomain4.com
05-04 11:35:50 soscleaner CONSOLE: Unable to locate user file - /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135/sos_commands/last/lastlog_-u_1000-60000
05-04 11:35:50 soscleaner CONSOLE: Continuing without processing users file
05-04 11:35:50 soscleaner CONSOLE: IP Obfuscation Network Created - 128.0.0.0/8
05-04 11:35:50 soscleaner CONSOLE: *** SOSCleaner Processing ***
05-04 11:39:05 soscleaner CONSOLE: *** SOSCleaner Statistics ***
05-04 11:39:05 soscleaner CONSOLE: IP Addresses Obfuscated - 39
05-04 11:39:05 soscleaner CONSOLE: Hostnames Obfuscated - 4
05-04 11:39:05 soscleaner CONSOLE: Domains Obfuscated - 4
05-04 11:39:05 soscleaner CONSOLE: Users Obfuscated - 0
05-04 11:39:05 soscleaner CONSOLE: Keywords Obfuscated - 0
05-04 11:39:05 soscleaner CONSOLE: Total Files Analyzed - 8372
05-04 11:39:05 soscleaner CONSOLE: *** SOSCleaner Artifacts ***
05-04 11:39:05 soscleaner CONSOLE: Creating IP Report - /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135-ip.csv
05-04 11:39:05 soscleaner CONSOLE: Creating Hostname Report - /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135-hostname.csv
05-04 11:39:05 soscleaner CONSOLE: Creating Domainname Report - /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135-dn.csv
05-04 11:39:05 soscleaner CONSOLE: Creating Username Report - /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135-username.csv
05-04 11:39:05 soscleaner CONSOLE: Creating MAC address Report - /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135-mac.csv
05-04 11:39:05 soscleaner CONSOLE: Creating keyword address Report - /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135-keyword.csv
05-04 11:39:05 soscleaner CONSOLE: Creating sosreport Report - /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135-sosreport.csv
05-04 11:39:05 soscleaner CONSOLE: Creating SOSCleaner Archive - /tmp/soscleaner-1323698215372135/soscleaner-1323698215372135.tar.gz
05-04 11:39:12 soscleaner CONSOLE: md5 checksum is: 7da9d1fb1166fb09bb6ecc63e8477f13
05-04 11:39:12 soscleaner CONSOLE: SOSCleaner Complete
I hope that makes sense
Hmm it is still not clear to me what part of the log-observe interface is required for soscleaner since from the above it does not appear to be accessing any of the actual system logs (and from my understanding it should only be accessing the existing sosreport) - can you please any denials that are listed in dmesg when running soscleaner without the log-observe interface being connected?
@arif-ali - can you respond to @alexmurray’s question? This cannot proceed without the requested information.
@arif-ali I am removing this request from our queue - if you can please provide the requested information then we can re-add it for processing again in the future.
@alexmurray, below is the output from dmesg, and at the time the DENIED action when running soscleaner
[289584.519549] audit: type=1400 audit(1591738920.754:22908): apparmor="DENIED" operation="capable" namespace="root//lxd-sosclean-test_<var-snap-lxd-common-lxd>" profile="snap.soscleaner.soscleaner" pid=2041897 comm="python2" capability=1 capname="dac_override"
While it is true that dac_override is granted by the log-observe interface, that meant for dealing with log files who have different ownership than a root-running process (as many logs tend to have different owners).
Sometimes a dac_override denial pops because ownership is not properly setup by the snap (if you are using the system-usernames, you might see "Ownership (discretionary access controls) in System usernames (though you might find it helpful for understanding how dac_override is enforced)). Is your snap using system-usernames?
What are the ownership and permissions of the files that snap.soscleaner.soscleaner is accessing when you see the denial? Perhaps there is something you can do in your snap or something we need to adjust in a different interface.
sosreport is a tool that collects wide variety of data for support personnel to then diagnose a problem on any given system. So the permissions of the files that soscleaner is inspecting will have various permissions that would be coming from /sys, /proc, /var/log and many others from the filesystem
but, more than happy to look at dac_override further if that is the route I need should take.
Ok, it wasn’t clear from @alexmurray’s question that soscleaner needs /var/log for files in /var/log. It sounds like in the particular case of the test run it didn’t and had a dac_override denial due to perms in the test dir.
I actually expected soscleaner to need access to /var/log and the fact that plugging that happens to also fix your test command is ok. Now that I see it is in your test env, I don’t think there is any reason to chase down the denial.
+1 to auto-connect log-observe. @reviewers - can others please vote?
1 Like
Thanks for the clarification @arif-ali - +1 from me for auto-connect of log-observe for soscleaner too.
+2 votes for, 0 votes against auto-connect of log-observe for soscleaner - this is now live.
2 Likes