How to change Docker data-dir folder

Hi forum!

This is my first thread in here, so first of all, hello everyone!

I started using snap consistently just today, since my old system literally blew up and thus I installed another system in my root partition.

Previously I was using docker installed system wide with a custom data directory, in order not to use the system partition and to be able to share Docker data between installations.

So today I installed Docker through snap, the process was smooth and I had docker usable within minutes.

The main problem comes when I try to link my previous docker data folder (which is in ~/.docker) with the docker-snap data folder (which is in /var/snap/docker/common/var-lib-docker).

I tried creating the symlink, changing permission to the ~/.docker folder to 750 and owned by root, restart docker snap using sudo snap restart docker but sudo snap docker logs always returns this message:

2020-04-16T11:21:32Z docker.dockerd[19836]: time="2020-04-16T13:21:32.323447844+02:00" level=error msg="transport: loopyWriter.run returning. Err: connection error: desc = \"transport is closing\"" module=grpc
2020-04-16T11:21:33Z docker.dockerd[19836]: Error starting daemon: rename /home/giacomo/.docker/runtimes /home/giacomo/.docker/runtimes-old: permission denied
2020-04-16T11:21:33Z systemd[1]: snap.docker.dockerd.service: Main process exited, code=exited, status=1/FAILURE
2020-04-16T11:21:33Z systemd[1]: snap.docker.dockerd.service: Failed with result 'exit-code'.
2020-04-16T11:21:33Z systemd[1]: snap.docker.dockerd.service: Service RestartSec=100ms expired, scheduling restart.
2020-04-16T11:21:33Z systemd[1]: snap.docker.dockerd.service: Scheduled restart job, restart counter is at 5.
2020-04-16T11:21:33Z systemd[1]: Stopped Service for snap application docker.dockerd.
2020-04-16T11:21:33Z systemd[1]: snap.docker.dockerd.service: Start request repeated too quickly.
2020-04-16T11:21:33Z systemd[1]: snap.docker.dockerd.service: Failed with result 'exit-code'.
2020-04-16T11:21:33Z systemd[1]: Failed to start Service for snap application docker.dockerd.

Where the interesting bit is here:

2020-04-16T11:21:33Z docker.dockerd[19836]: Error starting daemon: rename /home/giacomo/.docker/runtimes /home/giacomo/.docker/runtimes-old: permission denied

I also noticed that, due to this fact of not being able to rename the folder, someone or something in the docker-snap performs some write action in the folder, since some folder are created, those in particular:

drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes115886176
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes200242156
drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes214593504
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes326535364
drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes378553064
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes509367655
drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes599556730
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes619075387
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes658546954
drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes934189445

Have anyone of you tried to accomplish this task? Is this even possible?

I’m not totally sure which user is running the docker service inside the snap, so I’m not sure how to deal with permissions.

Another thing I tried was to change the data-dir attribute of /var/snap/docker/current/config/daemon.json, but once restarted, it was complaining about “flag attribute colliding with file attribute”.

Here I leave the relevant directory permissions:
Directory: /var/snap/docker/common

totale 12
drwxr-xr-x 3 root root 4096 apr 16 13:20 .
drwxr-xr-x 4 root root 4096 apr 16 12:55 ..
-rw-r--r-- 1 root root    0 apr 16 13:21 profile_reloaded
drwxr-xr-x 2 root root 4096 apr 16 11:49 run
lrwxrwxrwx 1 root root   21 apr 16 13:20 var-lib-docker -> /home/giacomo/.docker

Directory: ~/.docker

totale 152
drwx--x--x  27 root    root     4096 apr 16 13:21 .
drwx------ 106 giacomo giacomo  4096 apr 16 11:52 ..
drwxr-x---   2 root    root     4096 feb  1 13:43 builder
drwxr-x---   4 root    root     4096 feb  1 13:43 buildkit
-rwxr-x---   1 root    root      185 apr  1 21:18 config.json
drwxr-x---   3 root    root     4096 apr 16 12:17 containerd
drwxr-x---  19 root    root     4096 apr 15 19:36 containers
drwxr-x---  13 root    root     4096 mar 27 16:55 docker
drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes115886176
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes200242156
drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes214593504
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes326535364
drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes378553064
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes509367655
drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes599556730
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes619075387
drwx------   2 root    root     4096 apr 16 12:55 gen-runtimes658546954
drwx------   2 root    root     4096 apr 16 13:21 gen-runtimes934189445
drwxr-x---   3 root    root     4096 feb  1 13:43 image
drwxr-x---   4 root    root     4096 feb  5 20:09 machine
drwxr-x---   3 root    root     4096 feb  1 13:43 network
drwxr-x--- 314 root    root    40960 apr 15 23:06 overlay2
drwxr-x---   4 root    root     4096 feb  1 13:43 plugins
drwxr-x---   2 root    root     4096 apr 16 12:35 runtimes
drwxr-x---   2 root    root     4096 feb  1 13:43 swarm
drwxr-x---   2 root    root     4096 apr 16 13:21 tmp
drwxr-x---   2 root    root     4096 feb  1 13:43 trust
drwxr-x---  26 root    root     4096 apr 15 19:36 volumes

I’d like to thank any of you for any piece of advice you might be helping me with!

do never use symlinks but use bind mounts instead and snaps will be happy :wink:

1 Like

Oh Gosh, I was unaware of that possibility!

Well, can you provide me some guidance or documentation on how to do that? Do I have to modify the snap itself or is is possible to configure something in some yaml files?

The permission denied you see is expected, snaps are denied access to dotfiles in $HOME by default and need to use personal-files in order to access dotfiles. If you just changes this to use $HOME/docker instead it may just work.

1 Like

Thank you @ijohnson for your help! The problem was related, as you suggested, to the fact that the folder’s name was hidden.

So I changed the name, but the problem kinda persists, and by that I mean that I can download and use some images but when I need to start some containers docker is not able to create those containers since inside their content there are some hidden files.

I think the problem is related to the fact that snap cannot open/edit/use any hidden file inside the $HOME folder, thus the error.

@ogra is this problem solved by using bind-mount? And how to do that? Thanks

no, sorry … bind-mounts are useful to replace links but will not help with restrictions that exist by design …

1 Like

Ok!

So how could one move the docker data folder? For now, I’m simply leaving it where it is, so on my system partition, but I’d like to find a better solution, since this is not portable at all, unfortunately!

BTW, thanks a lot to anyone for the kind help! :slight_smile: Very much appreciated!

This should not be the case - specifically the restriction is on dotfiles in $HOME directly, not subdirs, so $HOME/.docker is denied, but $HOME/something/.docker should be fine. Do you see any denials when you try to create a docker container, i.e. what’s the output of journalctl -e --no-pager | grep DENIED ?

Actually, yes, using a bind-mount to make a path stored on /home be /var/snap/docker/common/var-lib-docker will solve the problem because the apparmor rules will marshal the latter path without resolving it to the /home path.

1 Like