Where is my data?

When I installed Devilbox a few years ago, I doubled-checked that HOST_PATH_MYSQL_DATADIR is set correctly and that the databases are in the data directory. Peace of mind.

Last week, as I was struggling against Covid, when I started the Devilbox my app couldn’t connect to the DB ; I noticed that for some reason Devilbox recreated the container for MySQL. No problem I thought, my databases are still there and this is a backed up folder. But wait! these files are old?!? more that two years!

Then I checked the current docker-compose.yml and I was horrified to see that: :scream:

- devilbox-${MYSQL_SERVER}:/var/lib/mysql:rw${MOUNT_OPTIONS}

What??? a named volume??? is this a nightmare caused by the disease?? :face_with_thermometer:

Please don’t tell me my data have been on a transient support all this time, please tell me I didn’t loose months of work… :pray: :pray: :pray:

Where is my work???

Why should you have lost your work?

You should have a look here for the correct upgrading instructions: devilbox/UPDATING.md at master · cytopia/devilbox · GitHub

Thanks for your answer.

Where did you find that putting valuable data in a docker volume is a best practice? it really doesn’t sound good to me. If the volume becomes dangling (which it will be now that “docker-composer down” removes the containers), a simple “prune” would erase the data without warning.
The upgrade instructions were to import at upgrade time, so it wouldn’t gave give me a recent backup of the data in the docker volume.

So what I should have done is to add another backup mechanism (in add to the ones I already have) to find out which of the many docker volumes has valuable data, find out which containers are connected to these volumes, spin up these containers, one at a time, trigger a live backup to a set of flat files for each of them and spin down the container… sound awfully complicated and error-prone.

How do you backup your databases?

Not sure If I get your problem correctly.

If you still have the data directories, you can downgrade to the latest v0.x.y version. Start all relevant container and then do a backup: Backup and restore MySQL — Devilbox 1.0 documentation

Once done, you switch back to the recent version and import the data

There’s nothing left to backup, the docker volumes are empty, as they have been re-created.

If there’s no data left, then I’m afraid I can’t help either. I’m still curious to know why those volumes have been recreated?

Docker volumes are transient by nature. If the “stack” is down (like in “docker-compose down”), the volumes, including named volumes, become orphans and will be potentially cleaned up (those volumes can pile up fast on a dev machine).
That’s why one should never put valuable data in docker volumes, unless one have set up a special mechanism to keep the volumes locked.

For those interested, I used the override mechanism like this:

version: '2.3'
services:
  mysql:
    image: devilbox/mysql:${MYSQL_SERVER}-0.7
    hostname: mysql
    volumes:
      - ${DEVILBOX_PATH}/data/mysql-${MYSQL_SERVER}:/var/lib/mysql:rw${MOUNT_OPTIONS}

now my databases are safely stored on my disk (well, relatively).

I would disagree here. It depends on the data. Files and configuration should be mounted, anything else like DB data or similar should be a named volume. Afaik this also has performance benefits.

Additionally I would recommend docker-compose stop instead of docker-compose down. Nonetheless, docker-compose down does not delete volumes by default, you need to explicitly add the -v switch to delete the volumes:

$ docker-compose down
Removing devilbox2_mysql_1 ... done
Removing devilbox2_httpd_1 ... done
Removing devilbox2_php_1   ... done
Removing devilbox2_bind_1  ... done
Removing network devilbox2_app_net
$ docker-compose down -v
Removing network devilbox2_app_net
WARNING: Network devilbox2_app_net not found.
Removing volume devilbox2_devilbox-mail
Removing volume devilbox2_devilbox-mysql-5.5
Removing volume devilbox2_devilbox-mysql-5.6
Removing volume devilbox2_devilbox-mysql-5.7
Removing volume devilbox2_devilbox-mysql-8.0
Removing volume devilbox2_devilbox-mariadb-5.5
Removing volume devilbox2_devilbox-mariadb-10.0
Removing volume devilbox2_devilbox-mariadb-10.1
Removing volume devilbox2_devilbox-mariadb-10.2
Removing volume devilbox2_devilbox-mariadb-10.3
Removing volume devilbox2_devilbox-mariadb-10.4
Removing volume devilbox2_devilbox-mariadb-10.5
Removing volume devilbox2_devilbox-mariadb-10.6
Removing volume devilbox2_devilbox-mariadb-10.7
Removing volume devilbox2_devilbox-mariadb-10.8
Removing volume devilbox2_devilbox-percona-5.5
Removing volume devilbox2_devilbox-percona-5.6
Removing volume devilbox2_devilbox-percona-5.7
Removing volume devilbox2_devilbox-percona-8.0
Removing volume devilbox2_devilbox-pgsql-9.0
Removing volume devilbox2_devilbox-pgsql-9.1
Removing volume devilbox2_devilbox-pgsql-9.2
Removing volume devilbox2_devilbox-pgsql-9.2-alpine
Removing volume devilbox2_devilbox-pgsql-9.3
Removing volume devilbox2_devilbox-pgsql-9.3-alpine
Removing volume devilbox2_devilbox-pgsql-9.4
Removing volume devilbox2_devilbox-pgsql-9.4-alpine
Removing volume devilbox2_devilbox-pgsql-9.5
Removing volume devilbox2_devilbox-pgsql-9.5-alpine
Removing volume devilbox2_devilbox-pgsql-9.6
Removing volume devilbox2_devilbox-pgsql-9.6-alpine
Removing volume devilbox2_devilbox-pgsql-10
Removing volume devilbox2_devilbox-pgsql-10-alpine
Removing volume devilbox2_devilbox-pgsql-11
Removing volume devilbox2_devilbox-pgsql-11-alpine
Removing volume devilbox2_devilbox-pgsql-12
Removing volume devilbox2_devilbox-pgsql-12-alpine
Removing volume devilbox2_devilbox-pgsql-13
Removing volume devilbox2_devilbox-pgsql-13-alpine
Removing volume devilbox2_devilbox-pgsql-14
Removing volume devilbox2_devilbox-pgsql-14-alpine
Removing volume devilbox2_devilbox-pgsql-latest
Removing volume devilbox2_devilbox-pgsql-alpine
Removing volume devilbox2_devilbox-mongo-2.8
Removing volume devilbox2_devilbox-mongo-3.0
Removing volume devilbox2_devilbox-mongo-3.2
Removing volume devilbox2_devilbox-mongo-3.4
Removing volume devilbox2_devilbox-mongo-3.6
Removing volume devilbox2_devilbox-mongo-4.0
Removing volume devilbox2_devilbox-mongo-4.2
Removing volume devilbox2_devilbox-mongo-4.4
Removing volume devilbox2_devilbox-mongo-5.0
Removing volume devilbox2_devilbox-mongo-latest

Volumes will be deleted if you use the “prune” feature, which I do on a regular basis to avoid keeping hundreds of stale volumes.

For future prune operations, I would recommend to exclude volumes you want to keep. See example below:

docker volume prune --filter label!=keep

Another crucial thing to consider is to make regular backups for any kind of data which is important to you. Devilbox has documentation for how to do it with its stateful stuff. See here for example:

https://devilbox.readthedocs.io/en/latest/maintenance/backup-and-restore-mysql.html