Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Here is my docker/docker-compose.yml :

version: "3.7"
services:
  myapp-main-db:
    env_file:
      - ../.env
    image: mysql:8
    container_name: myapp-main-db
    command: --default-authentication-plugin=mysql_native_password
    restart: always
    ports:
      - $SERVICE_DB_PORT:$SERVICE_DB_PORT
    environment:
      MYSQL_ROOT_PASSWORD: $SERVICE_DB_ROOT_PASSWORD
      MYSQL_DATABASE: myapp_service_db_$ENV
      MYSQL_USER: $SERVICE_DB_APP_USER
      MYSQL_PASSWORD: $SERVICE_DB_APP_PASSWORD
    volumes:
      - myapp-service-db-data:/var/lib/mysql
volumes:
  myapp-service-db-data:
When I run docker-compose -f docker/docker-compose.yml up -d from the project root directory, I see the MySQL container spin up without issue, and verify its running via docker ps.
However I am unable to connect to it using the expected connection string and credentials, which I am sure are correct. This leads me to believe that Docker Compose is not loading the ../.env file as I was expecting it to, and so none of the variables defined in ../.env are being injected as env vars.
Can anybody spot where I'm going awry?
Update
When I run docker-compose config I get:
$ docker-compose config
WARNING: The SERVICE_DB_PORT variable is not set. Defaulting to a blank string.
WARNING: The SERVICE_DB_ROOT_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The ENV variable is not set. Defaulting to a blank string.
WARNING: The SERVICE_DB_APP_USER variable is not set. Defaulting to a blank string.
WARNING: The SERVICE_DB_APP_PASSWORD variable is not set. Defaulting to a blank string.
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.myapp-main-db.ports contains an invalid type, it should be a number, or an object
                I am not sure, but could it be that the path that you give to the "env_file" is relative to the current dir where you launch docker-compose from and not to the directory containing docker-compose.yml? In this case you could try by changing it to ./.env but, I repeat, I am not sure
– EuberDeveloper
                Jun 3, 2021 at 10:57
                First of all, for debugging purposes, I'd suggest you try the command docker-compose config to display the whole, implied configuration (with the variables substituted…)
– ErikMD
                Jun 3, 2021 at 11:02
                Thanks @EuberDeveloper (+1) but when I change it to ./.env and re-run the command I get ERROR: Couldn't find env file: /Users/myuser/workspace/myapp-service/docker/.env, so I do believe ../.env is correct.
– hotmeatballsoup
                Jun 3, 2021 at 11:08
                Thanks @ErikMD (+1) please see my edit above which includes the output of docker-compose config. So it seems my suspicions are correct, that its not loading the variables defined in the ../.env file.
– hotmeatballsoup
                Jun 3, 2021 at 11:11
                Taking a closer look at your docker-compose.yml, it seems the two notions env_file and .env are somewhat entangled, while there are definitely different notions! Maybe the SO thread Docker Compose extra_hosts from env_file and not from default .env file could be useful to you?
– ErikMD
                Jun 3, 2021 at 11:18
Compose has two separate layers of environment file handling:
When the docker-compose.yml file is read in, it reads a .env file from the same directory as the docker-compose.yml, and uses that to expand $VARIABLE references.
When an individual container is launched, it reads the env_file: and uses that to set environment variables in the container.
Since .env isn't in the same directory as the docker-compose.yml file, Compose won't read it before expanding variables in the file.  That leads to the container getting empty strings for the database credentials, and the error because ports: is malformed.
You can either move .env into the same directory as docker-compose.yml
mv .env docker
docker-compose -f docker/docker-compose.yml up
or pass a docker-compose --env-file option
docker-compose -f docker-docker-compose.yml --env-file .env up
You don't need env_file: in the docker-compose.yml file for this particular use and can remove it.
To debug this further, you can docker-compose run a container with an alternate command:.  You could for example docker-compose run myapp-main-db env to print out its environment instead of launching a database.  The flow I'd expect you to see here is:
Compose reads docker/.env (from the same directory as the docker-compose.yml file), which is absent
Compose replaces variables in the file; I'd expect to see MYSQL_USER to be an empty string
Compose reads the env_file:; the container will have the additional environment variable SERVICE_DB_APP_USER=my-service-user from that file
According to default behavior of docker, Dockerfile and docker-compose never read any file or value from parent directory. It always read from current directory or from Child directory.
so for your code  I suggest please use docker-compose file in source/root directory.
Example:
myapp/
    docker-compose.yml
        <my source code here>
    config.properties
Quote from the official Docker docs: Using the “--env-file” option.
You can use an .env file from anywhere, using the --env-file option:
By passing the file as an argument, you can store it anywhere and name
it appropriately, for example, .env.ci, .env.dev, .env.prod. Passing
the file path is done using the --env-file option:
docker-compose --env-file ./config/.env.dev up
Obviously, you cannot go to a parent directory of your current directory to get the .env file when you are inside the docker-compose, thus, not like this:
version: "3.7"
services:
  myapp-main-db:
    env_file:
      - ../.env
One way to use an .env file from somewhere else like here at the parent folder is to use the --project-directory parameter of the docker-compose command. According to the official Docker docs:
The .env file path is as follows:
Starting with +v1.28, .env file is placed at the base of the project
directory
For previous versions, it is placed in the current working
directory where the Docker Compose command is executed unless a
--project-directory is defined which overrides the path for the .env file. This inconsistency is addressed in +v1.28 by limiting the
filepath to the project directory."
Which would mean in your case:
docker-compose --project-directory .. up
Or: you can reach any .env file on your system if you pass it as the --env-file parameter of the docker-compose command.
docker-compose --env-file ../.env up
If you use neither --project-directory nor --env-file you need to put the .env file where the docker-compose file is.
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.