How to setup dbt using docker containers on Windows 10

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

After reading Dbt documentation, I've had a hard time to figure out how to install dbt-core (or any other packages i.e. dbt-postgres, dbt-snowflake, etc) on Windows 10.

I have Docker Desktop installed, running a couple of containers already (mostly nodeJS containers, and Kafka). However, it was hard to understand how I would have those new Dbt containers available in my Docker Desktop Console.

I can see docker images were installed properly

$docker image ls
REPOSITORY                      TAG          IMAGE ID       CREATED        SIZE
**ghcr.io/dbt-labs/dbt-core       1.2.1        802a0d70aedc   4 weeks ago    538MB**
**ghcr.io/dbt-labs/dbt-bigquery   1.2.latest   b7502bcd3b35   2 months ago   559MB**
postgres                        latest       f8dd270e5152   7 weeks ago    376MB
dpage/pgadmin4                  latest       d13c9d7d0193   2 months ago   382MB
wurstmeister/kafka              latest       a692873757c0   4 months ago   468MB
wurstmeister/zookeeper          latest       3f43f72cb283   3 years ago    510MB
Does anyone know how to I them to the Desktop Console?
                Unless you really know Docker and/or have a reason to use it, it's probably much easier to just install Python and then pip install dbt-core on Windows 10. Personally I use WSL2, so I can run Ubuntu on Windows, which is amazing, but is a whole different can of worms.
– tconbeer
                Sep 30, 2022 at 21:07
                Indeed. Much easier that way. I'd installed dbt adapters via pip before, I've tried via docker. Perfectionism
– iuri
                Oct 3, 2022 at 15:00
I'm currently on Windows 10 and use a Docker image for my dbt project without needing WSL. Below is my Dockerfile and requirements.txt file with dbt-core and dbt-snowflake but feel free to swap the packages you need.
In my repo, my dbt project is in a folder at the root level named dbt.
requirements.txt
dbt-core==1.1.0
dbt-snowflake==1.1.0
Dockerfile
FROM public.ecr.aws/docker/library/python:3.8-slim-buster
COPY . /dbt
# Update and install system packages
RUN apt-get update -y && \
  apt-get install --no-install-recommends -y -q \
  git libpq-dev python-dev && \
  apt-get clean && \
  rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Install dbt
RUN pip install -U pip
RUN pip install -r dbt/requirements.txt
# TEMP FIX due to dependency updates. See https://github.com/dbt-labs/dbt-core/issues/4745
RUN pip install --force-reinstall MarkupSafe==2.0.1
# Install dbt dependencies
WORKDIR /dbt
RUN dbt deps
# Specify profiles directory
ENV DBT_PROFILES_DIR=.dbt
# Expose port for dbt docs
EXPOSE 8080
And then you can build and run it (I personally put both of these commands in a dbt_run.sh file and run with bash dbt_run.sh):
docker build -t dbt_image .
docker run \
-p 8080:8080 \
--env-file .env \
-it \
--mount type=bind,source="$(pwd)",target=/dbt \
dbt_image bash
If you make changes to your dbt project while the container is running they will be reflected in the container which makes it great for developing locally. Hope this helps!
I finally was able to pull the image. To add a container in the Docker desktop, I just needed to actually run it.
However, running a dbt-core container in docker, it returns an error:
right after I start the container it stops and returns exit(1), as per the screenshot.
        Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.