Collectives™ on Stack Overflow
Find centralized, trusted content and collaborate around the technologies you use most.
Learn more about Collectives
Teams
Q&A for work
Connect and share knowledge within a single location that is structured and easy to search.
Learn more about Teams
Hi guys need some help.
I created a custom docker image and push it to docker hub but when I run it in CI/CD it gives me this error.
exec /usr/bin/sh: exec format error
Where :
Dockerfile
FROM ubuntu:20.04
RUN apt-get update
RUN apt-get install -y software-properties-common
RUN apt-get install -y python3-pip
RUN pip3 install robotframework
.gitlab-ci.yml
robot-framework:
image: rethkevin/rf:v1
allow_failure: true
script:
- pip3 --version
Output
Running with gitlab-runner 15.1.0 (76984217)
on runner zgjy8gPC
Preparing the "docker" executor
Using Docker executor with image rethkevin/rf:v1 ...
Pulling docker image rethkevin/rf:v1 ...
Using docker image sha256:d2db066f04bd0c04f69db1622cd73b2fc2e78a5d95a68445618fe54b87f1d31f for rethkevin/rf:v1 with digest rethkevin/rf@sha256:58a500afcbd75ba477aa3076955967cebf66e2f69d4a5c1cca23d69f6775bf6a ...
Preparing environment
00:01
Running on runner-zgjy8gpc-project-1049-concurrent-0 via 1c8189df1d47...
Getting source from Git repository
00:01
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/reth.bagares/test-rf/.git/
Checking out 339458a3 as main...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:00
Using docker image sha256:d2db066f04bd0c04f69db1622cd73b2fc2e78a5d95a68445618fe54b87f1d31f for rethkevin/rf:v1 with digest rethkevin/rf@sha256:58a500afcbd75ba477aa3076955967cebf66e2f69d4a5c1cca23d69f6775bf6a ...
exec /usr/bin/sh: exec format error
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
any thoughts on this to resolve the error?
The problem is that you built this image for arm64/v8 -- but your runner is using a different architecture.
If you run:
docker image inspect rethkevin/rf:v1
You will see this in the output:
"Architecture": "arm64",
"Variant": "v8",
"Os": "linux",
Try building and pushing your image from your GitLab CI runner so the architecture of the image will match your runner's architecture.
Alternatively, you can build for multiple architectures using docker buildx . Alternatively still, you could also run a GitLab runner on ARM architecture so that it can run the image for the architecture you built it on.
–
In my case, I was building it using buildx
docker buildx build --platform linux/amd64 -f ./Dockerfile -t image .
however the problem was in AWS lambda 
There could be many reasons why this error occurs while working with containers:
Using the wrong script header such as adding a space in between.
Using some incompatible character encodings while writing the scripts
Non-matching CPU architecture
Missing file permissions
Demonestrate here the reason (3) with example:
rohits-MacBook> catalog % docker version
Its darwin/arm64 architectute of my Laptop and the same docker Engine is installed.
Client:
Cloud integration: v1.0.31
Version: 20.10.23
API version: 1.41
Go version: go1.18.10
Git commit: 7155243
Built: Thu Jan 19 17:35:19 2023
OS/Arch: darwin/arm64
Context: desktop-linux
Experimental: true
Server: Docker Desktop 4.17.0 (99724)
Engine:
Version: 20.10.23
API version: 1.41 (minimum version 1.12)
Go version: go1.18.10
Git commit: 6051f14
Built: Thu Jan 19 17:31:28 2023
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.6.18
GitCommit: 2456e983eb9e37e47538f59ea18f2043c9a73640
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
#################################################
Executing in my current laptop
rohits-MacBook> catalog % make image
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-X github.ibm.com/cloud-sre/version.Major= -X github.ibm.com/cloud-sre/version.Minor= -X github.ibm.com/cloud-sre/version.Build= -X github.ibm.com/cloud-sre/version.Timestamp=202303170049" -o osscatimporter ./cmd/osscatimporter
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-X github.ibm.com/cloud-sre/version.Major= -X github.ibm.com/cloud-sre/version.Minor= -X github.ibm.com/cloud-sre/version.Build= -X github.ibm.com/cloud-sre/version.Timestamp=202303170049" -o osscatpublisher ./cmd/osscatpublisher
jq: error: Could not open file metadata.json: No such file or directory
docker buildx build --platform=linux/amd64 -t api-osscatalog:latest .
[+] Building 153.2s (11/11) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 560B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker-virtual.artifactory.swg-devops.com/ubi8/ubi-minimal:8.7 34.2s
=> [auth] ubi8/ubi-minimal:pull token for docker-virtual.artifactory.swg-devops.com 0.0s
=> [1/5] FROM docker-virtual.artifactory.swg-devops.com/ubi8/ubi-minimal:8.7@sha256:65a240ad8bd3f2fff3e18a22ebadc40da0b145616231fc1e1625 91.3s
=> => resolve docker-virtual.artifactory.swg-devops.com/ubi8/ubi-minimal:8.7@sha256:65a240ad8bd3f2fff3e18a22ebadc40da0b145616231fc1e16251 0.0s
=> => sha256:0214a28336e387c66493c61bb394e86a18f3bea8dbc46de74a26f173ff553c89 429B / 429B 0.0s
=> => sha256:591670a5d6a620931ec51c1e7436300894f5320e76e9737f0366fc62114addd4 6.23kB / 6.23kB 0.0s
=> => sha256:d7c06497d5cebd39c0a4feb14981ec940b5c863e49903d320f630805b049cbff 39.28MB / 39.28MB 90.8s
=> => sha256:65a240ad8bd3f2fff3e18a22ebadc40da0b145616231fc1e16251f3c6dee087a 1.47kB / 1.47kB 0.0s
=> => extracting sha256:d7c06497d5cebd39c0a4feb14981ec940b5c863e49903d320f630805b049cbff 0.4s
=> [internal] load build context 0.2s
=> => transferring context: 23.77MB 0.2s
=> [2/5] RUN microdnf update && microdnf install procps ; 27.5s
=> [3/5] COPY osscatimporter / 0.0s
=> [4/5] COPY mailtemplate.tmpl / 0.0s
=> [5/5] COPY osscatpublisher / 0.0s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:8af2a24809c709c2ea80b1f3ed0c0d1dc1381c84219e0e779be43ab8542e8c0d 0.0s
=> => naming to docker.io/library/api-osscatalog:latest 0.0s
##############################
rohits-MacBook> catalog % docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
api-catalog latest 8af2a24809c7 9 seconds ago 140MB
######################################
rohits-MacBook> osscatalog % docker image inspect api-catalog
"Architecture": "arm64",
"Os": "linux",
"Size": 148693891,
"VirtualSize": 148693891,
Produces an arm64 image
########################################
Issue after running instance in its log file shows:
rohits-MacBook> api-osscatalog % kubectl -n api logs -f catimporter-rohit-gst45
exec /bin/sh: exec format error
rohits-MacBook> api-catalog %
Comment:
This usually happens when you’re working on projects on a system with ARM architecture, like with the new Apple M-series chipsets. When you push a code to your production environment, which is using an x86 system, it results in the “exec user process caused: exec format error”. This is because every piece of code when converted to the lower level of instructions is different for both ARM and x86. Docker detects the Apple M1 Pro platform as “linux/arm64/v8“.
################## SOLUTION ########################
/cloud-sre/catalog/Makefile
Change:
docker build -t $(IMAGE_NAME):latest .
docker buildx build --platform=linux/amd64 -t $(IMAGE_NAME):latest .
But for the above to work, need to change the Dockerfile also
/cloud-sre/catalog/Dockerfile
Change:
FROM docker-virtual.artifactory.swg-devops.com/ubi8/ubi-minimal:8.7
FROM --platform=linux/amd64 docker-virtual.artifactory.swg-devops.com/ubi8/ubi-minimal:8.7
#################### TEST ######################
rohits-MacBook> catalog % docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
Deleted all image
#####################################
rohits-MacBook> catalog %
rohits-MacBook> catalog % make image
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-X github.ibm.com/cloud-sre/version.Major= -X github.ibm.com/cloud-sre/version.Minor= -X github.ibm.com/cloud-sre/version.Build= -X github.ibm.com/cloud-sre/version.Timestamp=202303170049" -o osscatimporter ./cmd/osscatimporter
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-X github.ibm.com/cloud-sre/version.Major= -X github.ibm.com/cloud-sre/version.Minor= -X github.ibm.com/cloud-sre/version.Build= -X github.ibm.com/cloud-sre/version.Timestamp=202303170049" -o osscatpublisher ./cmd/osscatpublisher
jq: error: Could not open file metadata.json: No such file or directory
docker buildx build --platform=linux/amd64 -t api-osscatalog:latest .
[+] Building 153.2s (11/11) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 560B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker-virtual.artifactory.swg-devops.com/ubi8/ubi-minimal:8.7 34.2s
=> [auth] ubi8/ubi-minimal:pull token for docker-virtual.artifactory.swg-devops.com 0.0s
=> [1/5] FROM docker-virtual.artifactory.swg-devops.com/ubi8/ubi-minimal:8.7@sha256:65a240ad8bd3f2fff3e18a22ebadc40da0b145616231fc1e1625 91.3s
=> => resolve docker-virtual.artifactory.swg-devops.com/ubi8/ubi-minimal:8.7@sha256:65a240ad8bd3f2fff3e18a22ebadc40da0b145616231fc1e16251 0.0s
=> => sha256:0214a28336e387c66493c61bb394e86a18f3bea8dbc46de74a26f173ff553c89 429B / 429B 0.0s
=> => sha256:591670a5d6a620931ec51c1e7436300894f5320e76e9737f0366fc62114addd4 6.23kB / 6.23kB 0.0s
=> => sha256:d7c06497d5cebd39c0a4feb14981ec940b5c863e49903d320f630805b049cbff 39.28MB / 39.28MB 90.8s
=> => sha256:65a240ad8bd3f2fff3e18a22ebadc40da0b145616231fc1e16251f3c6dee087a 1.47kB / 1.47kB 0.0s
=> => extracting sha256:d7c06497d5cebd39c0a4feb14981ec940b5c863e49903d320f630805b049cbff 0.4s
=> [internal] load build context 0.2s
=> => transferring context: 23.77MB 0.2s
=> [2/5] RUN microdnf update && microdnf install procps ; 27.5s
=> [3/5] COPY osscatimporter / 0.0s
=> [4/5] COPY mailtemplate.tmpl / 0.0s
=> [5/5] COPY osscatpublisher / 0.0s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:8af2a24809c709c2ea80b1f3ed0c0d1dc1381c84219e0e779be43ab8542e8c0d 0.0s
=> => naming to docker.io/library/api-osscatalog:latest 0.0s
##############################################
rohits-MacBook> catalog % docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
api-catalog latest 8af2a24809c7 9 seconds ago 140MB
################################################
rohits-MacBook> catalog % docker image inspect api-catalog:latest
"Architecture": "amd64",
"Os": "linux",
"Size": 140425582,
— Which I used to get in my old laptop always.
############################################
Now checking the log from the new instance:
rohits-MacBook> api-ocatalog % kubectl -n api logs -f catimporter-rohit-9bzk9
INFO 19:53:29 osscatimporter Version: .. - 202303170049
INFO 19:53:29 sending *osscatimporter* Starting Run osscatimporter using the slack-webhook-url Slack webhook key.
Test COS connection passed: oss-rmc-data-test
Test COS connection passed: oss-rmc-emergency-test
AUDIT 19:53:30 Operating in read-only mode on all services and components found from all sources (with visibility=private and above)
INFO 19:53:30 Forcing -check-owner=true
INFO 19:53:30 Optional run action ENABLED: Services
INFO 19:53:30 Optional run action ENABLED: Tribes
INFO 19:53:30 Optional run action ENABLED: Environments
INFO 19:53:30 Optional run action ENABLED: Deployments
INFO 19:53:30 Optional run action ENABLED: Monitoring
INFO 19:53:30 Optional run action ENABLED: RMC
INFO 19:53:30 Optional run action ENABLED: RMC-Rescan
INFO 19:53:30 Optional run action DISABLED: Environments-Native
INFO 19:53:30 Optional run action DISABLED: ProductInfo-Parts
INFO 19:53:30 Optional run action DISABLED: ProductInfo-Parts-Refresh
INFO 19:53:30 Optional run action DISABLED: ProductInfo-ClearingHouse
INFO 19:53:30 Optional run action DISABLED: Dependencies-ClearingHouse
INFO 19:53:30 Optional run action DISABLED: ScorecardV1
INFO 19:53:30 Optional run action DISABLED: Doctor
INFO 19:53:30 Skip reloading Segment and Tribe Info from ScorecardV1
No Error
You should also be sure that what you're calling is indeed an executable. We made this mistake:
COPY --from=build /usr/src/service/serviceExec /bin/service
COPY --from=build /usr/config/file.txt /bin/service
RUN chmod a+x service
CMD ["./service"]
And the error was:
exec ./service: exec format error
The OS was not able to execute a txt file and threw a architecture-like error.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
