Collectives™ on Stack Overflow
Find centralized, trusted content and collaborate around the technologies you use most.
Learn more about Collectives
Teams
Q&A for work
Connect and share knowledge within a single location that is structured and easy to search.
Learn more about Teams
I am using odo library to transfer a
pandas dataframe
to S3. However I am getting following error :
import pandas as pd
df = pd.DataFrame([[1, 2], [3, 4], [5, 6], [7, 8]], columns=["A", "B"])
odo(df,'s3://path_to_s3_folder')
S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Anonymous access is forbidden for this operation</Message><RequestId>F5958774D56AD29E</RequestId><HostId>zOH8JOxpSgB5Scgc/YrtHO1+e9lXoKAF89IhRSeAiSoGHAJxyjXKBVFIYETeO4gSLZOUgXmwKLM=</HostId></Error>
Now I have the AWS credentials setup correctly as I can see in my ~/.aws/credentials
file
cat credentials
[default]
aws_access_key_id = XXXXX
aws_secret_access_key = XXXXXXXXXX
The aws cli works correctly for me and I can run aws ls
and cp
commands correctly (I guess this means I do have the required permissions).
aws s3 ls s3://path_to_s3
Also boto3
is able to access s3 resources and does not give an error.
import boto3
s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
print(bucket.name)
What would be possible wrong/missing in the configuration ?
–
As pointed by @mootmoot, odo
uses boto
and not boto3
.
boto3
can get aws credentials from ~/.aws/credentials
as stated here. However for boto
you need to setup env variable as stated here
Just add,
$ export AWS_ACCESS_KEY_ID=XXXXX
$ export AWS_SECRET_ACCESS_KEY=XXXXXXXXXX
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.