联合登录助手(aws-google-auth) ( Federated Login Helper (aws-google-auth) )

aws-google-auth is an authentication helper Python package, offered by CEVO (Docker also available at git repo). It is invoked via command line using docker and can be used to generate STS credentials using your Federated Google account. The package can also be installed locally to your existing Python environment, or via building the docker.

aws-google-auth 是 CEVO 提供的身份验证帮助Python软件包(Docker也可在git repo上获得)。它使用docker通过命令行调用，并可用于使用您的联合Google帐户生成STS凭据。该软件包还可以本地安装到您现有的Python环境中，或通过构建docker安装。

选项1：本地Python安装 ( Option 1: Local Python Installation )

If you prefer to use the your local Python installation, then install aws-google-auth via pip:

如果您希望使用本地Python安装，请通过pip安装 aws-google-auth ：

$ pip install aws-google-auth

选项2：Docker安装 ( Option 2: Docker Installation )

Clone CEVO git repo,

克隆CEVO git repo，

$ git clone https://github.com/cevoaustralia/aws-google-auth \
aws-google-auth

and build the image using:

并使用以下命令构建图像：

$ cd aws-google-auth
$ docker build --rm -t aws-google-auth .

The docker image is used for the examples throughout the article.

本文中的示例均使用docker映像。

查找Google身份和SAML提供者ID(IDPID / SPID) ( Locating Google Identity and SAML Provider IDs (IDPID/SPID) )

Before being able to use the image, you will need to know the following details :

在使用图像之前，您需要了解以下详细信息：

Google Identity Provider ID ( idpid )

Google身份提供商ID( idpid )
SAML Service Provider ID assigned by Google (spid)

由Google分配的SAML服务提供商ID (spid)

Your AWS System admins should provide these details. If they are not provided, then you should be able to find these through the Google App AWS link address details.

您的AWS系统管理员应提供这些详细信息。如果未提供它们，那么您应该可以通过Google App AWS链接地址详细信息找到它们。

From your web browser, go to Google Apps launcher. Scroll until you see the AWS Google App…. hover your mouse over the icon to view the link address details in your browser’s status bar. Alternatively you can right click on the icon and copy/paste the link address into Notepad.

在网络浏览器中，转到Google Apps启动器。滚动直到看到AWS Google App…。将鼠标悬停在该图标上可以查看浏览器状态栏中的链接地址详细信息。或者，您可以右键单击该图标，然后将链接地址复制/粘贴到记事本中。

Make a note of your idpid and spid . They are required as parameter inputs to aws-google-auth .

记下您的 idpid 和 spid 。必须将它们作为 aws-google-auth 参数输入。

生成STS临时凭证 ( Generate STS Temporary Credentials )

The aws-google-auth helper stores/adds authentication data to $HOME/.aws/config and $HOME/.aws/credentials files so it's important that the storage hosting these files is encrypted at rest.

aws-google-auth 帮助程序将身份验证数据存储/添加到 $HOME/.aws/config 和 $HOME/.aws/credentials 文件中，因此托管这些文件的存储在 $HOME/.aws/credentials 进行加密非常重要。

The Docker can be run with a host:container mount $HOME/.aws:/root/.aws . This will allow direct access to files $HOME/.aws/config & $HOME/.aws/credentials . Alternatively, you can setup an alternate host location and have the container generate config/credentials into this location.

Docker可以与 host:container mount $HOME/.aws:/root/.aws 。这将允许直接访问文件 $HOME/.aws/config 和 $HOME/.aws/credentials 。或者，您可以设置备用主机位置，并使容器在该位置生成配置/凭据。

To directly edit your default credentials/config files, then specify the following mount option in your docker command:

要直接编辑默认凭据/配置文件，然后在docker命令中指定以下安装选项：

Linux-based hosts -v $HOME/.aws:/root/.aws

基于Linux的主机 -v $HOME/.aws:/root/.aws
Windows host -v c:/Users/<username>/.aws:/root/.aws

Windows主机 -vc:/Users/<username>/.aws:/root/.aws

运行Docker映像 ( Run the Docker Image )

To get a full list of command line options for aws-google-auth , run the docker with -h option.

要获取 aws-google-auth 的命令行选项的完整列表，请使用 -h 选项运行docker。

For the examples in the article, the following options are used,

对于本文中的示例，使用以下选项，

$ docker run -it aws-google-auth -h
..
...
-u USERNAME  (Google Apps username)
-I IDPID     (Google Identity Provider identifier)
-S SPID      (SAML Provider identifier )
-R REGION    (AWS region endpoint)
-d DURATION  (Credential duration in SECONDS)
-p PROFILE   (AWS profile - defaults to value of $AWS_PROFILE)
-r ROLE_ARN  (The ARN of the role to assume.  If not supplied
              ,a list roles to choose from will be listed)
..
...

with the following sample values,

具有以下示例值，

username=eg@myexample.com
idpid=X7x0Xxxx
spid=000111000111
region=us-east-1
duration=3600
profile=sts-temp
role_arn=arn:aws:iam::111222333444:role/saml-init

Using these values, we generate STS temporary credentials as follows:

使用这些值，我们将生成STS临时凭证，如下所示：

$ docker run -it -v $HOME/.aws:/root/.aws aws-google-auth \
-u eg@myexample.com \
-I X7x0Xxxx \
-S 000111000111 \
-R us-east-1 \
-d 3600 \
-p sts-temp \
-r arn:aws:iam::111222333444:role/saml-init

For MFA with SMS verification enabled, you will be required to enter your Google password and respond with the SMS code.

对于启用了 SMS验证的 MFA，您需要输入Google密码并以SMS代码进行回复。

Google Password: *******
Enter SMS token: G-nnnnnn

After authentication is complete, the expiration time for temporary credentials should be displayed

身份验证完成后，应显示临时凭证的到期时间

Assuming arn:aws:iam::111222333444:role/saml-init
Credentials Expiration: 2020-07-12 00:41:15+00:00

The AWS profile, sts-temp ( -p sts-temp ) should now contain the STS temporary credentials.

AWS配置文件 sts-temp ( -p sts-temp )现在应包含STS临时凭证。

验证AWS Config和凭证 ( Verify AWS Config & Credentials )

The credentials and config files can be checked to ensure profile sts-temp has been updated/generated with valid entries.

可以检查 credentials 和 config 文件，以确保已使用有效条目更新/生成配置文件 sts-temp 。

$HOME/.aws/config

[profile sts-temp]
region = us-east-1
google_config.ask_role = False
google_config.keyring = False
google_config.duration = 3600
google_config.google_idp_id = X7x0Xxxx
google_config.role_arn = arn:aws:iam::111222333444:role/saml-init
google_config.google_sp_id = 000111000111
google_config.u2f_disabled = False
google_config.google_username = eg@myexample.com
google_config.bg_response = None

$HOME/.aws/credentials

[sts-temp]
aws_access_key_id = exampleAAAAAAAAAAAAAAAAAAAAAA
aws_secret_access_key = exampleBBBBBBBBBBBBBBBBBBBBB
aws_security_token = exampleCCCCCCCCCCCCCCCCCC
aws_session_expiration = exampleDDDDDDDDDDDDDDDDDDDDDD
aws_session_token = exampleEEEEEEEEEEEEEEEEEEEE

刷新AWS STS凭证 ( Refreshing AWS STS Credentials )

To refresh/update STS credentials for the same profile is only a matter of running the docker command again, supplying the same profile name ( -p sts-temp ).

要刷新/更新同一配置文件的STS凭据，只需再次运行docker命令，并提供相同的配置文件名称( -p sts-temp )。

使用STS凭据切换/承担辅助角色 ( Switch to/Assume Secondary Role using STS Credentials )

As mentioned earlier on, to perform Dev activities, we need to assume/switch to secondary role arn:aws:iam::888777666555:role/assumed-dev using the STS temporary credentials.

如前所述，要执行开发活动，我们需要使用STS临时凭据承担/切换到辅助角色 arn:aws:iam::888777666555:role/assumed-dev 。

This secondary role has been provisioned with access to an Elastic Container Registry (ECR) and S3. To use the temporary STS credentials with this role, we need to create a new AWS profile, specifying the role_arn .

已为该次要角色提供了对弹性容器注册表(ECR)和S3的访问权限。要使用具有此角色的临时STS凭证，我们需要创建一个新的AWS配置文件，并指定 role_arn 。

The following examples show the AWS profile being created with name dev-env for our secondary role.

以下示例显示了以次要角色名称 dev-env 创建的AWS概要文件。

$HOME/.aws/config

[profile dev-env]
region = us-east-1
source_profile = sts-temp
role_arn = arn:aws:iam::888777666555:role/assumed-dev

If you prefer, you can use the aws configure command to setup the new AWS profile.

如果愿意，可以使用 aws configure 命令来设置新的AWS配置文件。

$ aws configure set profile.dev-env.region us-east-1
$ aws configure set profile.dev-env.source_profile sts-temp
$ aws configure \
set profile.dev-env.role_arn \ arn:aws:iam::888777666555:role/assumed-dev

When the profile above is specified in AWS CLI commands ( --profile dev-env ), the secondary role ( arn:aws:iam::888777666555:role/assumed-dev ) is "assumed" with the STS temporary credentials being sourced from section [sts-temp] in $HOME/.aws/credentials .

当在AWS CLI命令( --profile dev-env )中指定了上面的配置文件时，辅助角色( arn:aws:iam::888777666555:role/assumed-dev )被“假定”，并且STS临时凭证来自 $HOME/.aws/credentials [sts-temp] 部分。

全部放在一起 ( Putting it all Together )

The following is a re-cap of the process.

以下是该过程的总结。

1.使用联合用户进行身份验证并检索STS凭据 ( 1. Authenticate with Federated User and Retrieve STS Credentials )

Authenticate with Federated Google account using aws-google-auth and obtain STS temporary credentials for profile sts-temp .

使用 aws-google-auth 使用联合Google帐户进行 aws-google-auth 并获取配置文件 sts-temp STS临时凭据。

$ docker run -it -v $HOME/.aws:/root/.aws aws-google-auth \
-u eg@myexample.com \
-I X7x0Xxxx \
-S 000111000111 \
-R us-east-1 \
-d 3600 \
-p sts-temp \
-r arn:aws:iam::111222333444:role/saml-init

2.将新的AWS Profile配置为辅助角色以使用STS ( 2. Configure New AWS Profile for Secondary Role to use STS )

Create new AWS profile for the secondary role
为辅助角色创建新的AWS配置文件

The below shows the setup required for profile dev-env

下面显示了概要文件 dev-env 所需的设置

$HOME/.aws/config

[profile dev-env]
region = us-east-1
source_profile = sts-temp
role_arn = arn:aws:iam::888777666555:role/assumed-dev

or, alternatively, add the profile to $HOME/.aws/config via aws configure .

或者，通过 aws configure 将配置文件添加到 $HOME/.aws/config 。

$ aws configure set profile.dev-env.region us-east-1
$ aws configure set profile.dev-env.source_profile sts-temp
$ aws configure set profile.dev-env.role_arn \ arn:aws:iam::888777666555:role/assumed-dev

3.开始使用CLI命令 ( 3. Start Using CLI Commands )

The examples that follow show commands which will transparently “assume” the secondary role arn:aws:iam::888777666555:role/assumed-dev .

以下示例显示了将透明地“假定”辅助角色 arn:aws:iam::888777666555:role/assumed-dev 。

S3 list bucket subdir/key
S3列表存储区子目录/密钥

$ aws s3 ls s3://mybucket/mydir --profile dev-env

ECR get docker registry login, get-login
ECR获取Docker注册表登录，获取登录

$ aws ecr get-login --registry-ids xxxxxxxxxxxx --no-include-email --profile dev-env

4.更新/刷新过期的STS凭据 ( 4. Update/Refresh Expired STS credentials )

Once the STS temporary credentials have expired, re-execute aws-google-auth to refresh,

STS临时凭证过期后，请重新执行 aws-google-auth 进行刷新，

$ docker run -it -v $HOME/.aws:/root/.aws aws-google-auth \
-u eg@myexample.com \
-I X7x0Xxxx \
-S 000111000111 \
-R us-east-1 \
-d 3600 \
-p sts-temp \
-r arn:aws:iam::111222333444:role/saml-init

and continue using the profile for the secondary role.

并继续使用配置文件作为次要角色。

使用Python和Boto进行编程访问 ( Programmatic Access with Python & boto )

The below sample Python code shows the secondary role being “assumed” by specifying profile dev-env . Authentication happens transparently due to the configuration performed earlier on for this profile.

以下示例Python代码通过指定配置文件 dev-env 展示了“承担”的次要角色。由于此配置文件先前已执行了配置，因此透明地进行身份验证。

import boto3

session = boto3.Session(profile_name='dev-env')
dev_s3_client = session.client('s3')

for key in dev_s3_client.list_objects(Bucket='mybucket')['Contents']:
    print(key['Key'])

最后评论 ( Final Comments )

Some of the benefits of enabling Federated logins with temporary credentials are as follow:

使用临时凭证启用联合登录的一些好处如下：

Removing the user’s account from the linked Identity Provider/Organisation (eg. Google GSuite) ensures the user no longer has access to the AWS account.
从链接的身份提供商/组织(例如Google GSuite)中删除用户的帐户可确保用户不再有权访问AWS帐户。
The use of temporary credentials with a short expiration period (as opposed to issuing non-expiring keys), enforces the concept of key rotation.
使用具有较短到期期限的临时凭证(与颁发非到期密钥相反)可以实施密钥轮换的概念。
Managing comprised temporary credentials is likely to be simpler than managing comprised permanent keys.
管理包含的临时证书可能比管理包含的永久密钥更简单。

翻译自: https://medium.com/@anthony.f.tannous/aws-sts-credentials-and-google-apps-federated-user-9b3e1bf64585

aws联合用户

aws联合用户This write-up outlines methods of working with the AWS Secure Token Service (STS) and Federated user accounts, where Google has been established as the Identity Provider. It is based on a recen... 网站：www.ilove aws .cn Hello大家好，欢迎来到《 AWS 解决方案架构师认证 Professional(SAP)中文视频培训课程》，我们今天的课程还是 AWS STS 服务的内容， AWS STS – 获取临时凭证的自动化执行。我们开始今天的课程内容。我们上节课讲了一个用户案例，通过将...

官方文档：https://docs.amazon aws .cn/IAM/latest/UserGuide/id_credentials_temp_request.html 描述一下这篇文章的背景：使用cognito 控制 app 接入： APP 通过cognito服务的开发者授权验证方式接入到的cognito服务身份池。通过userid 作为开发人员标识换到cognito indentity pool 的唯一标识 indentityid。而每一个通过验证的用户都可以用身份映射到一个 AWS IAM

使用Spark构建 AWS 数据湖时可能会遇到以下问题： 1. 数据格式不一致：在构建数据湖时，数据来自不同的来源，可能会有不同的格式。这可能会导致Spark无法正确解析数据。解决方法是使用Spark的Schema推断功能来自动推断数据格式，或手动指定Schema。 2. 数据质量问题：数据湖中的数据可能存在质量问题，如缺失值、异常值等。可以使用Spark的数据清洗功能来处理这些问题。 3. 数据量过大：数据湖中的数据量可能非常大，可能会导致Spark的性能问题。可以使用Spark的分布式计算功能来处理大规模数据。 4. 数据安全问题：数据湖中的数据可能包含敏感信息，需要进行安全保护。可以使用 AWS 的安全服务来保护数据湖中的数据。 5. 数据管理问题：数据湖中的数据可能需要进行管理，如备份、恢复、版本控制等。可以使用 AWS 的数据管理服务来管理数据湖中的数据。处理这些问题的方法包括使用Spark的功能和 AWS 的服务来解决。同时，需要根据具体情况进行调整和优化，以确保数据湖的稳定和可靠性。