使用Presto设置EMR中JDBC连接的基本密码身份验证开发者社区

相关文章推荐

叛逆的充电器 · Jenkins+Sonar质量门禁【实践篇- ...· 2 周前 ·

率性的麦片 · github star history-掘金· 1 年前 ·

无邪的消防车 · 【免费】centos7.8,7.9升级ope ...· 2 年前 ·

有腹肌的大白菜 · Delphi的字符串与16进制的相互转换函数 ...· 2 年前 ·

爱搭讪的木瓜 · Pytorch调用预训练模型输出结果时报错a ...· 2 年前 ·

多情的大象 · 求一个比较复杂的SQL语句,字段值作为列名的方法· 2 年前 ·

腾讯云

开发者社区

文档意见反馈控制台

首页 TVP

文章/答案/技术大牛

发布

    {
        classification: 'spark-hive-site',
        configurationProperties: {
            'hive.metastore.client.factory.class': 'com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory',
        classification: 'emrfs-site',
        configurationProperties: {
            'fs.s3.maxConnections': '5000',
            'fs.s3.maxRetries': '200',
        classification: 'presto-connector-hive',
        configurationProperties: {
            'hive.metastore.glue.datacatalog.enabled': 'true',
            'hive.parquet.use-column-names': 'true',
            'hive.max-partitions-per-writers': '7000000',
            'hive.table-statistics-enabled': 'true',
            'hive.metastore.glue.max-connections': '20',
            'hive.metastore.glue.max-error-retries': '10',
            'hive.s3.use-instance-credentials': 'true',
            'hive.s3.max-error-retries': '200',
            'hive.s3.max-client-retries': '100',
            'hive.s3.max-connections': '5000',
    },

configurations: [
        classification: 'spark-hive-site',
        configurationProperties: {
            'hive.metastore.client.factory.class': 'com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory',
        classification: 'emrfs-site',
        configurationProperties: {
            'fs.s3.maxConnections': '5000',
            'fs.s3.maxRetries': '200',
        classification: 'presto-connector-hive',
        configurationProperties: {
            'hive.metastore.glue.datacatalog.enabled': 'true',
            'hive.parquet.use-column-names': 'true',
            'hive.max-partitions-per-writers': '7000000',
            'hive.table-statistics-enabled': 'true',
            'hive.metastore.glue.max-connections': '20',
            'hive.metastore.glue.max-error-retries': '10',
            'hive.s3.use-instance-credentials': 'true',
            'hive.s3.max-error-retries': '200',
            'hive.s3.max-client-retries': '100',
            'hive.s3.max-connections': '5000',
        classification: 'trino-config',
        configurationProperties: {
            'query.max-memory-per-node': `${instanceMemory * 0.15}GB`, // 25% of a node
            'query.max-total-memory-per-node': `${instanceMemory * 0.5}GB`, // 50% of a node
            'query.max-memory': `${instanceMemory * 0.5 * coreInstanceGroupNodeCount}GB`, // 50% of the cluster
            'query.max-total-memory': `${instanceMemory * 0.8 * coreInstanceGroupNodeCount}GB`, // 80% of the cluster
            'query.low-memory-killer.policy': 'none',
            'task.concurrency': vcpuCount.toString(),
            'task.max-worker-threads': (vcpuCount * 4).toString(),
            'http-server.authentication.type': 'PASSWORD',
            'http-server.http.enabled': 'false',
            'internal-communication.shared-secret': 'abcdefghijklnmopqrstuvwxyz',
            'http-server.https.enabled': 'true',
            'http-server.https.port': '8443',
            'http-server.https.keystore.path': '/home/hadoop/fullCert.pem',
        classification: 'trino-password-authenticator',
        configurationProperties: {
            'password-authenticator.name': 'file',
            'file.password-file': '/home/hadoop/password.db',
            'file.refresh-period': '5s',
            'file.auth-token-cache.max-size': '1000',
],

'http-server.https.enabled': 'true',
                'http-server.https.port': '8443',
                'http-server.https.keystore.path': '/home/hadoop/fullCert.pem',

[hadoop@ip-10-0-10-245 ~]$ trino-cli --server https://localhost:8446 --catalog awsdatacatalog --user hadoop --password --insecure
trino> select 1;
Query 20220701_201620_00001_9nksi failed: Insufficient active worker nodes. Waited 5.00m for at least 1 workers, but only 0 workers are active

2022-07-01T21:30:12.966Z        WARN    http-client-node-manager-51     io.trino.metadata.RemoteNodeState       Error fetching node state from https://ip-10-0-10-245.ec2.internal:8446/v1/info/state: Failed communicating with server: https://ip-10-0-10-245.ec2.internal:8446/v1/info/state
2022-07-01T21:30:13.902Z        ERROR   Announcer-0     io.airlift.discovery.client.Announcer   Service announcement failed after 8.11ms. Next request will happen within 1000.00ms
2022-07-01T21:30:14.913Z        ERROR   Announcer-1     io.airlift.discovery.client.Announcer   Service announcement failed after 10.35ms. Next request will happen within 1000.00ms
2022-07-01T21:30:15.921Z        ERROR   Announcer-3     io.airlift.discovery.client.Announcer   Service announcement failed after 8.40ms. Next request will happen within 1000.00ms
2022-07-01T21:30:16.930Z        ERROR   Announcer-0     io.airlift.discovery.client.Announcer   Service announcement failed after 8.59ms. Next request will happen within 1000.00ms