如何在Snowflake JDBC中执行批量更新

3 人关注

我试图执行一个雪花批处理更新的PreparedStatement,但没有运气。 我能够执行每个单独的查询(preparedStatement.executeQuery()),所以查询工作。 但是,一旦我切换到 batchUpdate(),我得到一个异常,似乎它没有设置我的变量(被抛在 insertEventStatement.executeBatch())。

net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error: error line 1 at position 134
Bind variable ? not set.
at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:88)
at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:384)
at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:421)
at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:240)
at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:180)
at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:152)
at net.snowflake.client.core.SFStatement.execute(SFStatement.java:637)
at net.snowflake.client.jdbc.SnowflakeStatementV1.executeUpdateInternal(SnowflakeStatementV1.java:135)
at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.executeBatch(SnowflakePreparedStatementV1.java:1045)

有人知道我可能错过了什么吗?

public void insertBotEvents(List<SnowflakeRecord> snowflakeRecords) throws SQLException
        PreparedStatement insertEventStatement = connection.prepareStatement(INSERT_EVENT);
        PreparedStatement insertMetaEventStatement = connection.prepareStatement(INSERT_META_EVENT);
        for (SnowflakeRecord snowflakeRecord : snowflakeRecords)
            insertEventStatement.setString(1, snowflakeRecord.getUserId());
            insertEventStatement.setString(2, snowflakeRecord.getSessionId());
            insertEventStatement.setString(3, snowflakeRecord.getEventTime().toString());
            insertEventStatement.setString(4, snowflakeRecord.getEventType());
            insertEventStatement.setString(5, snowflakeRecord.getEventValue());
            insertEventStatement.setString(6, snowflakeRecord.getAdditionalInfo());
            insertEventStatement.setString(7, snowflakeRecord.getAffiliateName());
            System.out.println("\nINSERTING NEW RECORD: \n" + snowflakeRecord);
            insertEventStatement.addBatch();
            if (snowflakeRecord.getSmVuiEventDto() != null && snowflakeRecord.getSmVuiEventDto().getParameters() != null)
                for (ParametersDto parametersDto : snowflakeRecord.getSmVuiEventDto().getParameters())
                    insertMetaEventStatement.setString(1, snowflakeRecord.getSmVuiEventDto().getName());
                    insertMetaEventStatement.setString(2, parametersDto.getName());
                    insertMetaEventStatement.setString(3, parametersDto.getValue());
                    insertMetaEventStatement.setString(4, snowflakeRecord.getUserId());
                    insertMetaEventStatement.setString(5, snowflakeRecord.getSessionId());
                    insertMetaEventStatement.setString(6, snowflakeRecord.getEventTime().toString());
                    insertMetaEventStatement.setString(7, snowflakeRecord.getEventType());
                    insertMetaEventStatement.setString(8, snowflakeRecord.getEventValue());
                    insertMetaEventStatement.setString(9, snowflakeRecord.getAdditionalInfo());
                    insertMetaEventStatement.addBatch();
        insertEventStatement.executeBatch();
        insertMetaEventStatement.executeBatch();
        System.out.println("\nCommitting changes to snowflake.");
        connection.commit();
    catch (Exception e)
        connection.rollback();
        System.err.println("Unable to execute SQL. Rolling back snowflake import");
        e.printStackTrace();
        throw e;

以下是查询结果。

private static final String INSERT_EVENT =
        "INSERT INTO EVENT " +
                "(USER_ID, " +
                " SESSION_ID, " +
                " EVENT_DATETIME, " +
                " EVENT_TYPE, " +
                " EVENT_VALUE, " +
                " ADDITIONAL_INFO, " +
                " AFFILIATE_ID " +
                ") " +
                "SELECT " +
                " ?, " +
                " ?, " +
                " ?, " +
                " ?, " +
                " ?, " +
                " ?, " +
                " AFFILIATE_ID " +
                "FROM AFFILIATE_LOOKUP " +
                "WHERE AFFILIATE_NAME = ?";
private static final String INSERT_META_EVENT =
        "INSERT INTO EVENT_META " +
        "(EVENT_ID, " +
        "   META_TYPE, " +
        "   META_NAME, " +
        "   META_VALUE " +
        ") " +
        "SELECT EVENT_ID, ?, ?, ? " +
        "FROM SM_VUI_EVENT " +
        "WHERE USER_ID=? AND " +
        "   SESSION_ID=? AND " +
        "   EVENT_DATETIME=? AND " +
        "   EVENT_TYPE=? AND " +
        "   EVENT_VALUE=? AND " +
        "   ADDITIONAL_INFO=?";

编辑:显然,在我使用的JDBC版本(3.0.18)中没有这个功能,但在最近的版本(3.0.21)中增加了这个功能。 如果不成功,我将进一步更新。

jdbc
snowflake-cloud-data-platform
K. Wheeler
K. Wheeler
发布于 2017-06-15
2 个回答
Howard Yu
Howard Yu
发布于 2017-06-16
0 人赞同

刚刚确认,Snowflake目前不支持在包含子查询的INSERT语句中提供一个绑定数组,但我们正在努力解决。

谢谢你的信息。你知道那是什么时候发布的吗?我目前没有太多的数据被发送。但是,我预计很快会有大量的数据,根据我所做的测试,单独执行查询是不可行的。 或者有什么更好的方法让我上传这些数据?
它应该在下周发布。然而,其实现仍然是客户端的循环,并单独执行查询。如果你想加载大量的数据,你应该考虑复制命令。
Howard Yu
Howard Yu
发布于 2017-06-16
0 人赞同