echo 'startdate: '$startdate echo 'enddate: '$enddate echo "-----------------------------------" #序列1-300,表示遍历300次,因为有结束时间的限制,所以实际上不会循环300次 for i in `seq 1 300`; do #当开始时间小于结束时间时,直接结束脚本 if [[ $startdate -lt $enddate ]]; then break echo $startdate #执行hiveSQL脚本,我是需要按日期执行hiveSQL,这里可以无视 hive -hiveconf input_date=$startdate -f mergeSmallFiles.sql # >> mergeSmallFiles.txt #每次执行后,使开始日期减一天,如果要正序,将下面-1换成+1即可,当然开始时间和结束时间也要换一下 startdate=$(date -d "$startdate -1 day" +%Y%m%d)

运行结果如下:

2. 按小时倒序遍历日期 ( 正序修改一下日期增减函数即可 )

因为shell中没有小时加减的函数,所以这里要做几个步骤:

#!/bin/bash
#倒序按小时遍历时间
startdate="$1"
enddate="$2"
echo 'startdate: '$startdate
echo 'enddate: '$enddate
echo "-----------------------------------"
for i in `seq 1 300`; do
  if [[ $startdate -lt $enddate ]]; then
    break
  echo $startdate
  #拼接成YYYYmmdd HH的格式,否则使用下个命令中+%s,会无法转换
  startdate=${startdate:0:4}"-"${startdate:4:2}"-"${startdate:6:2}" "${startdate:8:2}
  #转换成时间戳,+%s为时间格式
  startdate=$(date +%s -d "$startdate")
  #每次执行后,开始日期减一个小时
  startdate=$(($startdate-1*60*60))
  #时间戳转正常格式时间
  startdate=$(date +%Y%m%d%H -d "1970-01-01 UTC $startdate seconds")  
  #执行hiveSQL脚本,可以无视掉
  hive -hiveconf input_date=$startdate -f mergeSmallFiles.sql # >> mergeSmallFiles.txt

运行结果如下:

通常在修改完数据处理的程序后,需要回溯之前若干天的数据,这时候一个日期遍历shell脚本就派上用场了。基本代码简述如下:#!/bin/bashSTARTDAY=20160316 ENDDAY=20160508DAY=$STARTDAY while [[ $DAY < $ENDDAY ]] CMD="echo $DAY" # build any cmd as u want while (( $firstDay < $endDay )) beeline -e "select * from dw.dw_doc_phone_$firstDay " > $firstDay.csv firstDay=`date -d "+1 day $firstDay" +%
#echo $first let first=`date -d "-1 days ago ${first}" +%Y%m%d` echo `date -d "${first}" +%Y-%m-%d` 假设将上述的内容放到名为test.sh的脚本中...
Shell执行任务时有时需要对hive表做遍历操作并按照dt进行分区,dt数较少时可以采用for循环,但有时需要对一个时间段内的日期进行遍历,这时候就可以使用while对日期进行遍历。 下面分别采用for循环和while循环对日期进行遍历: For循环: for i in {20190801,20190802,20190803} echo ${i} While...
qq_15684959: 遇到了一样的问题,删除catalog后还是一直在访问oracle,提了issue,给这么回答的When creating a Jdbc Catalog, a connection check is performed by default. This is performed on a random FE and BE. If the password is incorrect, the creation of the Catalog will be prohibited. This avoids frequent attempts to use the wrong password on the source library. If the password of the source library is changed midway, it is recommended to stop using or delete the current Catalog first, and then change the password of the Catalog or create the Catalog with the new password after changing the source password. This is to avoid the connection pool constantly trying to connect with the old password to keep the connection alive during the password change process. ,还是建议升级到较新的发行版分支上。 泰勒公式(泰勒展开式)通俗+本质详解 夏末未殇: 可以对sinx展开,然后把所有x带入x方