Friday, April 19, 2019

migrating big query TABLE_DATE_RANGE equivalent in standard sql

To query a big query table with date time function called  TABLE_DATE_RANGE into standard  sql
like for  example if you run this  :
FROM (TABLE_DATE_RANGE([hd-cust-account-prod:exported_LB_logs_v2.requests_],DATE_ADD(CURRENT_TIMESTAMP(), -10, 'MINUTE'),CURRENT_TIMESTAMP()))
you will get the error "Table-valued function not found: TABLE_DATE_RANGE" 



To fix this you  can use the following in standard sql to get the last x minutes in big query using a combination of _TABLE_SUFFIX and TIMESTAMP_SUB like so:

SELECT *
FROM `hd-gcpproject-prod.someapptable_output_log_*`
WHERE timestamp BETWEEN
  TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 MINUTE) AND
  CURRENT_TIMESTAMP() and PARSE_DATE('%Y%m%d', _TABLE_SUFFIX) BETWEEN
  DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) AND
  CURRENT_DATE();