python - 将嵌套的 JSON 字符串展平到 Google BigQuery 中的不同列

Question

我在其中一个 BigQuery 表中有列，如下所示。

{"name": "name1", "last_delivered": {"push_id": "push_id1", "time": "time1"}, "session_id": "session_id1", "source": "SDK", "properties": {"UserId": "u1"}}

在 GBQ 中有没有得到这样的输出？？ (基本上把整列压平成不同的列)

name    last_delivered.push_id   last_delivered.time   session_id   source   properties.UserId

name1       push_id1                     time1         session_id1   SDK          uid1

假设

a = {"name": "name1", "last_delivered": {"push_id": "push_id1", "time": "time1"}, "session_id": "session_id1", "source": "SDK", "properties": {"UserId": "u1"}}

我曾尝试使用 json_normalize(a) 在 Pandas Python 中获得所需的输出，但每次尝试时都会出现以下错误

有谁知道我怎样才能得到我想要的输出。我错过了什么吗？？

任何帮助将不胜感激!!

Answer 1

以下示例适用于 BigQuery 标准 SQL

#standardSQL
WITH `project.dataset.table` AS (
  SELECT '{"name": "name1", "last_delivered": {"push_id": "push_id1", "time": "time1"}, "session_id": "session_id1", "source": "SDK", "properties": {"UserId": "u1"}}' col
)
SELECT 
  JSON_EXTRACT_SCALAR(col, '$.name') name,
  STRUCT(
    JSON_EXTRACT_SCALAR(col, '$.last_delivered.push_id') AS push_id,
    JSON_EXTRACT_SCALAR(col, '$.last_delivered.time') AS time
  ) last_delivered,
  JSON_EXTRACT_SCALAR(col, '$.session_id') session_id,
  JSON_EXTRACT_SCALAR(col, '$.source') source,
  STRUCT(
    JSON_EXTRACT_SCALAR(col, '$.properties.UserId') AS UserId
  ) properties
FROM `project.dataset.table`   

并按预期/要求产生结果

Row name    last_delivered.push_id  last_delivered.time session_id  source  properties.UserId    
1   name1   push_id1                time1               session_id1 SDK     u1