python - 数据库表不是 JSON 可序列化的

Question

我正在使用 Ibm Watson 在 Jupyter Notebook 中为机器学习项目设置一些 python 代码，当我尝试从我的 Postgresql 数据库表中添加数据时，我不断收到 TypeError is not JSON serializable。

为清楚起见，这里是完整的输出:

TypeError:                                           description
0   Lorem ipsum sjvh  hcx bftiyf,  hufcil, igfgvju...
1   Lorem ajjgvc wiufcfboitf iujcvbnb hjnkjc  ivjh...
2   Lorem aiv ibveikb jvk igvcib ok blnb v  hb b h...
3   Lorem jsvc smc cbd ciecdbbc d vd bcvdvbj obcvb...
4   Lorem jsvc smc cbd ciecdbbc d vd bcvdvbj obcvb...
5   Lorem jsvc smc cbd ciecdbbc d vd bcvdvbj obcvb...
6   Lorem jsvc smc cbd ciecdbbc d vd bcvdvbj obcvb...
7   Lorem jsvc smc cbd ciecdbbc d vd bcvdvbj obcvb...
8   Lorem jsvc smc cbd ciecdbbc d vd bcvdvbj obcvb...
9   Lorem jsvc smc cbd ciecdbbc d vd bcvdvbj obcvb...
10  Lorem sivbnogc hbiuygv bnjiuygv bmkjygv nmjhgv...  is not JSON serializable

这是我的 python 代码，它部署了 ML 模型来分析这些数据，如您所见，这些数据是随机句子，但在测试后将成为产品描述:

from watson_developer_cloud import NaturalLanguageClassifierV1
import pandas as pd
import psycopg2


# Connecting to my database.
conn_string = 'host={} port={}  dbname={}  user={}  password={}'.format('159.***.20.***', 5432, 'searchdb', 'lcq09', 'Mys3cr3tPass')
conn_cbedce9523454e8e9fd3fb55d4c1a52e = psycopg2.connect(conn_string)
data_df_1 = pd.read_sql('SELECT description from public."search_product"', con=conn_cbedce2drf563454e8e9fd3fb8776fgh2e)

# Connecting to the ML model.
natural_language_classifier = NaturalLanguageClassifierV1(
    iam_apikey='TB97dFv8Dgug6rfi945F3***************'
)

# Apply the ML model to db datas
classes = natural_language_classifier.classify('9841d0z5a1-ncc-9076', data_df_1)
print(json.dumps(classes, indent=2)) # Pretty sure the error is happening here

我该如何解决这个问题？

编辑 1:

我尝试打印此方法:print(data_df_1.to_json()) 以查看格式现在是否为 Json，这是我得到的输出:

{"description":{"0":"Lorem ipsum sjvh hcx bftiyf, hufcil, igfgvjuoigv gvj ifcil ,ghn fgbcggtc yfctgg h vgchbvju.","1":"Lorem ajjgvc wiufcfboitf iujcvbnb hjnkjc ivjhn oikgjvn uhn uhn 09iuvhb oiuvh boiuhb mkjhv mkiuhygv m，khbgv mkjhgv mkjhgv。”，“2”:“Lorem aiv ibveikb jvk igvcib ok blnb v hb b hb bnjb bhb bhn bn vf vbgfc vbgv nbhgv bb nb nbh nj mjhbv nmkjhb3”，“:"Lorem jsvc smc cbd ciecdbbc d vd bcvdvbj obcvb vcibs j dvx","4":"Lorem jsvc smc cbd ciecdbbc d vd bcvdvbj obcvb vcibs j dvx","5":"Lorem jsvc smc cbd ciecdbbc d vd bcvdvbcibj obc j dvx"}}

但是当我添加这个方法时我现在得到这个错误:classes = natural_language_classifier.classify('9841d0z5a1-ncc-9076', data_df_1.to_json()) print(json.dumps(classes, indent=2)) 到 classes:

TypeError                                 Traceback (most recent call last)
<ipython-input-16-e72fac39b809> in <module>()
      1 classes = natural_language_classifier.classify('998520s521-nlc-1398', data_df_1.to_json())
----> 2 print(json.dumps(classes, indent=2))

/opt/conda/envs/DSX-Python35/lib/python3.5/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    235         check_circular=check_circular, allow_nan=allow_nan, indent=indent,
    236         separators=separators, default=default, sort_keys=sort_keys,
--> 237         **kw).encode(obj)
    238 
    239 

/opt/conda/envs/DSX-Python35/lib/python3.5/json/encoder.py in encode(self, o)
    198         chunks = self.iterencode(o, _one_shot=True)
    199         if not isinstance(chunks, (list, tuple)):
--> 200             chunks = list(chunks)
    201         return ''.join(chunks)
    202 

/opt/conda/envs/DSX-Python35/lib/python3.5/json/encoder.py in _iterencode(o, _current_indent_level)
    434                     raise ValueError("Circular reference detected")
    435                 markers[markerid] = o
--> 436             o = _default(o)
    437             yield from _iterencode(o, _current_indent_level)
    438             if markers is not None:

/opt/conda/envs/DSX-Python35/lib/python3.5/json/encoder.py in default(self, o)
    177 
    178         """
--> 179         raise TypeError(repr(o) + " is not JSON serializable")
    180 
    181     def encode(self, o):

TypeError: <watson_developer_cloud.watson_service.DetailedResponse object at 0x7f64ee350240> is not JSON serializable

-

如您在代码中所见，我想将机器学习文本分类器部署到我数据库中的描述表。

我可以用下面的代码对单个句子进行分类，但我想对整个数据库的描述表进行分类:

classes = natural_language_classifier.classify('998260x551-nlc-1018', '今天会有多热？') 打印(json.dumps(classes.result, indent=2))

这就是为什么我用数据框 data_df_1 替换句子的原因。

Answer 1

您需要将表转换为特定的 JSON 格式并使用 classify_collection() 方法，如 official documentation 中的示例所示

import json
from ibm_watson import NaturalLanguageClassifierV1

natural_language_classifier = NaturalLanguageClassifierV1(
    iam_apikey='{apikey}',
    url='{url}')

classes = natural_language_classifier.classify_collection('10D41B-nlc-1', [{'text':'How hot will it be today?'}, {'text':'Is it hot outside?'}]).get_result()
print(json.dumps(classes, indent=2))