python - 超拉斯 : 'List' object has no attribute 'shape'

Question

我正在尝试从 TSV 文件中读取一些数据以与 Hyperas 一起使用，但无论我怎么做，我似乎都会遇到同样的错误:

Traceback (most recent call last):
  File "/path/to/cnn_search.py", line 233, in <module>
    trials=trials)
  File "~/miniconda3/lib/python3.6/site-packages/hyperas/optim.py", line 67, in minimize
    verbose=verbose)
  File "~/miniconda3/lib/python3.6/site-packages/hyperas/optim.py", line 133, in base_minimizer
    return_argmin=True),
  File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 312, in fmin
    return_argmin=return_argmin,
  File "~/miniconda3/lib/python3.6/site-packages/hyperopt/base.py", line 635, in fmin
    return_argmin=return_argmin)
  File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 325, in fmin
    rval.exhaust()
  File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 204, in exhaust
    self.run(self.max_evals - n_done, block_until_done=self.async)
  File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 178, in run
    self.serial_evaluate()
  File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 97, in serial_evaluate
    result = self.domain.evaluate(spec, ctrl)
  File "~/miniconda3/lib/python3.6/site-packages/hyperopt/base.py", line 840, in evaluate
    rval = self.fn(pyll_rval)
  File "~/temp_model.py", line 218, in keras_fmin_fnct
AttributeError: 'list' object has no attribute 'shape'

从我看到的其他问题来看，这个错误是由使用常规数组引起的，而应该使用 NumPy 数组。因此，我尝试在每一步将我正在读取的 TSV 转换为 NumPy 数组:

from hyperas import optim
...
import numpy as np
import csv

def data():
    dataPath="/path/to/fm.labeled.10m.txt"

    X = []
    Y = []
    with open(dataPath) as dP:
            reader = csv.reader(dP, delimiter="\t")
            for row in reader:

                    #skip the first two columns, and the last column is labels
                    X.append(np.array(row[2:-1]))

                    #labels
                    Y.append(row[-1])


    encoder = LabelBinarizer()
    Y_categorical = encoder.fit_transform(Y)

    #split data into test and train 
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y_categorical, test_size=0.25)

    X_train_np = np.array(X_train)
    X_test_np = np.array(X_test)

    Y_train_np = np.array([np.array(y) for y in Y_train])
    Y_test_np = np.array([np.array(y) for y in Y_test])

    return X_train_np, Y_train_np, X_test_np, Y_test_np

...
trials = Trials()
best_run, best_model = optim.minimize(model=model_name,
                                      data=data,
                                      algo=tpe.suggest,
                                      max_evals=numRuns,
                                      trials=trials)

我还认为有一种更有效的方法可以做到这一点，而无需创建如此多的中间数组 - 这会很棒，因为我将读取数百万行数据。

我做错了什么？

编辑:Hyperopt wiki描述试验。

Answer 1

您是否考虑过使用np.genfromtxt('your_file.tsv')？ 对于读取 csv 和 tsv 数据非常有效，我最近在这方面获得了很好的经验。此外，如果您需要更详细的答案，您可能应该提供有关您的具体问题的更多信息(数据类型、布局等)。