我想从 map<string, int>
类型的 cassandra 列族读取数据并想将其转换为 Pandas 数据框。我还想用它来训练 python 中的模型,如上所述 here鸢尾花种类的分类。
如果,我会使用 csv 来训练模型。那么它会看起来像这样:
label, f1, f2, f3, f4, f5
0 , 11 , 1, 6 , 1, 2
1 , 5, 5, 1 , 2, 6
0 , 12, 9, 3 , 6, 8
0 , 9, 3, 8, 1, 0
Cassandra 列族:
FeatureSet | label
{'f1': 11, 'f2': 1, 'f3': 6, 'f4': 1, 'f5': 2} | 0
{'f1': 5, 'f2': 5, 'f3': 1, 'f4': 2, 'f5': 6} | 1
{'f1': 12, 'f2': 9, 'f3': 3, 'f4': 6, 'f5': 8} | 0
{'f1': 9, 'f2': 3, 'f3': 8, 'f4': 1, 'f5': 0} | 0
代码:
import pandas as pd
from sklearn2pmml import PMMLPipeline
from sklearn.tree import DecisionTreeClassifier
from cassandra.cluster import Cluster
CASSANDRA_HOST = ['172.16.X.Y','172.16.X1.Y1']
CASSANDRA_PORT = 9042
CASSANDRA_DB = "KEYSPACE"
CASSANDRA_TABLE = "COLUMNFAMILY"
cluster = Cluster(contact_points=CASSANDRA_HOST, port=CASSANDRA_PORT)
session = cluster.connect(CASSANDRA_DB)
sql_query = "SELECT * FROM {}.{};".format(CASSANDRA_DB, CASSANDRA_TABLE)
df = pd.DataFrame()
for row in session.execute(sql_query):
What should i write here and get X_train, Y_train in pandas dataframe
iris_pipeline = PMMLPipeline([
("classifier", DecisionTreeClassifier())
])
iris_pipeline.fit(X_train, Y_train)
最佳答案
您可以使用this approach :
import pandas as pd
from cassandra.cluster import Cluster
def pandas_factory(colnames, rows):
return pd.DataFrame(rows, columns=colnames)
CASSANDRA_HOST = ['172.16.X.Y','172.16.X1.Y1']
CASSANDRA_PORT = 9042
CASSANDRA_DB = "KEYSPACE"
CASSANDRA_TABLE = "COLUMNFAMILY"
cluster = Cluster(contact_points=CASSANDRA_HOST, port=CASSANDRA_PORT)
session = cluster.connect(CASSANDRA_DB)
session.row_factory = pandas_factory
session.default_fetch_size = None
query = "SELECT * FROM {}.{};".format(CASSANDRA_DB, CASSANDRA_TABLE)
rslt = session.execute(query, timeout=None)
df = rslt._current_rows
关于python - 如何将 Cassandra map 转换为 Pandas Dataframe,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/42420260/