在我的项目中,我的外部库是 spark-assemble-1.3.1-hadoop2.6.0
,如果我按“.”,IDE 会通知我 toDF()
,但它告诉我,当我编码时无法解析符号 toDF()
。很抱歉,我在 Apache 中找不到 toDF()
Spark 文档。
case class Feature(name:String, value:Double, time:String, period:String)
val RESRDD = RDD.map(tuple => {
var bson=new BasicBSONObject();
bson.put("name",name);
bson.put("value",value);
(null,bson);
})
RESRDD
.map(_._2)
.map(f => Feature(f.get("name").toString, f.get("value").toString.toDouble))
.toDF()
最佳答案
为了能够使用toDF
,您必须先导入sqlContext.implicits
:
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext.implicits._
case class Foobar(foo: String, bar: Integer)
val foobarRdd = sc.parallelize(("foo", 1) :: ("bar", 2) :: ("baz", -1) :: Nil).
map { case (foo, bar) => Foobar(foo, bar) }
val foobarDf = foobarRdd.toDF
foobarDf.limit(1).show
关于scala - Spark /SQL :spark can't resolve symbol toDF,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/31143840/