eclipse - 如何在sbt项目中声明对Scalding的依赖？

标签 eclipse scala sbt scalding

我正在尝试弄清楚如何为我自己创建一个 build.sbt 文件 Scalding基于项目。

Scalding源结构没有build.sbt文件。相反，它具有 project/Build.scala 构建定义。

将我自己的 sbt 项目与 Scalding 集成的正确方法是什么，这样我也可以稍后使用 sbt-eclipse 插件将其导入到 Eclipse 中？

更新:

对于以下代码:

import cascading.tuple.Fields
import com.twitter.scalding._

class Scan(args: Args) extends Job(args) {
  val output = TextLine("tmp/out.txt")

  val wordsList = List(
    ("john"),
    ("liza"),
    ("nina"),
    ("x"))

  val orderedPipe =
    IterableSource[(String)](wordsList, ('word))
      .debug
      .write(output)
}

使用这个build.sbt:

name := "Scan"

version := "1.0"

libraryDependencies := Seq("com.twitter" %% "scalding" % "0.11.1")

我收到错误:

$ sbt
[info] Loading global plugins from /home/test/.sbt/0.13/plugins
[info] Set current project to Scan (in build file:/home/test/Cascading/Scala/Scan/)
> compile
[info] Updating {file:/home/test/Cascading/Scala/Scan/}scan...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading http://repo1.maven.org/maven2/com/twitter/scalding_2.10/0.11.1/scalding_2.10-0.11.1.jar ...
[info]  [SUCCESSFUL ] com.twitter#scalding_2.10;0.11.1!scalding_2.10.jar (641ms)
[info] Done updating.
[info] Compiling 1 Scala source to /home/test/Cascading/Scala/Scan/target/scala-2.10/classes...
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:1: not found: object cascading
[error] import cascading.tuple.Fields
[error]        ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:2: object twitter is not a member of package com
[error] import com.twitter.scalding._
[error]            ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:5: not found: type Job
[error] class Scan(args: Args) extends Job(args) {
[error]                                ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:5: not found: type Args
[error] class Scan(args: Args) extends Job(args) {
[error]                  ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:5: too many arguments for constructor Object: ()Object
[error] class Scan(args: Args) extends Job(args) {
[error]           ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:6: not found: value TextLine
[error]   val output = TextLine("tmp/out.txt")
[error]                ^
[error] /home/test/Cascading/Scala/Scan/src/main/scala/Scan.scala:15: not found: value IterableSource
[error]     IterableSource[(String)](wordsList, ('word))
[error]     ^
[error] 7 errors found
[error] (compile:compile) Compilation failed

更新2

在执行 git clone git@github.com:twitter/scalding.git 他们的存储库 和 sbtpublishLocal 之后，我仍然有相同的编译错误。

但是添加您建议的两行到 build.sbt 允许我编译我的代码。所以下面的 build.sbt 确实有效，谢谢!

name := "BlockScan"

version := "1.0"

libraryDependencies := Seq("com.twitter" %% "scalding" % "0.11.1")

lazy val scaldingCore = ProjectRef(uri("https://github.com/twitter/scalding.git"), "scalding-core")

lazy val myProject = project in file(".") dependsOn scaldingCore

“sbt eclipse”创建的 Eclipse 项目无法在 Eclipse 下编译并报告以下错误:

Project 'Scan' is missing required Java project: 'scalding-core'
More than one scala library found in the build path (/home/test/usr/eclipse-scala-3.0.3/configuration/org.eclipse.osgi/bundles/290/1/.cp/lib/scala-library.jar, /home/test/wks/Cascading/Scala/scalding/target/scala-2.9.3/scalding-assembly-0.10.0.jar).At least one has an incompatible version. Please update the project build path so it contains only compatible scala libraries. 

scalacheck_2.9.3-1.10.0.jar is cross-compiled with an incompatible version of Scala (2.9.3). 

specs_2.9.3-1.6.9.jar is cross-compiled with an incompatible version of Scala (2.9.3).

最佳答案

由于他们似乎没有将其库发布到远程存储库，您可以在其中提取必要的依赖项，因此您必须在项目的 GitHub 存储库上声明源依赖项。

lazy val scaldingCore = ProjectRef(uri("https://github.com/twitter/scalding.git"), "scalding-core")

lazy val myProject = project in file(".") dependsOn scaldingCore

通过上述构建定义，sbt 将 git clone RootProject并加载构建。

➜  scalding  xsbt
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
Cloning into '/Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding'...
[info] Loading project definition from /Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding/project
[info] Updating {file:/Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding/project/}scalding-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.eed3si9n/sbt-assembly/scala_2.10/sbt_0.13/0.10.2/jars/sbt-assembly.jar ...
[info]  [SUCCESSFUL ] com.eed3si9n#sbt-assembly;0.10.2!sbt-assembly.jar (3600ms)
[info] Done updating.
[info] Compiling 3 Scala sources to /Users/jacek/.sbt/0.13/staging/e1da2accb95841ffb1df/scalding/project/target/scala-2.10/sbt-0.13/classes...
[warn] there were 8 deprecation warning(s); re-run with -deprecation for details
[warn] there were 2 feature warning(s); re-run with -feature for details
[warn] two warnings found
[info] Set current project to myProject (in build file:/Users/jacek/sandbox/scalding/)
> projects
[info] In file:/Users/jacek/sandbox/scalding/
[info]   * myProject
[info] In https://github.com/twitter/scalding.git
[info]     maple
[info]     scalding
[info]     scalding-args
[info]     scalding-avro
[info]     scalding-commons
[info]     scalding-core
[info]     scalding-date
[info]     scalding-hadoop-test
[info]     scalding-jdbc
[info]     scalding-json
[info]     scalding-parquet
[info]     scalding-repl

build设置应该允许您访问烫金类。

> console
[info] Starting scala interpreter...
[info]
Welcome to Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_60).
Type in expressions to have them evaluated.
Type :help for more information.

scala> import com.twitter.scalding._
import com.twitter.scalding._

还有Scan类编译良好 - 它位于 src/main/scala目录。

> show sources
[info] ArrayBuffer(/Users/jacek/sandbox/scalding/src/main/scala/Scan.scala)
[success] Total time: 0 s, completed Jul 15, 2014 12:21:14 AM
> compile
[info] Updating {file:/Users/jacek/sandbox/scalding/}myProject...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/jacek/sandbox/scalding/target/scala-2.10/classes...
[success] Total time: 4 s, completed Jul 15, 2014 12:21:20 AM

您还可以git clone git@github.com:twitter/scalding.git他们的存储库和 sbt publishLocal能够在 build.sbt 中声明二进制依赖关系如下:

libraryDependencies := Seq("com.twitter" %% "scalding" % "0.11.1")

有了依赖关系(无论哪种方式)，执行 sbt eclipse并完成它!

关于eclipse - 如何在sbt项目中声明对Scalding的依赖？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/24736847/

上一篇：parsing - 快乐:这不是关键字

下一篇：reactjs - React 未定义(无法读取未定义的属性 'createElement')

相关文章：

java - 在 IntelliJ 中使用 build.sbt 排除 JAR 文件中的依赖项签名文件

scala - 创建 hfile 时出现 Spark 问题 - 添加了一个在词法上不大于先前单元格的键

java - 设置字符串的行长度

java - 如何在 SBT 中创建拼图模块？

scala - 并行执行测试

java - JUnit Eclipse 插件？

android - 如何在模拟器中运行安卓应用程序？

java - LDAP+ Java : How can I know my active directory's names

java eclipse 修补了 JRE 类 Xbootclasspath 无法找到或加载主类

scala - 如何从打包中排除不必要的非托管依赖项？

©2024 IT工具网  联系我们