java - Maven在尝试连接到Cloudera URL以获取CDH Jar位置时显示错误

Question

我想使用我想从cloudera网站下载的hadoop cdh jar运行一个单词计数程序。要连接到cloudera网站并下载，我正在使用以下pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>uk.co.arunhorne</groupId>
    <artifactId>hadoop-wordcount</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <jdkLevel>1.6</jdkLevel>
        <requiredMavenVersion>[2.1,)</requiredMavenVersion>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.build.outputEncoding>UTF-8</project.build.outputEncoding>
    </properties>

    <repositories>
        <repository>
            <id>cloudera-releases</id>
            <!-- <url>http://10.118.200.138/cloudera-jars/repodata/</url> -->
            <!--<url>https://repository.cloudera.com/content/repositories/releases/</url> -->
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>0.20.2-cdh3u3</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.1</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.1</version>
                <configuration>
                    <source>${jdkLevel}</source>
                    <target>${jdkLevel}</target>
                    <showDeprecation>true</showDeprecation>
                    <showWarnings>true</showWarnings>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.2.1</version>
                <configuration>
                    <descriptors>
                        <descriptor>src/main/assembly/hadoop-job.xml</descriptor>
                    </descriptors>
                    <archive>
                        <manifest>
                            <mainClass>uk.co.arunhorne.mapred.wordcount.WordCount</mainClass>
                        </manifest>
                    </archive>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

但是我在构建控制台时遇到以下错误:-

[WARNING] 
[WARNING] Some problems were encountered while building the effective settings
[WARNING] Unrecognised tag: 'repositories' (position: START_TAG seen ...</offline>\r\n<repositories>... @9:15)  @ D:\installations\apache-maven-3.0.4\conf\settings.xml, line 9, column 15
[WARNING] Unrecognised tag: 'repositories' (position: START_TAG seen ...</offline>\r\n<repositories>... @9:15)  @ C:\Users\Sailaja_Sahoo\.m2\settings.xml, line 9, column 15
[WARNING] 
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building hadoop-wordcount 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/hadoop/hadoop-core/0.20.2-cdh3u3/hadoop-core-0.20.2-cdh3u3.pom
Feb 4, 2013 5:55:04 PM org.apache.maven.wagon.providers.http.httpclient.client.protocol.RequestProxyAuthentication process
SEVERE: Proxy authentication error: Invalid name provided (Mechanism level: Could not load configuration file C:\Windows\krb5.ini (The system cannot find the file specified))
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.152s
[INFO] Finished at: Mon Feb 04 17:55:04 IST 2013
[INFO] Final Memory: 3M/15M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hadoop-wordcount: Could not resolve dependencies for project uk.co.arunhorne:hadoop-wordcount:jar:1.0-SNAPSHOT: Failed to collect dependencies for [org.apache.hadoop:hadoop-core:jar:0.20.2-cdh3u3 (provided), org.apache.commons:commons-lang3:jar:3.1 (compile)]: Failed to read artifact descriptor for org.apache.hadoop:hadoop-core:jar:0.20.2-cdh3u3: Could not transfer artifact org.apache.hadoop:hadoop-core:pom:0.20.2-cdh3u3 from/to cloudera-releases (https://repository.cloudera.com/artifactory/cloudera-repos/): Not authorized by proxy, ReasonPhrase:Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  ). -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

当我尝试创建本地目录并下载jar时，它构建成功。但是尝试连接到该cloudera url时出现错误，因为我只需要使用cloudera url，而不使用其他任何url

Answer 1

尝试从Maven Repo使用此功能:

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-core</artifactId>
    <version>1.1.1</version>
</dependency>

更新

我认为您是使用this page创建应用的。

我创建类似的项目并运行mvn package。 Maven从cloudera存储库下载了所有必需的库，您可以看到full maven log

我也给你link到我的项目中(也许对你有帮助)

因此，看看所有这些，我可以假设您的Maven配置有问题。