1、下載Scala

? ? wget ?http://www.scala-lang.org/files/archive/scala-2.10.3.tgz

? ?tar xvzf scala-2.10.3.tgz -C /usr/local

2、下載Spark

? ?wget http://www.apache.org/dist/spark/spark-1.0.0/spark-1.0.0-bin-hadoop2.tgz

? ?tar -zxvf spark-1.0.0-bin-hadoop2.tgz

3、設(shè)置spark的配置文件

mv?spark-env.sh.template?spark-env.sh

cat >> spark-env.sh

export JAVA_HOME=/usr/java/jdk1.6.0_26
export SCALA_HOME=/usr/local/scala-2.10.3
export HADOOP_HOME=/root/hadoop-2.2.0
SPARK_LOCAL_DIR="/data/spark/tmp"

4、環(huán)境變量設(shè)置

cat >> /etc/profile

export SCALA_HOME=/usr/local/scala-2.10.3
export SPARK_HOME=/root/spark
export PATH=$SCALA_HOME/bin:$PATH

source /etc/profile

5、啟動(dòng)Hadoop相關(guān)進(jìn)程

start-all.sh

6、啟動(dòng)Spark Worker

sbin/start-all.sh

此時(shí)通過(guò)jps可知啟動(dòng)了例如以下進(jìn)程：

21624 Worker
10664 MainGenericRunner
20515 SecondaryNameNode
21057 Master
20311 NameNode
20689 ResourceManager

7、測(cè)試

root@master:~/spark/bin# ./run-example ?org.apache.spark.examples.SparkPi 1 local

輸出：

...

[main] INFO org.apache.spark.SparkContext - Job finished: reduce at SparkPi.scala:39, took 5.0273654 s
Pi is roughly 3.13986

....

附：用Eclipse配置Spark源代碼閱讀環(huán)境http://www.cnblogs.com/lam99v/p/3716207.html

總結(jié)

以上是生活随笔為你收集整理的Spark1.0 安装的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。