티스토리 뷰

Dev/용어

hive on spark [HDP]

hellotheresy 2017. 1. 10. 18:46


set spark.home=/location/to/sparkHome;

export SPARK_HOME=/usr/hdp/2.4.2.0-xyz/spark/lib

Link the spark-assembly jar to HIVE_HOME/lib.

set hive.execution.engine=spark;



set spark.home=/location/to/sparkHome;

export SPARK_HOME=/usr/hdp/2.4.2.0-xyz/spark/lib

Link the spark-assembly jar to HIVE_HOME/lib.



set hive.execution.engine=spark;

set spark.home=/usr/hdp/2.5.0.0-1245/spark;

set spark.master=yarn-client;

set spark.eventLog.enabled=true; 

set spark.executor.memory=512m;

set spark.executor.cores=2;



add jar /usr/hdp/2.5.0.0-1245/spark/lib/spark-assembly-1.6.2.2.5.0.0-1245-hadoop2.7.3.2.5.0.0-1245.jar;

set spark.home=/usr/hdp/2.5.0.0-1245/spark;

set hive.execution.engine=spark;




select count(*) from tpch.region;



Failed to execute spark task, with exception 'java.lang.IllegalStateException(RPC channel is closed.)'




'Dev > 용어' 카테고리의 다른 글

Polyglot  (0) 2017.01.05
댓글
공지사항
최근에 올라온 글
최근에 달린 댓글
Total
Today
Yesterday
링크
«   2024/05   »
1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31
글 보관함