hive?on?spark?报错
1.自从spark2.0.0发布没有assembly的包了，在jars里面，是很多小jar包
修改目录查找jar
2.异常HiveConf of name hive.enable.spark.execution.engine does not exist
在hive-site.xml中：
hive.enable.spark.execution.engine过时了,配置删除即可
3.异常
Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create spark client.)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
Spark与hive版本不对,spark的编译，在这里我使用的是hive稳定版本2.01,查看他的pom.xml需要的spark版本是1.5.0。hive与spark版本必须对应着
重新编译完报
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder 在spark-env.sh里面添加
export SPARK_DIST_CLASSPATH=$(hadoop classpath)
spark master可以起来了，但是slaves仍然是上面错误
用scala./dev/change-scala-version.sh mvn -Pyarn -Phadoop-2.4 -Dscala-2.11 -DskipTests clean package
4.异常
4] shutting down ActorSystem [sparkMaster]
java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/Module
删除hive/lib
jackson-annotations-2.4.0.jar
jackson-core-2.4.2.jar
jackson-databind-2.4.2.jar
cp $HADOOP_HOME/share/hadoop/tools/lib/jackson-annotations-2.2.3.jar ./
cp $HADOOP_HOME/share/hadoop/tools/lib/jackson-core-2.2.3.jar ./
cp $HADOOP_HOME/share/hadoop/tools/lib/jackson-databind-2.2.3.jar ./
Spark运行时的日志，查看加载jar包的地方,添加上述jar
5.异常
?java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate root
在hadoop中在core-site.xml添加如下属性，其中
??????hadoop.proxyuser.root.groups
??????root
??????Allow the superuser oozie to impersonate any members of the group group1 and group2
??????hadoop.proxyuser.root.hosts
??????*
??????The superuser can connect only from host1 and host2 to impersonate a user
6.异常
?MetaException(message:Hive Schema version 2.1.0 does not match metastore's schema version 2.0.0 Metastore is not up
graded or corrupt)
HADOOP或者hive对应着版本太高