Spark2.1 + Hadoop 2.7.3 Spark on Yarn, use client deploy-mode, submit one spark job from client. It runs the following errors. Do some debugs, it is due to the class conflicts for jersey library. For Hadoop-yarn, it uses jersey 1.19, however, spark-2.x uses jersey 2.22. This is the common issue when we do some applications on distributed computation framework. Resolution is straightforward. Please disable the feature of timeline service from yarn perspective. Add the following config into the yarn-site.xml From Binospace, post Spark2.X cannot work with Hadoop 2.7.X timeline service 文章的脚注信息由WordPress的wp-posturl插件自动生成
Remote Shuffle Service for better resource allocation for better cost efficiency and scalability in public cloud environment.