1
程序员文章站
2024-03-21 19:50:40
...
<div class="blog-content-box">
<div class="article-header-box">
<div class="article-header">
<div class="article-title-box">
<span class="article-type type-1 float-left">原</span> <h1 class="title-article">Yarn下Mapreduce的内存参数理解&xml参数配置</h1>
</div>
<div class="article-info-box">
<div class="article-bar-top" style="height: 23px;">
<span class="time">2017年11月26日 15:39:15</span>
<a class="follow-nickName" href="https://me.csdn.net/z3935212" target="_blank">z3935212</a>
<span class="read-count">阅读数：2035</span>
<span class="tags-box artic-tag-box">
<span class="label">标签：</span>
<a data-track-click="{"mod":"popu_626","con":"mapreduce"}" class="tag-link" href="http://so.csdn.net/so/search/s.do?q=mapreduce&t=blog" target="_blank">mapreduce </a><a data-track-click="{"mod":"popu_626","con":"内存"}" class="tag-link" href="http://so.csdn.net/so/search/s.do?q=内存&t=blog" target="_blank">内存 </a><a data-track-click="{"mod":"popu_626","con":"yarn"}" class="tag-link" href="http://so.csdn.net/so/search/s.do?q=yarn&t=blog" target="_blank">yarn </a><a data-track-click="{"mod":"popu_626","con":"框架"}" class="tag-link" href="http://so.csdn.net/so/search/s.do?q=框架&t=blog" target="_blank">框架 </a>
<span class="article_info_click">更多</span></span>
<div class="tags-box space">
<span class="label">个人分类：</span>
<a class="tag-link" href="https://blog.csdn.net/z3935212/article/category/7310584" target="_blank">大数据 </a>
</div>
</div>
<div class="operating">
</div>
</div>
</div>
</div>
<article>
<div id="article_content" class="article_content clearfix csdn-tracking-statistics" data-pid="blog" data-mod="popu_307" data-dsm="post" style="height: 1896px; overflow: hidden;">
<div class="markdown_views">
<!-- flowchart 箭头图标 勿删 -->
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;"><path stroke-linecap="round" d="M5,0 0,2.5 5,5z" id="raphael-marker-block" style="-webkit-tap-highlight-color: rgba(0, 0, 0, 0);"></path></svg>
<p>参考：<a href="http://dongxicheng.org/mapreduce-nextgen/hadoop-yarn-configurations-resourcemanager-nodemanager/" rel="nofollow" target="_blank">http://dongxicheng.org/mapreduce-nextgen/hadoop-yarn-configurations-resourcemanager-nodemanager/</a></p>
<h2 id="container是什么"><a name="t0"></a><strong>Container是什么？</strong></h2>
<p>Container就是一个yarn的java进程，在Mapreduce中的AM，MapTask，ReduceTask都作为Container在Yarn的框架上执行，你可以在RM的网页上【8088端口】看到Container的状态</p>
<h2 id="基础"><a name="t1"></a><strong>基础</strong></h2>
<p>Yarn的ResourceManger（简称RM）通过逻辑上的队列分配内存，CPU等资源给application，默认情况下RM允许最大AM申请Container资源为8192MB(“yarn.scheduler.maximum-allocation-mb“)，默认情况下的最小分配资源为1024M(“yarn.scheduler.minimum-allocation-mb“)，AM只能以增量（”yarn.scheduler.minimum-allocation-mb“）和不会超过(“yarn.scheduler.maximum-allocation-mb“)的值去向RM申请资源，AM负责将(“mapreduce.map.memory.mb“)和(“mapreduce.reduce.memory.mb“)的值规整到能被(“yarn.scheduler.minimum-allocation-mb“)整除，RM会拒绝申请内存超过8192MB和不能被1024MB整除的资源请求【内存增量】。</p>
<p><strong>本文基于Hadoop2.6；下面提到的配置项最好都要设置以覆盖默认值，以便调优。</strong></p>
<h2 id="1yarn-sitexml-设置"><a name="t2"></a><strong>1、yarn-site.xml 设置</strong></h2>
<p>yarn进行资源管理的时候：以contain作为最小单位来进行资源分配的。 <br>
<strong>事实上Hadoop所有设置都可以写在同一个xml中，</strong>但是为了自己查看方便，模块化，所以分开写。 <br>
yarn-site.xml主要设置yarn的基本配置，以yarn开头的配置项。修改后必须重启集群生效。 <br>
mapred-site.xml主要设置mapreduxe任务的配置项；这个文件修改后不需重启集群。</p>
<h2 id="11-nm设置nodemanager"><a name="t3"></a><strong>1.1 NM设置（NodeManager）</strong></h2>
<p>NM的内存资源配置，主要是通过下面两个参数进行的（这两个值是Yarn平台特性，应在yarn-sit.xml中配置） ： <br>
yarn.nodemanager.resource.memory-mb 默认8GB <br>
yarn.nodemanager.vmem-pmem-ratio 默认2.1 <br>
说明：第一个参数：该结点向操作系统申请的内存总量，RM中的两个值不能超过此值。此数值可以用于计算container最大数目，即：用此值除以RM中的最小容器内存。第二个：虚拟内存率，是占task所用内存的百分比，默认值为2.1倍。如报错虚拟内存溢出，则提高该值。 <br>
注意：第一个参数默认大小是8G，即使计算机内存不足8G也会按着8G内存来使用，所有计算机内存小于8GB应该调低这个值。</p>
<h2 id="12-rm设置resourcemanager"><a name="t4"></a><strong>1.2 RM设置（ResourceManager）</strong></h2>
<p>RM的内存资源配置，主要是通过下面的两个参数进行的（这两个值是Yarn平台特性，应在yarn-sit.xml中配置）： <br>
yarn.scheduler.minimum-allocation-mb 默认1GB <br>
yarn.scheduler.maximum-allocation-mb 默认8GB <br>
说明：单个容器可申请的最小与最大内存，应用在运行申请内存时不能超过最大值，应用申请小于最小值则会分配最小值，从这个角度看，最小值有点像操作系统中的页。最小值还有另外一种用途，用来计算一个节点的最大container数目。注：这两个值一经设定不能动态改变(此处所说的动态改变是指需要重启集群生效)。</p>
<h1 id="2mapred-sitexml-设置见文后附录"><a name="t5"></a><strong>2、mapred-site.xml 设置见文后附录</strong></h1>
<h2 id="21-am设置applicationmanager"><a name="t6"></a><strong>2.1 AM设置（ApplicationManager）</strong></h2>
<p>AM内存配置相关参数，此处以MapReduce为例进行说明（这两个值是AM特性，应在mapred-site.xml中配置），如下： <br>
mapreduce.map.memory.mb 默认 <br>
mapreduce.reduce.memory.mb 默认 <br>
说明：单个Map/Reduce task 申请的内存大小，其值应该在RM中的最大和最小container值之间。如果没有配置则通过如下简单公式获得： <br>
max(MIN_CONTAINER_SIZE, (Total Available RAM) / containers)) <br>
<strong>一般reduce内存大小应该是map的2倍。</strong>注：这两个值可以在应用启动时通过参数改变，可以动态调整；</p>
<h2 id="22-am-jvm设置"><a name="t7"></a><strong>2.2 AM JVM设置</strong></h2>
<p>AM中其它与内存相关的参数，还有JVM相关的参数，这些参数可以通过，如下选项配置： <br>
mapreduce.map.java.opts 默认 <br>
mapreduce.reduce.java.opts 默认 <br>
说明：这两个参主要是为需要运行JVM程序（java、scala等）准备的，通过这两个设置可以向JVM中传递参数的，与内存有关的是，-Xmx，-Xms等选项。此数值大小，应该在AM中的map.mb和reduce.mb之间。</p>
<h2 id="23总结"><a name="t8"></a><strong>2.3总结</strong></h2>
<p>AM为Map/Reduce task向RM申请资源，RM按照mini和max、资源增量的值为task分配一个固定大小的container供task使用。RM可以分配下去的资源总量受到NM配置的值限制。</p>
<h2 id="3注意"><a name="t9"></a><strong>3、注意：</strong></h2>
<p>在hadoop2及以上版本中，map和reduce task 是运行在container中的。mapreduce.{map|reduce}.memory.mb 被yarn用来设置container的内存大小。如果container的内存超限，会被yarn杀死。在container中，为了执行map和reduce task，yarn会在contaner中启动一个jvm来执行task任务。mapreduce.{map|reduce}.java.opts用来设置container启动的jvm相关参数，通过设置Xmx来设置map 或者reduce task的最大堆内存。 <br>
理论上，{map|reduce}.java.opts设置的最大堆内存要比{map|reduce}.memory.mb小。一般设置为一般设置为0.75倍的memory.mb即可；因为在yarn container这种模式下，JVM进程跑在container中，需要为java code等非JVM的内存使用预留些空间。 <br>
运行中的设置方法例如：xml中也可设置 <br>
hadoop jar -Dmapreduce.reduce.memory.mb=4096 -Dmapreduce.map.java.opts=-Xmx3276</p>
<h2 id="4常见内存溢出报错解决"><a name="t10"></a><strong>4、常见内存溢出报错&解决</strong></h2>
<p>默认情况下，yarn.nodemanager.vmem-pmem-ratio被设置为2.1，这意味着，每个map或者task任务只能使用2.1倍（”mapreduce.reduce.memory.mb”) or (“mapreduce.map.memory.mb”) 大小的虚拟内存，如果使用的量超出则会被nm杀掉。 <br>
例如：日志中常见报错： <br>
1、Container xxx is running beyond physical memory limits <br>
2、java heap space <br>
3、Error: GC overhead limit exceeded</p>
<pre class="prettyprint" name="code"><code class="hljs lasso has-numbering">报错<span class="hljs-number">1</span>:Current usage: <span class="hljs-number">1.1</span>gb of <span class="hljs-number">2.0</span>gb physical memory used; <span class="hljs-number">4.6</span>gb of <span class="hljs-number">4.2</span>gb virtual memory used<span class="hljs-built_in">.</span> Killing container<span class="hljs-built_in">.</span>【即虚拟内存溢出】；
<span class="hljs-subst">**</span>方法一：提高yarn<span class="hljs-built_in">.</span>nodemanager<span class="hljs-built_in">.</span>vmem<span class="hljs-attribute">-pmem</span><span class="hljs-attribute">-ratio</span> <span class="hljs-subst">=</span> <span class="hljs-number">5</span>或者更高；【推荐】<span class="hljs-subst">**</span>
方法二：yarn<span class="hljs-built_in">.</span>nodemanager<span class="hljs-built_in">.</span>vmem<span class="hljs-attribute">-check</span><span class="hljs-attribute">-enabled</span> <span class="hljs-subst">=</span><span class="hljs-literal">false</span> ；关闭虚拟内存检查；不推荐
方法三：提高物理内存分配，相应的虚拟内存自然就多了，但是这样不是最优</code><ul class="pre-numbering" style=""><li style="color: rgb(153, 153, 153);">1</li><li style="color: rgb(153, 153, 153);">2</li><li style="color: rgb(153, 153, 153);">3</li><li style="color: rgb(153, 153, 153);">4</li></ul></pre>
<pre class="prettyprint" name="code"><code class="hljs avrasm has-numbering">报错<span class="hljs-number">2</span>：Current usage: <span class="hljs-number">2.1</span>gb of <span class="hljs-number">2.0</span>gb physical memory used<span class="hljs-comment">; 3.6gb of 4.2gb virtual memory used. Killing container.【即物理内存溢出】；</span>
方法一：mapreduce<span class="hljs-preprocessor">.map</span><span class="hljs-preprocessor">.memory</span><span class="hljs-preprocessor">.mb</span> = <span class="hljs-number">3</span>GB以上，然后测试这个map/reduce task需要使用多少内存才够用，提高这个值直到不报错为止。
方法二：提高yarn<span class="hljs-preprocessor">.scheduler</span><span class="hljs-preprocessor">.minimum</span>-allocation-mb = <span class="hljs-number">3</span>GB以上，同理【不推荐】</code><ul class="pre-numbering" style=""><li style="color: rgb(153, 153, 153);">1</li><li style="color: rgb(153, 153, 153);">2</li><li style="color: rgb(153, 153, 153);">3</li></ul></pre>
<h2 id="5内存分配增量规整因子incrementmemory"><a name="t11"></a><strong>5、内存分配增量/规整因子/incrementMemory</strong></h2>
<p>为了易于管理资源和调度资源，Hadoop YARN内置了资源规整化算法，它规定了最小可申请资源量、最大可申请资源量和<strong>资源规整化因子</strong>，规整化因子是用来规整化应用程序资源的，<strong>应用程序申请的资源如果不是该因子的整数倍，则将被修改为最小的整数倍对应的值</strong>，公式为ceil(a/b)*b，其中a是应用程序申请的资源，b为规整化因子。对于规整化因子，不同调度器不同，具体如下： <br>
FIFO和Capacity Scheduler，规整化因子等于最小可申请资源量，不可单独配置。 <br>
Fair Scheduler：规整化因子通过参数yarn.scheduler.increment-allocation-mb和yarn.scheduler.increment-allocation-vcores设置，默认是1024和1。 <br>
通过以上介绍可知，应用程序申请到资源量可能大于资源申请的资源量，比如YARN的最小可申请资源内存量为1024，规整因子是1024，如果一个应用程序申请1500内存，则会得到2048内存，如果规整因子是512，则得到1536内存。</p>
<p>如下图：map container的内存(“mapreduce.map.memory.mb”)被设置为1536mb 。但是AM会为其申请了2048m的内存，因为am的最小分配单位/增量（yarn.scheduler.minimum-allocation-mb）被设置为1024，也就是以1GB为单位往上加。这是一种逻辑上的分配，这个值被NodeManager用来监控该进程内存资源的使用率，如果mapTask的堆内存使用率超过了2048MB，NM将会把这个task给杀掉。</p>
<p><img src="http://hackershell.cn/image/Copy_of_Yarn_mem_params.jpg" alt="这里写图片描述" title=""> <br>
从上面的图可以看出map，reduce，AM container的JVM，“JVM”矩形代表服务进程，“Max heap”，“Max virtual”矩形代表NodeManager对JVM进程的最大内存和虚拟内存的限制。</p>
<p>When a mapreduce job completes you will see several counters dumped at the end of the job.The three memory counters below show how much physical memory was allocated vs virtual memory.</p>
<hr>
<p>对于56G内存的NM来说，如果全部跑map则56/3大约跑18个container <br>
大概了解完以上的参数之后，mapreduce.map.java.opts和mapreduce.map.memory.mb参数之间，有什么联系呢？ <br>
通过上面的分析，我们知道如果一个yarn的container超除了heap设置的大小，这个task将会失败，我们可以根据哪种类型的container失败去相应增大mapreduce.{map|reduce}.memory.mb去解决问题。 <strong>但同时带来的问题是集群并行跑的container的数量少了</strong>，<strong><em>所以适当的调整内存参数对集群的利用率的提升尤为重要。</em></strong></p>
<hr>
<h2 id="对cpu的设置参数"><a name="t12"></a><strong>对cpu的设置参数：</strong></h2>
<p>在yarn的框架中，CPU的配置决定了一个任务的生死，CPU只是决定了任务的完成效率。所以CPU的设置只是简单列出来。 <br>
yarn.scheduler.minimum-allocation-vcores：最小可申请CPU数，默认是1 <br>
yarn.scheduler.maximum-allocation-vcores：最大可申请CPU数，默认是4 <br>
yarn.nodemanager.resource.cpu-vcores：集群向系统申请的总可用CPU数目，默认是8</p>
<p>mapreduce.map.cpu.vcores：task container申请的CPU数目，默认是1 <br>
完结！</p>
<hr>
<p>附录1-我的yarn-site.xml配置：</p>
<pre class="prettyprint" name="code"><code class="hljs xml has-numbering"><span class="hljs-pi"><?xml version="1.0"?></span>
<span class="hljs-tag"><<span class="hljs-title">configuration</span>></span>
<span class="hljs-comment"><!-- Site specific YARN configuration properties
下面的配置项中值的大小，特别是内存大小，通过每个节点logs中的yarn-hadoop-nodemanager-Slave1.log 文件中查看日志来判断；出错行形如：
running beyond virtual memory limits. Current usage: 484.0 MB of 2 GB physical memory used; 4.9 GB of 4.2 GB virtual memory used. Killing container.
在yarn-hadoop-resourcemanager-Slave1.log日志文件中：形如：表示内存和cpu资源分配：
例如：总的结点资源：<memory:16000, vCores:8>==<memory:10240, vCores:5>+<memory:5760, vCores:3>
Released container container_1511512572160_0003_01_000025 of capacity <memory:2048, vCores:1> on host Slave1:42899, which currently has 5 containers, <memory:10240, vCores:5> used and <memory:5760, vCores:3> available,
--></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.resourcemanager.hostname<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>Slave1<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.nodemanager.aux-services<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>mapreduce_shuffle<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.nodemanager.pmem-check-enabled<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>true<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>检测物理内存的使用是否超出分配值，若任务超出分配值，则将其杀掉，默认true。<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.nodemanager.vmem-check-enabled<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>true<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>检测虚拟内存的使用是否超出；若任务超出分配值，则将其杀掉，默认true。在确定内存不会泄漏的情况下可以设置此项为 False；<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.nodemanager.vmem-pmem-ratio<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>8<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>任务每使用1MB物理内存，最多可使用虚拟内存量比率，默认2.1；在上一项中设置为false不检测虚拟内存时，此项就无意义了<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.nodemanager.resource.cpu-vcores<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>12<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>该节点上YARN可使用的总核心数；一般设为cat /proc/cpuinfo| grep "processor"| wc -l 的值。默认是8个；<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.nodemanager.resource.memory-mb<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>16000<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>该节点上YARN可使用的物理内存总量，【向操作系统申请的总量】默认是8192（MB）<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.scheduler.minimum-allocation-mb<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>3072<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>单个容器/调度器可申请的最少物理内存量，默认是1024（MB）；一般每个contain都分配这个值；即：capacity memory:3072, vCores:1，如果提示物理内存溢出，提高这个值即可；<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.scheduler.maximum-allocation-mb<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>8000<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>单个容器申请最大值<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>The minimum allocation for every container request at the RM,
in terms of virtual CPU cores. Requests lower than this will throw a
InvalidResourceRequestException.<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.scheduler.minimum-allocation-vcores<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>1<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>The http address of the RM web application.<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.resourcemanager.webapp.address<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>${yarn.resourcemanager.hostname}:8088<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"></<span class="hljs-title">configuration</span>></span>
</code><ul class="pre-numbering" style=""><li style="color: rgb(153, 153, 153);">1</li><li style="color: rgb(153, 153, 153);">2</li><li style="color: rgb(153, 153, 153);">3</li><li style="color: rgb(153, 153, 153);">4</li><li style="color: rgb(153, 153, 153);">5</li><li style="color: rgb(153, 153, 153);">6</li><li style="color: rgb(153, 153, 153);">7</li><li style="color: rgb(153, 153, 153);">8</li><li style="color: rgb(153, 153, 153);">9</li><li style="color: rgb(153, 153, 153);">10</li><li style="color: rgb(153, 153, 153);">11</li><li style="color: rgb(153, 153, 153);">12</li><li style="color: rgb(153, 153, 153);">13</li><li style="color: rgb(153, 153, 153);">14</li><li style="color: rgb(153, 153, 153);">15</li><li style="color: rgb(153, 153, 153);">16</li><li style="color: rgb(153, 153, 153);">17</li><li style="color: rgb(153, 153, 153);">18</li><li style="color: rgb(153, 153, 153);">19</li><li style="color: rgb(153, 153, 153);">20</li><li style="color: rgb(153, 153, 153);">21</li><li style="color: rgb(153, 153, 153);">22</li><li style="color: rgb(153, 153, 153);">23</li><li style="color: rgb(153, 153, 153);">24</li><li style="color: rgb(153, 153, 153);">25</li><li style="color: rgb(153, 153, 153);">26</li><li style="color: rgb(153, 153, 153);">27</li><li style="color: rgb(153, 153, 153);">28</li><li style="color: rgb(153, 153, 153);">29</li><li style="color: rgb(153, 153, 153);">30</li><li style="color: rgb(153, 153, 153);">31</li><li style="color: rgb(153, 153, 153);">32</li><li style="color: rgb(153, 153, 153);">33</li><li style="color: rgb(153, 153, 153);">34</li><li style="color: rgb(153, 153, 153);">35</li><li style="color: rgb(153, 153, 153);">36</li><li style="color: rgb(153, 153, 153);">37</li><li style="color: rgb(153, 153, 153);">38</li><li style="color: rgb(153, 153, 153);">39</li><li style="color: rgb(153, 153, 153);">40</li><li style="color: rgb(153, 153, 153);">41</li><li style="color: rgb(153, 153, 153);">42</li><li style="color: rgb(153, 153, 153);">43</li><li style="color: rgb(153, 153, 153);">44</li><li style="color: rgb(153, 153, 153);">45</li><li style="color: rgb(153, 153, 153);">46</li><li style="color: rgb(153, 153, 153);">47</li><li style="color: rgb(153, 153, 153);">48</li><li style="color: rgb(153, 153, 153);">49</li><li style="color: rgb(153, 153, 153);">50</li><li style="color: rgb(153, 153, 153);">51</li><li style="color: rgb(153, 153, 153);">52</li><li style="color: rgb(153, 153, 153);">53</li><li style="color: rgb(153, 153, 153);">54</li><li style="color: rgb(153, 153, 153);">55</li><li style="color: rgb(153, 153, 153);">56</li><li style="color: rgb(153, 153, 153);">57</li><li style="color: rgb(153, 153, 153);">58</li><li style="color: rgb(153, 153, 153);">59</li><li style="color: rgb(153, 153, 153);">60</li><li style="color: rgb(153, 153, 153);">61</li><li style="color: rgb(153, 153, 153);">62</li><li style="color: rgb(153, 153, 153);">63</li><li style="color: rgb(153, 153, 153);">64</li><li style="color: rgb(153, 153, 153);">65</li><li style="color: rgb(153, 153, 153);">66</li><li style="color: rgb(153, 153, 153);">67</li><li style="color: rgb(153, 153, 153);">68</li><li style="color: rgb(153, 153, 153);">69</li></ul></pre>
<p>附录2-我的mapred-site.xml：</p>
<pre class="prettyprint" name="code"><code class="hljs xml has-numbering"><span class="hljs-pi"><?xml version="1.0"?></span>
<span class="hljs-pi"><?xml-stylesheet type="text/xsl" href="configuration.xsl"?></span>
<span class="hljs-comment"><!-- Put site-specific property overrides in this file.
--></span>
<span class="hljs-tag"><<span class="hljs-title">configuration</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.framework.name<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>yarn<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.jobhistory.address<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>Slave1:10020<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.jobhistory.webapp.address<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>Slave1:19888<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.map.cpu.vcores<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>2<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>每个MapTask容器申请的核心数；默认1<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.map.memory.mb<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>4096<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>每个Map task容器申请的内存大小；默认1G<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.map.java.opts<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>-Xmx3072m<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>map使用的JVM的堆大小heapsize；根据单个mapper/reducer容器内存进行调整，heapsize不能大于单个mapper/reducer容器内存值，一般设置为mapreduce.map.memory.mb的85%左右<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-comment"><!--reduce阶段的设置 --></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.reduce.cpu.vcores<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>2<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>每个ReduceTask容器申请的核心数；默认1<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.reduce.memory.mb<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>8192<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>Reduce task申请的内存大小<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.reduce.java.opts<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>-Xmx6144m<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>Reduce阶段的JVM的堆大小；同上<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"></<span class="hljs-title">configuration</span>></span>
</code><ul class="pre-numbering" style=""><li style="color: rgb(153, 153, 153);">1</li><li style="color: rgb(153, 153, 153);">2</li><li style="color: rgb(153, 153, 153);">3</li><li style="color: rgb(153, 153, 153);">4</li><li style="color: rgb(153, 153, 153);">5</li><li style="color: rgb(153, 153, 153);">6</li><li style="color: rgb(153, 153, 153);">7</li><li style="color: rgb(153, 153, 153);">8</li><li style="color: rgb(153, 153, 153);">9</li><li style="color: rgb(153, 153, 153);">10</li><li style="color: rgb(153, 153, 153);">11</li><li style="color: rgb(153, 153, 153);">12</li><li style="color: rgb(153, 153, 153);">13</li><li style="color: rgb(153, 153, 153);">14</li><li style="color: rgb(153, 153, 153);">15</li><li style="color: rgb(153, 153, 153);">16</li><li style="color: rgb(153, 153, 153);">17</li><li style="color: rgb(153, 153, 153);">18</li><li style="color: rgb(153, 153, 153);">19</li><li style="color: rgb(153, 153, 153);">20</li><li style="color: rgb(153, 153, 153);">21</li><li style="color: rgb(153, 153, 153);">22</li><li style="color: rgb(153, 153, 153);">23</li><li style="color: rgb(153, 153, 153);">24</li><li style="color: rgb(153, 153, 153);">25</li><li style="color: rgb(153, 153, 153);">26</li><li style="color: rgb(153, 153, 153);">27</li><li style="color: rgb(153, 153, 153);">28</li><li style="color: rgb(153, 153, 153);">29</li><li style="color: rgb(153, 153, 153);">30</li><li style="color: rgb(153, 153, 153);">31</li><li style="color: rgb(153, 153, 153);">32</li><li style="color: rgb(153, 153, 153);">33</li><li style="color: rgb(153, 153, 153);">34</li><li style="color: rgb(153, 153, 153);">35</li><li style="color: rgb(153, 153, 153);">36</li><li style="color: rgb(153, 153, 153);">37</li><li style="color: rgb(153, 153, 153);">38</li><li style="color: rgb(153, 153, 153);">39</li><li style="color: rgb(153, 153, 153);">40</li><li style="color: rgb(153, 153, 153);">41</li><li style="color: rgb(153, 153, 153);">42</li><li style="color: rgb(153, 153, 153);">43</li><li style="color: rgb(153, 153, 153);">44</li><li style="color: rgb(153, 153, 153);">45</li><li style="color: rgb(153, 153, 153);">46</li><li style="color: rgb(153, 153, 153);">47</li><li style="color: rgb(153, 153, 153);">48</li><li style="color: rgb(153, 153, 153);">49</li><li style="color: rgb(153, 153, 153);">50</li><li style="color: rgb(153, 153, 153);">51</li></ul></pre> </div>
<link href="https://csdnimg.cn/release/phoenix/mdeditor/markdown_views-7f770a53f2.css" rel="stylesheet">
</div>
<div class="hide-article-box text-center">
<a class="btn" id="btn-readmore" data-track-view="{"mod":"popu_376","con":",https://blog.csdn.net/z3935212/article/details/78637157,"}" data-track-click="{"mod":"popu_376","con":",https://blog.csdn.net/z3935212/article/details/78637157,"}">阅读更多</a>
</div>
<script>
(function(){
function setArticleH(btnReadmore,posi){
var winH = $(window).height();
var articleBox = $("div.article_content");
var artH = articleBox.height();
if(artH > winH*posi){
articleBox.css({
'height':winH*posi+'px',
'overflow':'hidden'
})
btnReadmore.click(function(){
articleBox.removeAttr("style");
$(this).parent().remove();
})
}else{
btnReadmore.parent().remove();
}
}
var btnReadmore = $("#btn-readmore");
if(btnReadmore.length>0){
if(currentUserName){
setArticleH(btnReadmore,3);
}else{
setArticleH(btnReadmore,1.2);
}
}
})()
</script>
</article>
</div>
<script>
$(".MathJax").remove();
if($('div.markdown_views pre.prettyprint code.hljs').length > 0 ){
$('div.markdown_views')[0].className = 'markdown_views';
}
</script>
<div class="t0 clearfix">
<a href="https://blog.csdn.net/z_xiaozhuT/article/details/80448087" target="_blank" data-track-view="{"mod":"popu_642","con": ",https://blog.csdn.net/z_xiaozhuT/article/details/80448087,"}" data-track-click="{"mod":"popu_642","con": ",https://blog.csdn.net/z_xiaozhuT/article/details/80448087,"}">
<div class="text-content">
<h4>关于<em>mapreduce</em> 几个<em>参数</em>的解释 - z_xiaozhut的博客(个人博客)</h4>
<div class="description">一、mapreduce.map.java.opts、mapreduce.reduce.java.opts 以map任务为例，Container其实就是在执行一个脚本文件，而脚本文件中，会执行一个...</div>
</div>
</a>
</div> <a id="commentBox"></a>
<div class="comment-box">
<div class="comment-edit-box d-flex">
<a id="commentsedit"></a>
<div class="user-img">
<img src="https://avatar.csdn.net/1/9/B/3_xjp8587.jpg">
</div>
<form id="commentform">
<input type="hidden" id="comment_replyId">
<textarea class="comment-content" name="comment_content" id="comment_content" placeholder="想对作者说点什么"></textarea>
<div class="opt-box"> <!-- d-flex -->
<div id="ubbtools" class="add_code">
<a href="#insertcode" code="code" target="_self"><i class="icon iconfont icon-daima"></i></a>
</div>
<input type="hidden" id="comment_replyId" name="comment_replyId">
<input type="hidden" id="comment_userId" name="comment_userId" value="">
<input type="hidden" id="commentId" name="commentId" value="">
<div style="display: none;" class="csdn-tracking-statistics tracking-click" data-mod="popu_384"><a href="#" target="_blank" class="comment_area_btn">发表评论</a></div>
<div class="dropdown" id="myDrap">
<a class="dropdown-face d-flex align-items-center" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">
<div class="txt-selected text-truncate">添加代码片</div>
<svg class="icon d-block" aria-hidden="true">
<use xlink:href="#csdnc-triangledown"></use>
</svg>
</a>
<ul class="dropdown-menu" id="commentCode" aria-labelledby="drop4">
<li><a data-code="html">HTML/XML</a></li>
<li><a data-code="objc">objective-c</a></li>
<li><a data-code="ruby">Ruby</a></li>
<li><a data-code="php">PHP</a></li>
<li><a data-code="csharp">C</a></li>
<li><a data-code="cpp">C++</a></li>
<li><a data-code="javascript">JavaScript</a></li>
<li><a data-code="python">Python</a></li>
<li><a data-code="java">Java</a></li>
<li><a data-code="css">CSS</a></li>
<li><a data-code="sql">SQL</a></li>
<li><a data-code="plain">其它</a></li>
</ul>
</div>
<div class="right-box">
<span id="tip_comment" class="tip">还能输入<em>1000</em>个字符</span>
<input type="submit" class="btn btn-sm btn-red btn-comment" value="发表评论">
</div>
</div>
</form>
</div>
<div class="comment-list-container">
<a id="comments"></a>
<div class="comment-list-box">
</div>
<div id="commentPage" class="pagination-box d-none"></div>
</div>
</div>
<div class="recommend-box">
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/qq_25302531/article/details/80608288,BlogCommendFromBaidu_1,index_0"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/qq_25302531/article/details/80608288,BlogCommendFromBaidu_1,index_0"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/qq_25302531/article/details/80608288" target="_blank" title="YARN and MapReduce的内存优化配置详解 - 青青青的博客(java、大数据小白，一直在学习，从未止步)">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>YARN</em> and <em>MapReduce</em>的<em>内存</em>优化配置详解 - 青青青的博客(java、大数据小白，一直在学习，从未止步) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/9/3/D/3_qq_25302531.jpg" alt="qq_25302531" class="avatar-pic">
<span class="namebox">
<span class="name">qq_25302531</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">06-07</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
217</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/qq_25302531/article/details/80608288" target="_blank" title="YARN and MapReduce的内存优化配置详解 - 青青青的博客(java、大数据小白，一直在学习，从未止步)">
<span class="desc oneline">在Hadoop2.x中, YARN负责管理MapReduce中的资源(内存, CPU等)并且将其打包成Container。使之专注于其擅长的数据处理任务, 将无需考虑资源调度. 如下图所示 ...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/qq_25302531">来自： <span class="blog_title"> 青青青的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/u014665013/article/details/80923044,BlogCommendFromBaidu_2,index_1"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/u014665013/article/details/80923044,BlogCommendFromBaidu_2,index_1"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u014665013/article/details/80923044" target="_blank" title="mapreduce内存分配 - zsweety(博客搬家了，本博同步更新...（zsweety.com）)">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>mapreduce</em><em>内存</em>分配 - zsweety(博客搬家了，本博同步更新...（zsweety.com）) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/9/5/4/3_u014665013.jpg" alt="u014665013" class="avatar-pic">
<span class="namebox">
<span class="name">u014665013</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">07-05</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
382</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u014665013/article/details/80923044" target="_blank" title="mapreduce内存分配 - zsweety(博客搬家了，本博同步更新...（zsweety.com）)">
<span class="desc oneline">稍微有点mapreduce使用经验的同学肯定对OOM不陌生，对的，我目前在mapReduce里面遇到的最多的报错也是内存分配出错，所以看到好多hadoop执行脚本里面有好多关于内存的参数，虽然是知道和...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/u014665013">来自： <span class="blog_title"> zsweety</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/u013980127/article/details/53998324,BlogCommendFromBaidu_3,index_2"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/u013980127/article/details/53998324,BlogCommendFromBaidu_3,index_2"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u013980127/article/details/53998324" target="_blank" title="MapReduce配置与优化 - 梓纾的专栏(分布式、大数据技术)">
<h4 class="text-truncate oneline" style="width: 860px;">
<em>MapReduce</em>配置与优化 - 梓纾的专栏(分布式、大数据技术) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/E/A/4/3_u013980127.jpg" alt="u013980127" class="avatar-pic">
<span class="namebox">
<span class="name">u013980127</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">01-03</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
5044</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u013980127/article/details/53998324" target="_blank" title="MapReduce配置与优化 - 梓纾的专栏(分布式、大数据技术)">
<span class="desc oneline">1. 流程简介参考MapReduce学习笔记之简介（一）2. 配置详解core-site.xml是全局配置，hdfs-site.xml和mapred-site.xml分别是hdfs和mapred的局部...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/u013980127">来自： <span class="blog_title"> 梓纾的专栏</span></a></span>
</p>
</div>
</div>
<!-- 第四范式广告开关 -->
<div class="p4courset3_target _4paradigm_box recommend-item-box clearfix isGreatIcon"><div class="_4paradigm_box clearfix T3" data-track-view="{"mod":"popu_648","con":",https://edu.csdn.net/topic/python115?utm_source=blogt3,"}" data-track-click="{"mod":"popu_648","con":",https://edu.csdn.net/topic/python115?utm_source=blogt3,"}"><a href="https://edu.csdn.net/topic/python115?utm_source=blogt3" target="_blank"><div class="content-box"><h4 class="text-truncate oneline" style="width: 912px;"><span style="color:#ca0c16;">Python全栈</span>学完需要多少钱？</h4><p class="content oneline" style="width: 952px;">零基础学爬虫，你要掌握学习那些技能？需要学多久？</p></div></a></div></div>
<script>window.p4sdk_enable_recommendBox=true</script>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/wer0735/article/details/78711374,BlogCommendFromBaidu_4,index_3"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/wer0735/article/details/78711374,BlogCommendFromBaidu_4,index_3"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/wer0735/article/details/78711374" target="_blank" title="mapreduce on yarn简单内存分配解释 - wer0735的博客(专注于大数据数据仓库)">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>mapreduce</em> on <em>yarn</em>简单<em>内存</em>分配解释 - wer0735的博客(专注于大数据数据仓库) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/1/7/3/3_wer0735.jpg" alt="wer0735" class="avatar-pic">
<span class="namebox">
<span class="name">wer0735</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">12-04</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
235</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/wer0735/article/details/78711374" target="_blank" title="mapreduce on yarn简单内存分配解释 - wer0735的博客(专注于大数据数据仓库)">
<span class="desc oneline">关于mapreduce程序运行在yarn上时内存的分配一直是一个让我蒙圈的事情，单独查任何一个资料都不能很好的理解透彻。于是，最近查了大量的资料，综合各种解释，终于理解到了一个比较清晰的程度，在这里将...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/wer0735">来自： <span class="blog_title"> wer0735的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/victory0508/article/details/69238222,BlogCommendFromBaidu_5,index_4"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/victory0508/article/details/69238222,BlogCommendFromBaidu_5,index_4"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/victory0508/article/details/69238222" target="_blank" title="mapreduce JVM参数配置的问题 - victory0508的专栏">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>mapreduce</em> JVM<em>参数</em>配置的问题 - victory0508的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/B/A/A/3_victory0508.jpg" alt="victory0508" class="avatar-pic">
<span class="namebox">
<span class="name">victory0508</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">04-05</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
276</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/victory0508/article/details/69238222" target="_blank" title="mapreduce JVM参数配置的问题 - victory0508的专栏">
<span class="desc oneline">在mapred-site.xml中添加参数：
mapreduce.map.java.opts
1536
mapreduce.reduce.java.opts
...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/victory0508">来自： <span class="blog_title"> victory0508的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/WuyouJie2008/article/details/50592233,BlogCommendFromBaidu_6,index_5"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/WuyouJie2008/article/details/50592233,BlogCommendFromBaidu_6,index_5"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/WuyouJie2008/article/details/50592233" target="_blank" title="Reduce内存不足的解决方案 - 劫无忧(想要钓到鱼，就需要了解鱼的心理)">
<h4 class="text-truncate oneline" style="width: 860px;">
Reduce<em>内存</em>不足的解决方案 - 劫无忧(想要钓到鱼，就需要了解鱼的心理) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/4/7/A/3_wuyoujie2008.jpg" alt="WuyouJie2008" class="avatar-pic">
<span class="namebox">
<span class="name">WuyouJie2008</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">01-27</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
8034</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/WuyouJie2008/article/details/50592233" target="_blank" title="Reduce内存不足的解决方案 - 劫无忧(想要钓到鱼，就需要了解鱼的心理)">
<span class="desc oneline">MapReduce作业运行时，任务可能会失败，报out of memory错误。这个时候可以采用以下几个过程调优简单粗暴： 加大内存哪个阶段报错就增加那个阶段的内存。以reduce阶段为例，map阶段...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/WuyouJie2008">来自： <span class="blog_title"> 劫无忧</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/leen0304/article/details/53789989,BlogCommendFromBaidu_7,index_6"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/leen0304/article/details/53789989,BlogCommendFromBaidu_7,index_6"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/leen0304/article/details/53789989" target="_blank" title="hadoop2中Mappers和Reducers堆栈的大小配置 - leen0304的博客(> < 一头飞翔的菜鸟 > ">
<h4 class="text-truncate oneline" style="width: 860px;">
hadoop2中Mappers和Reducers堆栈的大小配置 - leen0304的博客(> < 一头飞翔的菜鸟 > <) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/E/B/C/3_leen0304.jpg" alt="leen0304" class="avatar-pic">
<span class="namebox">
<span class="name">leen0304</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">12-21</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
1602</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/leen0304/article/details/53789989" target="_blank" title="hadoop2中Mappers和Reducers堆栈的大小配置 - leen0304的博客(> < 一头飞翔的菜鸟 > ">
<span class="desc oneline">If a YARN container grows beyond its heap size setting, the map or reduce task will fail with an err...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/leen0304">来自： <span class="blog_title"> leen0304的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/clamaa/article/details/70046258,BlogCommendFromBaidu_8,index_7"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/clamaa/article/details/70046258,BlogCommendFromBaidu_8,index_7"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/clamaa/article/details/70046258" target="_blank" title="MapReduce启动的Map/Reduce子任务简要分析 - clamaa的专栏">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>MapReduce</em>启动的Map/Reduce子任务简要分析 - clamaa的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/A/0/9/3_clamaa.jpg" alt="clamaa" class="avatar-pic">
<span class="namebox">
<span class="name">clamaa</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">04-11</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
694</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/clamaa/article/details/70046258" target="_blank" title="MapReduce启动的Map/Reduce子任务简要分析 - clamaa的专栏">
<span class="desc oneline">对于Hadoop来说，是通过在DataNode中启动Map/Reduce java进程的方式来实现分布式计算处理的，那么就从源码层简要分析一下hadoop中启动Map/Reduce任务的过程。
...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/clamaa">来自： <span class="blog_title"> clamaa的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-ad-box"><div id="kp_box_78" data-pid="60" data-track-view="{"mod":"kp_popu_78","con":",,"}"><div id="_cz52656hl3r"><iframe scrolling="no" src="https://pos.baidu.com/s?hei=60&wid=852&di=u3491668&ltu=https%3A%2F%2Fblog.csdn.net%2Fz3935212%2Farticle%2Fdetails%2F78637157&tcn=1541469145&drs=1&ant=0&dc=3&cec=UTF-8&cfv=0&cja=false&pis=-1x-1&cdo=-1&psr=1366x768&par=1366x728&dtm=HTML_POST&dai=1&exps=111000&cmi=2&col=zh-CN&prot=2&pss=1777x4488&pcs=1349x615&tlm=1541469145&ps=3009x402&tpr=1541469145114&chi=2&cce=true&ccd=24&dri=0&ti=Yarn%E4%B8%8BMapreduce%E7%9A%84%E5%86%85%E5%AD%98%E5%8F%82%E6%95%B0%E7%90%86%E8%A7%A3%26xml%E5%8F%82%E6%95%B0%E9%85%8D%E7%BD%AE%20-%20z3935212%E7%9A%84%E4%B8%93%E6%A0%8F%20-%20CSDN%E5%8D%9A%E5%AE%A2&dis=0&cpl=1&ari=2" width="852" height="60" frameborder="0"></iframe></div><script type="text/javascript" src="//rabc1.iteye.com/production/res/rxjg.js?pkcgstj=jm"></script></div><script>$(function(){csdn.track.viewCheck($("#kp_box_78"));});</script></div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/dxl342/article/details/53079155,BlogCommendFromBaidu_9,index_8"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/dxl342/article/details/53079155,BlogCommendFromBaidu_9,index_8"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/dxl342/article/details/53079155" target="_blank" title="[YARN] Yarn下Mapreduce的内存参数理解 - 一只刚刚上路的猿(猿类的多姿多彩)">
<h4 class="text-truncate oneline" style="width: 860px;">
[<em>YARN</em>] <em>Yarn</em>下<em>Mapreduce</em>的<em>内存</em><em>参数</em><em>理解</em> - 一只刚刚上路的猿(猿类的多姿多彩) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/3/F/C/3_dxl342.jpg" alt="dxl342" class="avatar-pic">
<span class="namebox">
<span class="name">dxl342</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">11-08</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
7232</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/dxl342/article/details/53079155" target="_blank" title="[YARN] Yarn下Mapreduce的内存参数理解 - 一只刚刚上路的猿(猿类的多姿多彩)">
<span class="desc oneline">这篇文章算是给自己重新缕清MR下内存参数的含义
Container是什么？
Container就是一个yarn的java进程，在Mapreduce中的AM，MapTask，ReduceTa...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/dxl342">来自： <span class="blog_title"> 一只刚刚上路的猿</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box type_hot_word">
<div class="content clearfix oneline" style="width: 952px;">
<h5 class="float-left">相关热词</h5>
<div class="float-left">
<span>
<a href="https://blog.csdn.net/zhouweiyu/article/details/79028532" target="_blank">
YArn</a>
</span>
<span>
<a href="https://blog.csdn.net/x550392236/article/details/79205812" target="_blank">
yarn</a>
</span>
<span>
<a href="https://blog.csdn.net/zhouweiyu/article/details/79028532" target="_blank">
Yarn</a>
</span>
<span>
<a href="https://blog.csdn.net/zhouweiyu/article/details/79028532" target="_blank">
YARN</a>
</span>
<span>
<a href="https://blog.csdn.net/qq1032355091/article/details/53436349" target="_blank">
yarn关闭</a>
</span>
</div>
</div>
</div>
<div class="recommend-item-box recommend-box-ident" data-track-view="{"mod":"popu_387","con":",https://edu.csdn.net/course/detail/7898?from=recdm&lessonid=161789,BlogCommendFromEdu_10,index_9"}" data-track-click="{"mod":"popu_387","con":",https://edu.csdn.net/course/detail/7898?from=recdm&lessonid=161789,BlogCommendFromEdu_10,index_9"}" data-flg="true">
<a href="https://edu.csdn.net/course/detail/7898?from=recdm&lessonid=161789" target="_blank">
<h4 class="text-truncate oneline" style="width: 814px;">
Hadoop 2.x初级入门到精通(全面深入的Hadoop实战教程) </h4>
<div class="info-box d-flex align-content-center">
<p>
<span class="read-num">学院</span>
</p>
<p>
<span class="date">01-01</span>
</p>
</div>
<p class="content oneline" style="width: 952px;">
适合人群：Hadoop初学者、具有一定Linux系统、Java使用经验,章节：MapReduce编程优化之一配置参数的几种设置方式及优先级 </p>
</a>
</div>
<div class="recommend-item-box blog-expert-recommend-box" style="display: block;">
<div class="d-flex">
<div class="blog-expert-recommend">
<div class="blog-expert">
<div class="blog-expert-flexbox" data-track-view="{"mod":"popu_709","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><div class="blog-expert-item"><div class="blog-expert-info-box"><div class="blog-expert-img-box" data-track-click="{"mod":"popu_709","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><a href="https://blog.csdn.net/toto12974885041" target="_blank"><img src="https://avatar.csdn.net/3/F/F/3_toto1297488504.jpg" alt="toto1297488504"><svg class="icon" aria-hidden="true"><use xlink:href="#csdnc-blogexpert"></use></svg></a></div><div class="info"><span data-track-click="{"mod":"popu_709","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><a href="https://blog.csdn.net/toto1297488504" target="_blank"><h5 class="oneline">toto1297488504</h5></a></span> <p><span data-track-click="{"mod":"popu_710","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><span class="blog-expert-button-follow btn-red-follow" data-name="toto1297488504" data-nick="toto1297488504">关注</span></span><span class="article-num"> 1589篇文章</span></p></div></div></div><div class="blog-expert-item"><div class="blog-expert-info-box"><div class="blog-expert-img-box" data-track-click="{"mod":"popu_709","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><a href="https://blog.csdn.net/Androidlushangderen1" target="_blank"><img src="https://avatar.csdn.net/9/F/0/3_androidlushangderen.jpg" alt="Android路上的人"><svg class="icon" aria-hidden="true"><use xlink:href="#csdnc-blogexpert"></use></svg></a></div><div class="info"><span data-track-click="{"mod":"popu_709","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><a href="https://blog.csdn.net/Androidlushangderen" target="_blank"><h5 class="oneline">Android路上的人</h5></a></span> <p><span data-track-click="{"mod":"popu_710","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><span class="blog-expert-button-follow btn-red-follow" data-name="Androidlushangderen" data-nick="Android路上的人">关注</span></span><span class="article-num"> 297篇文章</span></p></div></div></div><div class="blog-expert-item"><div class="blog-expert-info-box"><div class="blog-expert-img-box" data-track-click="{"mod":"popu_709","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><a href="https://blog.csdn.net/ZYC888881" target="_blank"><img src="https://avatar.csdn.net/C/3/C/3_zyc88888.jpg" alt="ZhaoYingChao88"></a></div><div class="info"><span data-track-click="{"mod":"popu_709","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><a href="https://blog.csdn.net/ZYC88888" target="_blank"><h5 class="oneline">ZhaoYingChao88</h5></a></span> <p><span data-track-click="{"mod":"popu_710","con":"https://blog.csdn.net/z3935212/article/details/78637157"}"><span class="blog-expert-button-follow btn-red-follow" data-name="ZYC88888" data-nick="ZhaoYingChao88">关注</span></span><span class="article-num"> 330篇文章</span></p></div></div></div></div>
</div>
</div>
<div class="blog-expert-load-new" data-index="3" style="display: block;">
<svg version="1.1" viewBox="0 0 200 200" style="enable-background:new 0 0 200 200;" xml:space="preserve">
<path d="M142.5,57.5c-1.3-1.2-2.1-3.3-2.1-5.4c0-4.2,3.3-7.5,7.5-7.5h19.6c4.2,0,7.5,3.3,7.5,7.5s-3.3,7.5-7.5,7.5h-4.2
C170.8,71.2,175,85,175,100c0,41.7-33.8,75-75,75c-4.2,0-7.5-3.3-7.5-7.5c0-4.2,3.3-7.5,7.5-7.5c33.3,0,60-27.1,60-60
C160,83.7,153.8,68.7,142.5,57.5L142.5,57.5z M57.5,142.5c1.2,1.2,2.1,3.3,2.1,5.4c0,4.2-3.3,7.5-7.5,7.5H32.5
c-4.2,0-7.5-3.3-7.5-7.5c0-4.2,3.3-7.5,7.5-7.5h4.2C29.2,128.8,25,115,25,100c0-41.7,33.7-75,75-75c4.2,0,7.5,3.3,7.5,7.5
S104.2,40,100,40c-33.3,0-60,27.1-60,60C40,116.2,46.2,131.2,57.5,142.5z"></path>
</svg>
<span>换一批</span>
</div>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/u010226454/article/details/80907088,BlogCommendESEnWordWeight_11,index_10"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/u010226454/article/details/80907088,BlogCommendESEnWordWeight_11,index_10"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u010226454/article/details/80907088" target="_blank" title="Yarn Container计算和MapReduce的内存配置 - 王茗灏的专栏(it技术分享)">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>Yarn</em> Container计算和<em>MapReduce</em>的<em>内存</em>配置 - 王茗灏的专栏(it技术分享) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/D/5/F/3_u010226454.jpg" alt="u010226454" class="avatar-pic">
<span class="namebox">
<span class="name">u010226454</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">07-04</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
261</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u010226454/article/details/80907088" target="_blank" title="Yarn Container计算和MapReduce的内存配置 - 王茗灏的专栏(it技术分享)">
<span class="desc oneline">在Hadoop2.0中, YARN负责管理MapReduce中的资源(内存, CPU等)并且将其打包成Container. 这样可以精简MapReduce, 使之专注于其擅长的数据处理任务, 将无需考...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/u010226454">来自： <span class="blog_title"> 王茗灏的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/dxl342/article/details/52840455,BlogCommendFromBaidu_12,index_11"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/dxl342/article/details/52840455,BlogCommendFromBaidu_12,index_11"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/dxl342/article/details/52840455" target="_blank" title="YARN的内存和CPU配置 - 一只刚刚上路的猿(猿类的多姿多彩)">
<h4 class="text-truncate oneline" style="width: 860px;">
<em>YARN</em>的<em>内存</em>和CPU配置 - 一只刚刚上路的猿(猿类的多姿多彩) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/3/F/C/3_dxl342.jpg" alt="dxl342" class="avatar-pic">
<span class="namebox">
<span class="name">dxl342</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">10-17</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
2424</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/dxl342/article/details/52840455" target="_blank" title="YARN的内存和CPU配置 - 一只刚刚上路的猿(猿类的多姿多彩)">
<span class="desc oneline">Hadoop YARN同时支持内存和CPU两种资源的调度，本文介绍如何配置YARN对内存和CPU的使用。
YARN作为一个资源调度器，应该考虑到集群里面每一台机子的计算资源，然后根据applic...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/dxl342">来自： <span class="blog_title"> 一只刚刚上路的猿</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/lookqlp/article/details/52096169,BlogCommendFromBaidu_13,index_12"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/lookqlp/article/details/52096169,BlogCommendFromBaidu_13,index_12"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/lookqlp/article/details/52096169" target="_blank" title="关于mapreduce解析xml的方法 - 空中的鱼(毛燥的孩子)">
<h4 class="text-truncate oneline" style="width: 868px;">
关于<em>mapreduce</em>解析<em>xml</em>的方法 - 空中的鱼(毛燥的孩子) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/C/7/C/3_lookqlp.jpg" alt="lookqlp" class="avatar-pic">
<span class="namebox">
<span class="name">lookqlp</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">07-13</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
699</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/lookqlp/article/details/52096169" target="_blank" title="关于mapreduce解析xml的方法 - 空中的鱼(毛燥的孩子)">
<span class="desc oneline">mapreduce的TextInputFormat很方便的处理行行的文本，但遇到xml的时候就很纠结了，曾经采用分隔数据重写FileInputFormat（网上有资料），可以解决此问题，但会获取很多噪...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/lookqlp">来自： <span class="blog_title"> 空中的鱼</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-ad-box"><div id="kp_box_557" data-pid="61" data-track-view="{"mod":"kp_popu_557","con":",,"}"><div class="mediav_ad"><newsfeed class="newsfeed QIHOO__WEB__SO__1541469145050_36" id="QIHOO__WEB__SO__1541469145050_36" style="display:block;margin:0;padding:0;border:none;width:900px;height:84px;overflow-y:hidden;overflow-x:hidden;position:relative;text-align:left;"><info-div id="QIHOO__WEB__SO__1541469145050_36-info" style="zoom:1"><info-div class="QIHOO__WEB__SO__1541469145050_36 singleImage clk" data-href="https://s1.milkjpg.com/s?type=2&r=20&mv_ref=blog.csdn.net&enup=CAABb8xxxQgAAsVxzG8A&mvid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&bid=12b322b56a2c8829&price=AAAAAFvg89UAAAAAAAsS+FSXhpGjwozC0mT6kA==&finfo=DAABCAABAAABnQgAAgAAAGEEAAM/YNU7Ckf2GgAIAAIAAAADCgADWHNtpFdEbu4IAAQAAADJBgAGLbcIAAgAPwTQCgAJAAAAAAACEBgGAAoAAAA&ugi=FcTgigEVyNtrTBUCFSgVKBUAABWRxv+RAxaACBXIAQA&uai=FaixkwIlAhUEFub/oJSPo7LzsAEV8ggls76KjQslABUaAA&ubi=FeDPSBX2ga8CFcicshYVvuq5VhUEFRwWnKuZ+hYW5v+1+5jptvOwATQCFrDAkIAIJQYVvtHnjA8VvgUVADbGnsS0porn1JUBAA&clickid=0&cpx=__OFFSET_X__&cpy=__OFFSET_Y__&cs=__EVENT_TIME_START__&ce=__EVENT_TIME_END__&csign=2a0b5d93e9c8beec&url=http%3A%2F%2Fwww.yuanmeibang.cn%2F" data-pv="https://s1.milkjpg.com/s?type=1&r=20&tid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&finfo=DAABCAABAAABnQgAAgAAAGEEAAM/YNU7Ckf2GgAIAAIAAAADCgADWHNtpFdEbu4IAAQAAADJBgAGLbcIAAgAPwTQCgAJAAAAAAACEBgGAAoAAAA&mv_ref=blog.csdn.net&enup=CAABb8xxxQgAAsVxzG8A&mvid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&bid=12b322b56a2c8829&ugi=FcTgigEVyNtrTBUCFSgVKBUAABWRxv+RAxaACBXIAQA&uai=FaixkwIlAhUEFub/oJSPo7LzsAEV8ggls76KjQslABUaAA&ubi=FeDPSBX2ga8CFcicshYVvuq5VhUEFRwWnKuZ+hYW5v+1+5jptvOwATQCFrDAkIAIJQYVvtHnjA8VvgUVADbGnsS0porn1JUBAA&ds=1&price=AAAAAFvg89UAAAAAAAsS+FSXhpGjwozC0mT6kA==,https://max-l.mediav.com/rtb?type=2&d=100&b=12b322b56a2c8829&p=2255956&l=1136674&s=1&z=1&w=AAAAAFvg89UAAAAAAAsTPUV7yauyeS1wf9AcKw==&k=mJzKmgAAAAA=&i=ctpkINITYDNo&v=17773189023401228040018" data-clk="https://max-l.mediav.com/rtb?type=3&d=100&b=12b322b56a2c8829&p=2255956&l=1136674&s=1&k=mJzKmgAAAAA=&i=ctpkINITYDNo&v=17773189023401228040018&x=__OFFSET_X__&y=__OFFSET_Y__&st=__EVENT_TIME_START__&et=__EVENT_TIME_END__&tc=&turl=">
<info-div class="wrap">
<info-div class="singleImage-img singleImage-img-left">
<info-div class="img" style="background-image:url(https://s3m.mediav.com/galileo/594928-6ac0f44e6542e36ddaf87deede3e3f25.jpg)"><info-div class="ads-tag"></info-div></info-div>
</info-div>
<info-div class="singleImage-body singleImage-body-left">
<info-div class="singleImage-title">中关村股王8年不亏铁律“1272”曝光，震惊众人！</info-div>
<info-div class="singleImage-desc">知恩网络 · 燨燚</info-div>
</info-div>
</info-div>
<info-div style="clear:both;"></info-div>
</info-div>
</info-div></newsfeed></div>
<script type="text/javascript" src="//static.mediav.com/js/mvf_news_feed.js"></script>
<script>
NEWS_FEED({
w: 900,
h : 84,
showid : 'GNKXx7',
placeholderId: "three_ad13",
inject : 'define',
define : {
imagePosition : 'left',
imageBorderRadius : 3,
imageWidth: 90,
imageHeight: 60,
imageFill : 'clip',
displayImage : true,
displayTitle : true,
titleFontSize: 16,
titleFontColor: '#000',
titleFontFamily : 'Microsoft Yahei',
titleFontWeight: 'bold',
titlePaddingTop : 0,
titlePaddingRight : 0,
titlePaddingBottom : 5,
titlePaddingLeft : 16,
displayDesc : true,
descFontSize: 12,
descFontColor: '#8e959a',
descFontFamily : 'Microsoft Yahei',
paddingTop : 10,
paddingRight : 0,
paddingBottom : 0,
paddingLeft : 0,
backgroundColor: '#fff',
hoverColor: '#000'
}
})
</script></div><script>$(function(){csdn.track.viewCheck($("#kp_box_557"));});</script></div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/bloncar/article/details/68940957,BlogCommendFromBaidu_14,index_13"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/bloncar/article/details/68940957,BlogCommendFromBaidu_14,index_13"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/bloncar/article/details/68940957" target="_blank" title="MapReduce处理xml文件（使用旧API） - bloncar的博客">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>MapReduce</em>处理<em>xml</em>文件（使用旧API） - bloncar的博客 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/4/4/F/3_bloncar.jpg" alt="bloncar" class="avatar-pic">
<span class="namebox">
<span class="name">bloncar</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">04-01</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
590</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/bloncar/article/details/68940957" target="_blank" title="MapReduce处理xml文件（使用旧API） - bloncar的博客">
<span class="desc oneline">1）MapReduce项目引入jar包：hadoop-streaming-2.6.5.jar
2）main函数主要代码段：JobConf jobconf = new JobConf(new Conf...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/bloncar">来自： <span class="blog_title"> bloncar的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/huanmieuruoshui/article/details/48031169,BlogCommendFromBaidu_15,index_14"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/huanmieuruoshui/article/details/48031169,BlogCommendFromBaidu_15,index_14"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/huanmieuruoshui/article/details/48031169" target="_blank" title="大数据学习--使用Hadoop2.6.0遇到的相关问题 - 东方伊雪的博客(慢慢走，进步就好。)">
<h4 class="text-truncate oneline" style="width: 860px;">
大数据学习--使用Hadoop2.6.0遇到的相关问题 - 东方伊雪的博客(慢慢走，进步就好。) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/A/D/5/3_huanmieuruoshui.jpg" alt="huanmieuruoshui" class="avatar-pic">
<span class="namebox">
<span class="name">huanmieuruoshui</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">08-27</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
1155</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/huanmieuruoshui/article/details/48031169" target="_blank" title="大数据学习--使用Hadoop2.6.0遇到的相关问题 - 东方伊雪的博客(慢慢走，进步就好。)">
<span class="desc oneline">学习和使用Hadoop2已有半年多，其中遇到了一些常见问题，在网上搜索基本可以找到答案，出处已找不到，我只是对自己遇到的问题做个总结，也感谢解决这些问题的大神们，原谅我已经忘记到底从哪里获得的答案o(...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/huanmieuruoshui">来自： <span class="blog_title"> 东方伊雪的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/dd390738878/article/details/9785679,BlogCommendFromBaidu_16,index_15"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/dd390738878/article/details/9785679,BlogCommendFromBaidu_16,index_15"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/dd390738878/article/details/9785679" target="_blank" title="hadoop2.0 yarn 内存溢出问题解决 - dd390738878的专栏">
<h4 class="text-truncate oneline" style="width: 859px;">
hadoop2.0 <em>yarn</em> <em>内存</em>溢出问题解决 - dd390738878的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/7/C/3/3_dd390738878.jpg" alt="dd390738878" class="avatar-pic">
<span class="namebox">
<span class="name">dd390738878</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">08-06</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
1.3万</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/dd390738878/article/details/9785679" target="_blank" title="hadoop2.0 yarn 内存溢出问题解决 - dd390738878的专栏">
<span class="desc oneline">问题描述：在跑一个mapreduce程序的过程中，提示内存溢出的错误，确定非程序影响。
百度了很多可能的问题产生原因，依次排除后，终于解决了。想把前人的经验总结一下方便后人查阅。
异常提示...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/dd390738878">来自： <span class="blog_title"> dd390738878的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/Little_Fire/article/details/80612671,searchFromBaidu_17,index_16"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/Little_Fire/article/details/80612671,searchFromBaidu_17,index_16"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/Little_Fire/article/details/80612671" target="_blank" title="（转载）MapReduce作业配置参数 - little_fire的博客(致力于大数据分析、机器学习与推荐系统学习与研究的技术小白，但我相信不积跬步无以至千里，不积小流无以成江海)">
<h4 class="text-truncate oneline" style="width: 868px;">
（转载）<em>MapReduce</em>作业配置<em>参数</em> - little_fire的博客(致力于大数据分析、机器学习与推荐系统学习与研究的技术小白，但我相信不积跬步无以至千里，不积小流无以成江海) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/E/9/9/3_little_fire.jpg" alt="Little_Fire" class="avatar-pic">
<span class="namebox">
<span class="name">Little_Fire</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">06-07</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
125</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/Little_Fire/article/details/80612671" target="_blank" title="（转载）MapReduce作业配置参数 - little_fire的博客(致力于大数据分析、机器学习与推荐系统学习与研究的技术小白，但我相信不积跬步无以至千里，不积小流无以成江海)">
<span class="desc oneline">原文：https://blog.csdn.net/qiezikuaichuan/article/details/46682049Note：以下配置可在服务器的mapred-site.xml中配置，作为...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/Little_Fire">来自： <span class="blog_title"> Little_Fire的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/lb812913059/article/details/79900329,searchFromBaidu_18,index_17"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/lb812913059/article/details/79900329,searchFromBaidu_18,index_17"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/lb812913059/article/details/79900329" target="_blank" title="MapReduce优化——配置调优 - 雾幻的博客">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>MapReduce</em>优化——配置调优 - 雾幻的博客 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/1/6/7/3_lb812913059.jpg" alt="lb812913059" class="avatar-pic">
<span class="namebox">
<span class="name">lb812913059</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">04-11</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
335</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/lb812913059/article/details/79900329" target="_blank" title="MapReduce优化——配置调优 - 雾幻的博客">
<span class="desc oneline">
相关链接： MapReduce优化——Combiner与Partitioner
1、配置调优
调优总的原则给shuffle过程尽量多提供内存空间，在map端，可以通过避免多次溢出写磁盘来...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/lb812913059">来自： <span class="blog_title"> 雾幻的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-ad-box"><div id="kp_box_623" data-pid="62" data-track-view="{"mod":"kp_popu_623","con":",,"}"><div id="_yg2ce4fujc" style="width: 100%;"><iframe scrolling="no" src="https://pos.baidu.com/s?hei=60&wid=852&di=u3565311&ltu=https%3A%2F%2Fblog.csdn.net%2Fz3935212%2Farticle%2Fdetails%2F78637157&cpl=1&cja=false&ccd=24&par=1366x728&tlm=1541469145&cdo=-1&dtm=HTML_POST&exps=111000&ant=0&psr=1366x768&tcn=1541469145&dis=0&pcs=1349x615&prot=2&dri=0&cec=UTF-8&chi=2&ps=4088x402&col=zh-CN&ari=2&ti=Yarn%E4%B8%8BMapreduce%E7%9A%84%E5%86%85%E5%AD%98%E5%8F%82%E6%95%B0%E7%90%86%E8%A7%A3%26xml%E5%8F%82%E6%95%B0%E9%85%8D%E7%BD%AE%20-%20z3935212%E7%9A%84%E4%B8%93%E6%A0%8F%20-%20CSDN%E5%8D%9A%E5%AE%A2&cfv=0&cmi=2&dai=2&pss=1777x4555&cce=true&drs=1&dc=3&pis=-1x-1&tpr=1541469145114" width="852" height="60" frameborder="0"></iframe></div><script type="text/javascript" src="//rabc1.iteye.com/production/dkjgal.js?cxpuwupg=g"></script></div><script>$(function(){csdn.track.viewCheck($("#kp_box_623"));});</script></div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/zjwcsdn123/article/details/78958367,BlogCommendFromGuangxin_19,index_18"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/zjwcsdn123/article/details/78958367,BlogCommendFromGuangxin_19,index_18"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/zjwcsdn123/article/details/78958367" target="_blank" title="hadoop属性的配置调优记录 - zjwcsdn123的博客">
<h4 class="text-truncate oneline" style="width: 868px;">
hadoop属性的配置调优记录 - zjwcsdn123的博客 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/F/1/0/3_zjwcsdn123.jpg" alt="zjwcsdn123" class="avatar-pic">
<span class="namebox">
<span class="name">zjwcsdn123</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">01-03</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
142</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/zjwcsdn123/article/details/78958367" target="_blank" title="hadoop属性的配置调优记录 - zjwcsdn123的博客">
<span class="desc oneline">1.配置网页查看hadoop集群状态在浏览器输入http://ip地址:8088/cluster，即可用网页查看hadoop集群状态，配置好之后查看某个作业时会报错，这是因为没有设置域名重定向，我的电...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/zjwcsdn123">来自： <span class="blog_title"> zjwcsdn123的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/u013516966/article/details/72864373,BlogCommendFromGuangxin_20,index_19"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/u013516966/article/details/72864373,BlogCommendFromGuangxin_20,index_19"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u013516966/article/details/72864373" target="_blank" title="yarn 资源配置 - 张普的专栏">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>yarn</em> 资源配置 - 张普的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/F/B/6/3_u013516966.jpg" alt="u013516966" class="avatar-pic">
<span class="namebox">
<span class="name">u013516966</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">06-05</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
542</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u013516966/article/details/72864373" target="_blank" title="yarn 资源配置 - 张普的专栏">
<span class="desc oneline">转载： https://www.iteblog.com/archives/1945.html http://www.tuicool.com/articles/Rf6v6j
YARN...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/u013516966">来自： <span class="blog_title"> 张普的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/maoyg0821/article/details/79768249,BlogCommendFromGuangxin_21,index_20"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/maoyg0821/article/details/79768249,BlogCommendFromGuangxin_21,index_20"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/maoyg0821/article/details/79768249" target="_blank" title="Yarn 内存分配管理机制及相关参数配置(yarn性能调优) - maoyg0821的专栏">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>Yarn</em> <em>内存</em>分配管理机制及相关<em>参数</em>配置(<em>yarn</em>性能调优) - maoyg0821的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/4/5/3/3_maoyg0821.jpg" alt="maoyg0821" class="avatar-pic">
<span class="namebox">
<span class="name">maoyg0821</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">03-31</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
630</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/maoyg0821/article/details/79768249" target="_blank" title="Yarn 内存分配管理机制及相关参数配置(yarn性能调优) - maoyg0821的专栏">
<span class="desc oneline">一、相关配置情况关于Yarn内存分配与管理，主要涉及到了ResourceManage、ApplicationMatser、NodeManager这几个概念，相关的优化也要紧紧围绕着这几方面来开展。这里...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/maoyg0821">来自： <span class="blog_title"> maoyg0821的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/lively1982/article/details/50598847,BlogCommendFromGuangxin_22,index_21"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/lively1982/article/details/50598847,BlogCommendFromGuangxin_22,index_21"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/lively1982/article/details/50598847" target="_blank" title="Yarn资源分配性能调优 - 茄肥猫的窝(erp db os )">
<h4 class="text-truncate oneline" style="width: 859px;">
<em>Yarn</em>资源分配性能调优 - 茄肥猫的窝(erp db os ) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/6/8/B/3_lively1982.jpg" alt="lively1982" class="avatar-pic">
<span class="namebox">
<span class="name">lively1982</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">01-28</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
1.1万</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/lively1982/article/details/50598847" target="_blank" title="Yarn资源分配性能调优 - 茄肥猫的窝(erp db os )">
<span class="desc oneline">日志：
Container [pid=134663,containerID=container_1430287094897_0049_02_067966] is running beyond phy...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/lively1982">来自： <span class="blog_title"> 茄肥猫的窝</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/genghaihua/article/details/50441037,BlogCommendFromGuangxin_23,index_22"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/genghaihua/article/details/50441037,BlogCommendFromGuangxin_23,index_22"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/genghaihua/article/details/50441037" target="_blank" title="Yarn下的YarnChild启动个数决定参数 - genghaihua的专栏">
<h4 class="text-truncate oneline" style="width: 860px;">
<em>Yarn</em>下的<em>Yarn</em>Child启动个数决定<em>参数</em> - genghaihua的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/C/F/C/3_genghaihua.jpg" alt="genghaihua" class="avatar-pic">
<span class="namebox">
<span class="name">genghaihua</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">12-31</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
2579</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/genghaihua/article/details/50441037" target="_blank" title="Yarn下的YarnChild启动个数决定参数 - genghaihua的专栏">
<span class="desc oneline">Yarn下的mapper和reducer并发执行个数有什么决定的呢？
由调度的资源决定的，也就是说启动的YarnChild个数多少取决于资源的分配和free的资源量
参数说明：
conf/yar...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/genghaihua">来自： <span class="blog_title"> genghaihua的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-ad-box"><div id="kp_box_555" data-pid="63" data-track-view="{"mod":"kp_popu_555","con":",,"}"><div class="mediav_ad"><newsfeed class="newsfeed QIHOO__WEB__SO__1541469145594_805" id="QIHOO__WEB__SO__1541469145594_805" style="display:block;margin:0;padding:0;border:none;width:900px;height:84px;overflow-y:hidden;overflow-x:hidden;position:relative;text-align:left;"><info-div id="QIHOO__WEB__SO__1541469145594_805-info" style="zoom:1"><info-div class="QIHOO__WEB__SO__1541469145594_805 singleImage clk" data-href="https://ssxd.mediav.com/s?type=2&r=20&mv_ref=blog.csdn.net&enup=CAABb8xxxQgAAsVxzG8A&mvid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&bid=12b322b56a2c8829&price=AAAAAFvg89UAAAAAAAsT+KddP3zidrPDkGYTrg==&finfo=DAABCAABAAAAxQgAAgAAAHYEAAM/Vy2G4Oh1/gAIAAIAAAADCgADWHN2vQzrnAUIAAQAAACLBgAGLbcIAAgAHg9QCgAJAAAAAAACEBgGAAoAAAA&ugi=FcTgigEVyNtrTBUCFSgVKBUAABWRxv+RAxaACBXIAQA&uai=FaixkwIlAhUEFub/oJSPo7LzsAEV8ggls76KjQslABUaAA&ubi=Faq9RxXWma4CFeawrhYV9tmFVxUEFSIWzqGr9BYW5v/K4qKvu/OwATQEFrDAECUGFfmIzJcOFbIBFQA24qKC1rCqnLclAA&clickid=0&cpx=__OFFSET_X__&cpy=__OFFSET_Y__&cs=__EVENT_TIME_START__&ce=__EVENT_TIME_END__&csign=370fdc59e42ce4d3&url=http%3A%2F%2Fbaike.szaiqing9.cn%2Feducation%2Fself-study3%2F%3Fduo-1105-1" data-pv="https://ssxd.mediav.com/s?type=1&r=20&tid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&finfo=DAABCAABAAAAxQgAAgAAAHYEAAM/Vy2G4Oh1/gAIAAIAAAADCgADWHN2vQzrnAUIAAQAAACLBgAGLbcIAAgAHg9QCgAJAAAAAAACEBgGAAoAAAA&mv_ref=blog.csdn.net&enup=CAABb8xxxQgAAsVxzG8A&mvid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&bid=12b322b56a2c8829&ugi=FcTgigEVyNtrTBUCFSgVKBUAABWRxv+RAxaACBXIAQA&uai=FaixkwIlAhUEFub/oJSPo7LzsAEV8ggls76KjQslABUaAA&ubi=Faq9RxXWma4CFeawrhYV9tmFVxUEFSIWzqGr9BYW5v/K4qKvu/OwATQEFrDAECUGFfmIzJcOFbIBFQA24qKC1rCqnLclAA&ds=2&price=AAAAAFvg89UAAAAAAAsT+KddP3zidrPDkGYTrg==,https://max-l.mediav.com/rtb?type=2&d=100&b=12b322b56a2c8829&p=2255956&l=1136674&s=2&z=1&w=AAAAAFvg89UAAAAAAAsUQ/w43CsUeGLwEPM4WQ==&k=mJzKmgAAAAA=&i=cWbkINITYDN8&v=17773189023401228040018" data-clk="https://max-l.mediav.com/rtb?type=3&d=100&b=12b322b56a2c8829&p=2255956&l=1136674&s=2&k=mJzKmgAAAAA=&i=cWbkINITYDN8&v=17773189023401228040018&x=__OFFSET_X__&y=__OFFSET_Y__&st=__EVENT_TIME_START__&et=__EVENT_TIME_END__&tc=&turl=">
<info-div class="wrap">
<info-div class="singleImage-img singleImage-img-left">
<info-div class="img" style="background-image:url(https://s3m.mediav.com/galileo/585557-06c5c8ffee3356a1d78a582fa2d9c67c.jpeg)"><info-div class="ads-tag"></info-div></info-div>
</info-div>
<info-div class="singleImage-body singleImage-body-left">
<info-div class="singleImage-title">11月起，中关村面向上班族开放一年制本科课程，学信网可查</info-div>
<info-div class="singleImage-desc">金领教育 · 顶新</info-div>
</info-div>
</info-div>
<info-div style="clear:both;"></info-div>
</info-div>
</info-div></newsfeed></div>
<script type="text/javascript" src="//static.mediav.com/js/mvf_news_feed.js"></script>
<script>
NEWS_FEED({
w: 900,
h : 84,
showid : 'GNKXx7',
placeholderId: "three_ad23",
inject : 'define',
define : {
imagePosition : 'left',
imageBorderRadius : 3,
imageWidth: 90,
imageHeight: 60,
imageFill : 'clip',
displayImage : true,
displayTitle : true,
titleFontSize: 16,
titleFontColor: '#000',
titleFontFamily : 'Microsoft Yahei',
titleFontWeight: 'bold',
titlePaddingTop : 0,
titlePaddingRight : 0,
titlePaddingBottom : 5,
titlePaddingLeft : 16,
displayDesc : true,
descFontSize: 12,
descFontColor: '#8e959a',
descFontFamily : 'Microsoft Yahei',
paddingTop : 10,
paddingRight : 0,
paddingBottom : 0,
paddingLeft : 0,
backgroundColor: '#fff',
hoverColor: '#000'
}
})
</script></div><script>$(function(){csdn.track.viewCheck($("#kp_box_555"));});</script></div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/ZYC88888/article/details/78544229,BlogCommendFromQuerySearch_24,index_23"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/ZYC88888/article/details/78544229,BlogCommendFromQuerySearch_24,index_23"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/ZYC88888/article/details/78544229" target="_blank" title="HDP YARN MapReduce参数调优建议 - 赵英超的博客(相互学习，共同进步)">
<h4 class="text-truncate oneline" style="width: 868px;">
HDP <em>YARN</em> <em>MapReduce</em><em>参数</em>调优建议 - 赵英超的博客(相互学习，共同进步) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/C/3/C/3_zyc88888.jpg" alt="ZYC88888" class="avatar-pic">
<span class="namebox">
<span class="name">ZYC88888</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">11-15</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
521</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/ZYC88888/article/details/78544229" target="_blank" title="HDP YARN MapReduce参数调优建议 - 赵英超的博客(相互学习，共同进步)">
<span class="desc oneline">HDP平台参数调优建议
根据上面介绍的相关知识，我们就可以根据我们的实际情况作出相关参数的设置，当然还需要在运行测试过程中不断检验和调整。
以下是hortonworks给出的配置建议：...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/ZYC88888">来自： <span class="blog_title"> 赵英超的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/ruiyiin/article/details/77324634,BlogCommendFromQuerySearch_25,index_24"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/ruiyiin/article/details/77324634,BlogCommendFromQuerySearch_25,index_24"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/ruiyiin/article/details/77324634" target="_blank" title="Yarn下Mapreduce的内存参数理解 - ruiyiin的专栏">
<h4 class="text-truncate oneline" style="width: 868px;">
<em>Yarn</em>下<em>Mapreduce</em>的<em>内存</em><em>参数</em><em>理解</em> - ruiyiin的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/A/B/3/3_ruiyiin.jpg" alt="ruiyiin" class="avatar-pic">
<span class="namebox">
<span class="name">ruiyiin</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">08-17</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
130</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/ruiyiin/article/details/77324634" target="_blank" title="Yarn下Mapreduce的内存参数理解 - ruiyiin的专栏">
<span class="desc oneline">这篇文章算是给自己重新缕清MR下内存参数的含义
Container是什么？
Container就是一个yarn的Java进程，在Mapreduce中的AM，MapTask，ReduceTa...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/ruiyiin">来自： <span class="blog_title"> ruiyiin的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/xjping0794/article/details/77930796,BlogCommendFromQuerySearch_26,index_25"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/xjping0794/article/details/77930796,BlogCommendFromQuerySearch_26,index_25"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/xjping0794/article/details/77930796" target="_blank" title="yarn下mapreduce内存参数理解 - xjping0794的博客(数据仓库及大数据领域)">
<h4 class="text-truncate oneline" style="width: 876px;">
<em>yarn</em>下<em>mapreduce</em><em>内存</em><em>参数</em><em>理解</em> - xjping0794的博客(数据仓库及大数据领域) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/D/6/6/3_xjping0794.jpg" alt="xjping0794" class="avatar-pic">
<span class="namebox">
<span class="name">xjping0794</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">09-11</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
85</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/xjping0794/article/details/77930796" target="_blank" title="yarn下mapreduce内存参数理解 - xjping0794的博客(数据仓库及大数据领域)">
<span class="desc oneline">转自：http://blog.csdn.net/dxl342/article/details/53079155
Container是什么？
Container就是一个yarn的java...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/xjping0794">来自： <span class="blog_title"> xjping0794的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/u011750989/article/details/40424331,BlogCommendFromQuerySearch_27,index_26"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/u011750989/article/details/40424331,BlogCommendFromQuerySearch_27,index_26"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u011750989/article/details/40424331" target="_blank" title="yarn环境跑mapreduce报错某个container 600s未报告进度超时被杀 - 数据技术控(学习笔记备忘录。。。)">
<h4 class="text-truncate oneline" style="width: 860px;">
<em>yarn</em>环境跑<em>mapreduce</em>报错某个container 600s未报告进度超时被杀 - 数据技术控(学习笔记备忘录。。。) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/6/C/9/3_u011750989.jpg" alt="u011750989" class="avatar-pic">
<span class="namebox">
<span class="name">u011750989</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">10-25</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
1651</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u011750989/article/details/40424331" target="_blank" title="yarn环境跑mapreduce报错某个container 600s未报告进度超时被杀 - 数据技术控(学习笔记备忘录。。。)">
<span class="desc oneline">发现每次reduce阶段跑到98%，相关的container被杀，报出的log大概</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/u011750989">来自： <span class="blog_title"> 数据技术控</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/qq_15300683/article/details/80063167,BlogCommendFromQuerySearch_28,index_27"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/qq_15300683/article/details/80063167,BlogCommendFromQuerySearch_28,index_27"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/qq_15300683/article/details/80063167" target="_blank" title="Spark on YARN的重要参数 - qq_15300683的博客">
<h4 class="text-truncate oneline" style="width: 868px;">
Spark on <em>YARN</em>的重要<em>参数</em> - qq_15300683的博客 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/D/B/C/3_qq_15300683.jpg" alt="qq_15300683" class="avatar-pic">
<span class="namebox">
<span class="name">qq_15300683</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">04-24</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
329</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/qq_15300683/article/details/80063167" target="_blank" title="Spark on YARN的重要参数 - qq_15300683的博客">
<span class="desc oneline">Spark属性或者去源码找Class类 SparkSubmitArguments，最全了属性名称默认含义spark.yarn.am.memory512m用于客户端模式下的YARN Applicatio...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/qq_15300683">来自： <span class="blog_title"> qq_15300683的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-ad-box"><div id="kp_box_626" data-pid="64" data-track-view="{"mod":"kp_popu_626","con":",,"}"><div><iframe scrolling="no" src="//pos.baidu.com/s?hei=60&wid=852&di=u3565587&ltu=https%3A%2F%2Fblog.csdn.net%2Fz3935212%2Farticle%2Fdetails%2F78637157&prot=2&tlm=1541469145&chi=2&psr=1366x768&exps=111000,119011&pss=1767x5557&dtm=HTML_POST&pis=-1x-1&cec=UTF-8&drs=1&par=1366x728&ari=2&ti=Yarn%E4%B8%8BMapreduce%E7%9A%84%E5%86%85%E5%AD%98%E5%8F%82%E6%95%B0%E7%90%86%E8%A7%A3%26xml%E5%8F%82%E6%95%B0%E9%85%8D%E7%BD%AE%20-%20z3935212%E7%9A%84%E4%B8%93%E6%A0%8F%20-%20CSDN%E5%8D%9A%E5%AE%A2&cfv=0&dri=0&cdo=-1&tpr=1541469145114&dai=3&col=zh-CN&tcn=1541469146&ant=0&ccd=24&cmi=2&cja=false&cce=true&ps=5090x402&dis=0&cpl=1&pcs=1349x615&dc=3" width="852" height="60" frameborder="0"></iframe></div><script type="text/javascript" src="//rabc1.iteye.com/production/wxzgt3.js?rme=jljjon"></script></div><script>$(function(){csdn.track.viewCheck($("#kp_box_626"));});</script></div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/oDaiLiDong/article/details/50951138,BlogCommendFromBaidu_29,index_28"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/oDaiLiDong/article/details/50951138,BlogCommendFromBaidu_29,index_28"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/oDaiLiDong/article/details/50951138" target="_blank" title="YARN和MapReduce的内存设置参考 - 代立冬的技术之旅(专注架构 外功修行,内功修神)">
<h4 class="text-truncate oneline" style="width: 860px;">
<em>YARN</em>和<em>MapReduce</em>的<em>内存</em>设置参考 - 代立冬的技术之旅(专注架构 外功修行,内功修神) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/7/2/4/3_odailidong.jpg" alt="oDaiLiDong" class="avatar-pic">
<span class="namebox">
<span class="name">oDaiLiDong</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">03-21</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
6141</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/oDaiLiDong/article/details/50951138" target="_blank" title="YARN和MapReduce的内存设置参考 - 代立冬的技术之旅(专注架构 外功修行,内功修神)">
<span class="desc oneline">如何确定Yarn中容器Container，Mapreduce相关参数的内存设置，对于初始集群，由于不知道集群的类型(如CPU密集、IO密集还是网络密集)我们需要根据经验提供给我们一个参考配置值，来作为...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/oDaiLiDong">来自： <span class="blog_title"> 代立冬的技术之旅</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/strangerzz/article/details/51085922,BlogCommendFromBaidu_30,index_29"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/strangerzz/article/details/51085922,BlogCommendFromBaidu_30,index_29"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/strangerzz/article/details/51085922" target="_blank" title="Hadoop、Hive 参数说明 - harper(level up!)">
<h4 class="text-truncate oneline" style="width: 860px;">
Hadoop、Hive <em>参数</em>说明 - harper(level up!) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/5/D/5/3_strangerzz.jpg" alt="strangerzz" class="avatar-pic">
<span class="namebox">
<span class="name">strangerzz</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">04-07</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
1114</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/strangerzz/article/details/51085922" target="_blank" title="Hadoop、Hive 参数说明 - harper(level up!)">
<span class="desc oneline">最近在用破机器跑mapreduce任务，部署的是社区版hadoop，参数都要手动调整。虽然hadoop已经火了很久了，但是网上关于参数的介绍基本都处于’语焉不详’的状态，大部分还都是无责任转载，哪个版...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/strangerzz">来自： <span class="blog_title"> Harper</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/wisgood/article/details/42641731,BlogCommendFromBaidu_31,index_30"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/wisgood/article/details/42641731,BlogCommendFromBaidu_31,index_30"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/wisgood/article/details/42641731" target="_blank" title="新旧Java MapReduce API的差异 - wisgood的专栏">
<h4 class="text-truncate oneline" style="width: 860px;">
新旧Java <em>MapReduce</em> API的差异 - wisgood的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/9/6/B/3_wisgood.jpg" alt="wisgood" class="avatar-pic">
<span class="namebox">
<span class="name">wisgood</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">01-12</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
2902</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/wisgood/article/details/42641731" target="_blank" title="新旧Java MapReduce API的差异 - wisgood的专栏">
<span class="desc oneline">摘录自 Hadoop权威指南 1、版本区别Hadoop在0.20.0版本中第一次使用新的API，部分早期的0.20.0版本不支持使用旧的API，但在接下来的1.x和2.x版本中新旧API都可以使用。新...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/wisgood">来自： <span class="blog_title"> wisgood的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/aijiudu/article/details/72353510,BlogCommendFromBaidu_32,index_31"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/aijiudu/article/details/72353510,BlogCommendFromBaidu_32,index_31"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/aijiudu/article/details/72353510" target="_blank" title="MapReduce过程详解及其性能优化 - aijiudu的博客">
<h4 class="text-truncate oneline" style="width: 859px;">
<em>MapReduce</em>过程详解及其性能优化 - aijiudu的博客 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/F/A/8/3_aijiudu.jpg" alt="aijiudu" class="avatar-pic">
<span class="namebox">
<span class="name">aijiudu</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">05-16</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
1.4万</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/aijiudu/article/details/72353510" target="_blank" title="MapReduce过程详解及其性能优化 - aijiudu的博客">
<span class="desc oneline">从JVM的角度：Map阶段包括：
第一读数据：从HDFS读取数据
第二处理数据：
第三写数据到磁盘
Shuffle和Reduce阶段包括：
一、Copy
二、MergeSort
1）内存到内存Merg...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/aijiudu">来自： <span class="blog_title"> aijiudu的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/bbsyhl/article/details/49637695,BlogCommendFromBaidu_33,index_32"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/bbsyhl/article/details/49637695,BlogCommendFromBaidu_33,index_32"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/bbsyhl/article/details/49637695" target="_blank" title="如何判定并设置YARN 和MapReduce 内存 - bbsyhl的专栏">
<h4 class="text-truncate oneline" style="width: 868px;">
如何判定并设置<em>YARN</em> 和<em>MapReduce</em> <em>内存</em> - bbsyhl的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/E/2/6/3_bbsyhl.jpg" alt="bbsyhl" class="avatar-pic">
<span class="namebox">
<span class="name">bbsyhl</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">11-04</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
256</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/bbsyhl/article/details/49637695" target="_blank" title="如何判定并设置YARN 和MapReduce 内存 - bbsyhl的专栏">
<span class="desc oneline">hadoop 内存调优</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/bbsyhl">来自： <span class="blog_title"> bbsyhl的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-ad-box"><div id="kp_box_84" data-pid="65" data-track-view="{"mod":"kp_popu_84","con":",,"}"><div class="mediav_ad"><newsfeed class="newsfeed QIHOO__WEB__SO__1541469146504_439" id="QIHOO__WEB__SO__1541469146504_439" style="display:block;margin:0;padding:0;border:none;width:900px;height:84px;overflow-y:hidden;overflow-x:hidden;position:relative;text-align:left;"><info-div id="QIHOO__WEB__SO__1541469146504_439-info" style="zoom:1"><info-div class="QIHOO__WEB__SO__1541469146504_439 singleImage clk" data-href="https://ssxd.mediav.com/s?type=2&r=20&mv_ref=blog.csdn.net&enup=CAABb8xxxQgAAsVxzG8A&mvid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&bid=12b322b56a2c8829&price=AAAAAFvg89UAAAAAAAsU9ICuakWIFRCeu6uFig==&finfo=DAABCAABAAAApQgAAgAAAEIEAAM/XCjL8FmrdQAIAAIAAAADCgADWHN/1WGzAbIIAAQAAABgBgAGLbcIAAgAGS1QCgAJAAAAAAACEBgGAAoAAAA&ugi=FcTgigEVyNtrTBUCFSgVKBUAABWRxv+RAxaACBXIAQA&uai=FaixkwIlAhUEFub/oJSPo7LzsAEV8ggls76KjQslABUaAA&ubi=FdKoHhXSnvoBFcSwoBMV9KW4UBUEFSIWsMeuoBQW5v/fyaz1v/OwATQGFrDAECUGFYHj5ZcFFbYCFQA2wvy8nam69sSPAQA&clickid=0&cpx=__OFFSET_X__&cpy=__OFFSET_Y__&cs=__EVENT_TIME_START__&ce=__EVENT_TIME_END__&csign=06c94a4a4cd52eab&url=http%3A%2F%2Fwww.ledguhon.com" data-pv="https://ssxd.mediav.com/s?type=1&r=20&tid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&finfo=DAABCAABAAAApQgAAgAAAEIEAAM/XCjL8FmrdQAIAAIAAAADCgADWHN/1WGzAbIIAAQAAABgBgAGLbcIAAgAGS1QCgAJAAAAAAACEBgGAAoAAAA&mv_ref=blog.csdn.net&enup=CAABb8xxxQgAAsVxzG8A&mvid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&bid=12b322b56a2c8829&ugi=FcTgigEVyNtrTBUCFSgVKBUAABWRxv+RAxaACBXIAQA&uai=FaixkwIlAhUEFub/oJSPo7LzsAEV8ggls76KjQslABUaAA&ubi=FdKoHhXSnvoBFcSwoBMV9KW4UBUEFSIWsMeuoBQW5v/fyaz1v/OwATQGFrDAECUGFYHj5ZcFFbYCFQA2wvy8nam69sSPAQA&ds=3&price=AAAAAFvg89UAAAAAAAsU9ICuakWIFRCeu6uFig==,https://max-l.mediav.com/rtb?type=2&d=100&b=12b322b56a2c8829&p=2255956&l=1136674&s=3&z=1&w=AAAAAFvg89UAAAAAAAsVMs7nxraArhk11A4G8A==&k=mJzKmgAAAAA=&i=cWTkINITYD4P&v=17773189023401228040018" data-clk="https://max-l.mediav.com/rtb?type=3&d=100&b=12b322b56a2c8829&p=2255956&l=1136674&s=3&k=mJzKmgAAAAA=&i=cWTkINITYD4P&v=17773189023401228040018&x=__OFFSET_X__&y=__OFFSET_Y__&st=__EVENT_TIME_START__&et=__EVENT_TIME_END__&tc=&turl=">
<info-div class="wrap">
<info-div class="singleImage-img singleImage-img-left">
<info-div class="img" style="background-image:url(https://s3m.mediav.com/galileo/231af29e1fe0bbff50f5e621f01c3ad0.jpg)"><info-div class="ads-tag"></info-div></info-div>
</info-div>
<info-div class="singleImage-body singleImage-body-left">
<info-div class="singleImage-title">广州巨宏光电草帽LED灯珠选购定制</info-div>
<info-div class="singleImage-desc">巨宏光电 · 顶新</info-div>
</info-div>
</info-div>
<info-div style="clear:both;"></info-div>
</info-div>
</info-div></newsfeed></div>
<script type="text/javascript" src="//static.mediav.com/js/mvf_news_feed.js"></script>
<script>
NEWS_FEED({
w: 900,
h : 84,
showid : 'GNKXx7',
placeholderId: "three_ad33",
inject : 'define',
define : {
imagePosition : 'left',
imageBorderRadius : 3,
imageWidth: 90,
imageHeight: 60,
imageFill : 'clip',
displayImage : true,
displayTitle : true,
titleFontSize: 16,
titleFontColor: '#000',
titleFontFamily : 'Microsoft Yahei',
titleFontWeight: 'bold',
titlePaddingTop : 0,
titlePaddingRight : 0,
titlePaddingBottom : 5,
titlePaddingLeft : 16,
displayDesc : true,
descFontSize: 12,
descFontColor: '#8e959a',
descFontFamily : 'Microsoft Yahei',
paddingTop : 10,
paddingRight : 0,
paddingBottom : 0,
paddingLeft : 0,
backgroundColor: '#fff',
hoverColor: '#000'
}
})
</script></div><script>$(function(){csdn.track.viewCheck($("#kp_box_84"));});</script></div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/shenxiaoming77/article/details/17766781,BlogCommendFromBaidu_34,index_33"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/shenxiaoming77/article/details/17766781,BlogCommendFromBaidu_34,index_33"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/shenxiaoming77/article/details/17766781" target="_blank" title="关于yarn平台下mapreduce运行出现的虚拟内存溢出问题 - michael_shentu">
<h4 class="text-truncate oneline" style="width: 860px;">
关于<em>yarn</em>平台下<em>mapreduce</em>运行出现的虚拟<em>内存</em>溢出问题 - michael_shentu </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/7/4/3/3_shenxiaoming77.jpg" alt="shenxiaoming77" class="avatar-pic">
<span class="namebox">
<span class="name">shenxiaoming77</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">01-02</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
4089</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/shenxiaoming77/article/details/17766781" target="_blank" title="关于yarn平台下mapreduce运行出现的虚拟内存溢出问题 - michael_shentu">
<span class="desc oneline">最近在测试一个统计类型的mapreduce 应用的job新功能时候，reducetask在运行中直接抛出了下面的异常：
Container [pid=23708,containerID=contai...</span>
</a>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/stark_summer/article/details/48494391,BlogCommendFromBaidu_35,index_34"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/stark_summer/article/details/48494391,BlogCommendFromBaidu_35,index_34"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/stark_summer/article/details/48494391" target="_blank" title="yarn & mapreduce 配置参数总结 - stark_summer的专栏 (专注于开发分布式任务调度框架、分布式同步rpc、异步mq消息队列、分布式日志检索框架、hadoop、spark、scala等技术 如果我的写的文章能对您有帮助，请您能给点捐助,请看首页置顶)">
<h4 class="text-truncate oneline" style="width: 859px;">
<em>yarn</em> & <em>mapreduce</em> 配置<em>参数</em>总结 - stark_summer的专栏 (专注于开发分布式任务调度框架、分布式同步rpc、异步mq消息队列、分布式日志检索框架、hadoop、spark、scala等技术 如果我的写的文章能对您有帮助，请您能给点捐助,请看首页置顶) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/3/E/1/3_stark_summer.jpg" alt="stark_summer" class="avatar-pic">
<span class="namebox">
<span class="name">stark_summer</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">09-16</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
3.4万</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/stark_summer/article/details/48494391" target="_blank" title="yarn & mapreduce 配置参数总结 - stark_summer的专栏 (专注于开发分布式任务调度框架、分布式同步rpc、异步mq消息队列、分布式日志检索框架、hadoop、spark、scala等技术 如果我的写的文章能对您有帮助，请您能给点捐助,请看首页置顶)">
<span class="desc oneline">配置
设置nodemanager 总内存大小为32G，在yarn-site.xml 增加如下内容：
yarn.nodemanager.resource.memory-mb
32768
con...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/stark_summer">来自： <span class="blog_title"> stark_summer的专栏 </span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/u011317245/article/details/52835821,BlogCommendFromBaidu_36,index_35"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/u011317245/article/details/52835821,BlogCommendFromBaidu_36,index_35"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u011317245/article/details/52835821" target="_blank" title="Hadoop 参数 mapred.map.child.java.opts - zhuhailong(心无旁骛)">
<h4 class="text-truncate oneline" style="width: 860px;">
Hadoop <em>参数</em> mapred.map.child.java.opts - zhuhailong(心无旁骛) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/F/7/2/3_u011317245.jpg" alt="u011317245" class="avatar-pic">
<span class="namebox">
<span class="name">u011317245</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">10-17</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
6650</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u011317245/article/details/52835821" target="_blank" title="Hadoop 参数 mapred.map.child.java.opts - zhuhailong(心无旁骛)">
<span class="desc oneline">Hadoop mapred.map.child.java.opts 解决 outofmemory问题</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/u011317245">来自： <span class="blog_title"> zhuhailong</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/suifeng3051/article/details/45477773,BlogCommendFromBaidu_37,index_36"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/suifeng3051/article/details/45477773,BlogCommendFromBaidu_37,index_36"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/suifeng3051/article/details/45477773" target="_blank" title="Yarn 内存分配管理机制及相关参数配置 - heaven wang 的专栏">
<h4 class="text-truncate oneline" style="width: 859px;">
<em>Yarn</em> <em>内存</em>分配管理机制及相关<em>参数</em>配置 - heaven wang 的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/E/4/6/3_suifeng3051.jpg" alt="suifeng3051" class="avatar-pic">
<span class="namebox">
<span class="name">suifeng3051</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">05-04</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
1.8万</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/suifeng3051/article/details/45477773" target="_blank" title="Yarn 内存分配管理机制及相关参数配置 - heaven wang 的专栏">
<span class="desc oneline">理解Yarn的内存管理与分配机制，对于我们搭建、部署集群，开发维护应用都是尤为重要的，对于这方面我做了一些调研供大家参考。
一、相关配置情况
关于Yarn内存分配与管理，主要涉及到了ResourceM...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/suifeng3051">来自： <span class="blog_title"> Heaven Wang 的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/weixin_39182877/article/details/79315376,BlogCommendFromBaidu_38,index_37"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/weixin_39182877/article/details/79315376,BlogCommendFromBaidu_38,index_37"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/weixin_39182877/article/details/79315376" target="_blank" title="YARN and MapReduce的【内存】优化配置详解 - weixin_39182877的博客">
<h4 class="text-truncate oneline" style="width: 876px;">
<em>YARN</em> and <em>MapReduce</em>的【<em>内存</em>】优化配置详解 - weixin_39182877的博客 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/D/7/7/3_weixin_39182877.jpg" alt="weixin_39182877" class="avatar-pic">
<span class="namebox">
<span class="name">weixin_39182877</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">02-12</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
52</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/weixin_39182877/article/details/79315376" target="_blank" title="YARN and MapReduce的【内存】优化配置详解 - weixin_39182877的博客">
<span class="desc oneline">转载自：http://blog.itpub.net/30089851/viewspace-2127850/
在Hadoop2.x中, YARN负责管理MapReduce中的资源(内存, CPU等)并...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/weixin_39182877">来自： <span class="blog_title"> weixin_39182877的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-ad-box"><div id="kp_box_87" data-pid="66" data-track-view="{"mod":"kp_popu_87","con":",,"}"><div class="mediav_ad"><newsfeed class="newsfeed QIHOO__WEB__SO__1541469148860_337" id="QIHOO__WEB__SO__1541469148860_337" style="display:block;margin:0;padding:0;border:none;width:900px;height:84px;overflow-y:hidden;overflow-x:hidden;position:relative;text-align:left;"><info-div id="QIHOO__WEB__SO__1541469148860_337-info" style="zoom:1"><info-div class="QIHOO__WEB__SO__1541469148860_337 singleImage clk" data-href="https://s1.milkjpg.com/s?type=2&r=20&mv_ref=blog.csdn.net&enup=CAABb8xxxQgAAsVxzG8A&mvid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&bid=12b322b67646b827&price=AAAAAFvg89kAAAAAAARGdWsW1LFh2DoWa37I4Q==&finfo=DAABCAABAAAA1AgAAgAAAFkEAAM/Sgsn6kHG8wAIAAIAAAADCgADIazU9hYtoHMIAAQAAAELBgAGLbcIAAgAIFlACgAJAAAAAAACEBQGAAoAAAA&ugi=FcTgigEVyNtrTBUCFSgVKBUAABWRxv+RAxaACBXIAQA&uai=FeTGlAIlAhUEFuaPjKO49+WsQxXyCCWwjJbqDCUAFRoA&ubi=FZbfQhXE7JMCFai1/RQVmN/uUxUEFRwW3LvV2BYW5o+hisK96qxDNAIWqMCQgAglBhWrg5XABRW+BRUANquAwNrGwuTXogEA&clickid=0&cpx=__OFFSET_X__&cpy=__OFFSET_Y__&cs=__EVENT_TIME_START__&ce=__EVENT_TIME_END__&csign=ae758a17b72085a1&url=http%3A%2F%2Ftbhcb.top" data-pv="https://s1.milkjpg.com/s?type=1&r=20&tid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&finfo=DAABCAABAAAA1AgAAgAAAFkEAAM/Sgsn6kHG8wAIAAIAAAADCgADIazU9hYtoHMIAAQAAAELBgAGLbcIAAgAIFlACgAJAAAAAAACEBQGAAoAAAA&mv_ref=blog.csdn.net&enup=CAABb8xxxQgAAsVxzG8A&mvid=MTc3NzMxODkwMjM0MDEyMjgwNDAwMTg&bid=12b322b67646b827&ugi=FcTgigEVyNtrTBUCFSgVKBUAABWRxv+RAxaACBXIAQA&uai=FeTGlAIlAhUEFuaPjKO49+WsQxXyCCWwjJbqDCUAFRoA&ubi=FZbfQhXE7JMCFai1/RQVmN/uUxUEFRwW3LvV2BYW5o+hisK96qxDNAIWqMCQgAglBhWrg5XABRW+BRUANquAwNrGwuTXogEA&ds=1&price=AAAAAFvg89kAAAAAAARGdWsW1LFh2DoWa37I4Q==,https://max-l.mediav.com/rtb?type=2&d=100&b=12b322b67646b827&p=2265522&l=1136674&s=1&z=1&w=AAAAAFvg89kAAAAAAARGukpG3voQu1sBkYmz+w==&k=9Xar2AAAAAA=&i=C0AzIQjTYDcm&v=17773189023401228040018" data-clk="https://max-l.mediav.com/rtb?type=3&d=100&b=12b322b67646b827&p=2265522&l=1136674&s=1&k=9Xar2AAAAAA=&i=C0AzIQjTYDcm&v=17773189023401228040018&x=__OFFSET_X__&y=__OFFSET_Y__&st=__EVENT_TIME_START__&et=__EVENT_TIME_END__&tc=&turl=">
<info-div class="wrap">
<info-div class="singleImage-img singleImage-img-left">
<info-div class="img" style="background-image:url(https://s3m.mediav.com/galileo/546763-feb74f3ac898cc20c74d01e9560ce08d.jpg)"><info-div class="ads-tag"></info-div></info-div>
</info-div>
<info-div class="singleImage-body singleImage-body-left">
<info-div class="singleImage-title">老股民酒后无意说漏：20年炒股 坚持只看1指标</info-div>
<info-div class="singleImage-desc">东正金融 · 燨燚</info-div>
</info-div>
</info-div>
<info-div style="clear:both;"></info-div>
</info-div>
</info-div></newsfeed></div>
<script type="text/javascript" src="//static.mediav.com/js/mvf_news_feed.js"></script>
<script>
NEWS_FEED({
w: 900,
h : 84,
showid : 'Afihld',
placeholderId: "three_ad38",
inject : 'define',
define : {
imagePosition : 'left',
imageBorderRadius : 3,
imageWidth: 90,
imageHeight: 60,
imageFill : 'clip',
displayImage : true,
displayTitle : true,
titleFontSize: 16,
titleFontColor: '#000',
titleFontFamily : 'Microsoft Yahei',
titleFontWeight: 'bold',
titlePaddingTop : 0,
titlePaddingRight : 0,
titlePaddingBottom : 5,
titlePaddingLeft : 16,
displayDesc : true,
descFontSize: 12,
descFontColor: '#8e959a',
descFontFamily : 'Microsoft Yahei',
paddingTop : 10,
paddingRight : 0,
paddingBottom : 0,
paddingLeft : 0,
backgroundColor: '#fff',
hoverColor: '#000'
}
})
</script></div><script>$(function(){csdn.track.viewCheck($("#kp_box_87"));});</script></div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/qq_26803795/article/details/53820013,BlogCommendFromQuerySearch_39,index_38"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/qq_26803795/article/details/53820013,BlogCommendFromQuerySearch_39,index_38"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/qq_26803795/article/details/53820013" target="_blank" title="CDH集群中YARN的参数配置 - 静待花开(时光不语,静待花开。心平气和,淡泊流年。)">
<h4 class="text-truncate oneline" style="width: 860px;">
CDH集群中<em>YARN</em>的<em>参数</em>配置 - 静待花开(时光不语,静待花开。心平气和,淡泊流年。) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/1/9/E/3_qq_26803795.jpg" alt="qq_26803795" class="avatar-pic">
<span class="namebox">
<span class="name">qq_26803795</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">12-22</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
2446</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/qq_26803795/article/details/53820013" target="_blank" title="CDH集群中YARN的参数配置 - 静待花开(时光不语,静待花开。心平气和,淡泊流年。)">
<span class="desc oneline">前言：Hadoop 2.0之后，原先的MapReduce不在是简单的离线批处理MR任务的框架，升级为MapReduceV2（Yarn）版本，也就是把资源调度和任务分发两块分离开来。而在最新的CDH版本...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/qq_26803795">来自： <span class="blog_title"> 静待花开</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/sinat_38648491/article/details/79914438,BlogCommendFromQuerySearch_40,index_39"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/sinat_38648491/article/details/79914438,BlogCommendFromQuerySearch_40,index_39"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/sinat_38648491/article/details/79914438" target="_blank" title="弄清MapReduce和YARN这几点不同，才能真正掌握Hadoop的核心 - sinat_38648491的博客">
<h4 class="text-truncate oneline" style="width: 868px;">
弄清<em>MapReduce</em>和<em>YARN</em>这几点不同，才能真正掌握Hadoop的核心 - sinat_38648491的博客 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/2/7/B/3_sinat_38648491.jpg" alt="sinat_38648491" class="avatar-pic">
<span class="namebox">
<span class="name">sinat_38648491</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">04-12</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
197</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/sinat_38648491/article/details/79914438" target="_blank" title="弄清MapReduce和YARN这几点不同，才能真正掌握Hadoop的核心 - sinat_38648491的博客">
<span class="desc oneline">对于Hadoop1.0和Hadoop2.0的区别网上有很多资料，其中大部分是在对比MapReduce1.0和YARN，指出YARN的诞生很好的解决了MRv1存在的问题。但如果再引入MapReduce2...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/sinat_38648491">来自： <span class="blog_title"> sinat_38648491的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/zipo/article/details/54709488,BlogCommendFromQuerySearch_41,index_40"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/zipo/article/details/54709488,BlogCommendFromQuerySearch_41,index_40"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/zipo/article/details/54709488" target="_blank" title="二、关于HDFS、YARN及MapReduce的理解 - 夏光(大数据)">
<h4 class="text-truncate oneline" style="width: 860px;">
二、关于HDFS、<em>YARN</em>及<em>MapReduce</em>的<em>理解</em> - 夏光(大数据) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/8/2/C/3_zipo.jpg" alt="zipo" class="avatar-pic">
<span class="namebox">
<span class="name">zipo</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">01-24</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
2330</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/zipo/article/details/54709488" target="_blank" title="二、关于HDFS、YARN及MapReduce的理解 - 夏光(大数据)">
<span class="desc oneline">一、HDFS
HDFS是分布式文件系统，有高容错性的特点，可以部署在价格低廉的服务器上，主要包含namenode和datanode。
Namenode是hdfs中文件目录和文件分配管理者，它保存着文件...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/zipo">来自： <span class="blog_title"> 夏光</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/yu0_zhang0/article/details/80139315,BlogCommendFromQuerySearch_42,index_41"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/yu0_zhang0/article/details/80139315,BlogCommendFromQuerySearch_42,index_41"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/yu0_zhang0/article/details/80139315" target="_blank" title="Spark on yarn模式的参数设置即调优 - yu0_zhang0">
<h4 class="text-truncate oneline" style="width: 868px;">
Spark on <em>yarn</em>模式的<em>参数</em>设置即调优 - yu0_zhang0 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/6/B/C/3_yu0_zhang0.jpg" alt="yu0_zhang0" class="avatar-pic">
<span class="namebox">
<span class="name">yu0_zhang0</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">04-28</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
892</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/yu0_zhang0/article/details/80139315" target="_blank" title="Spark on yarn模式的参数设置即调优 - yu0_zhang0">
<span class="desc oneline">1 启动方式
执行命令./spark-shell --master yarn默认运行的是client模式。
执行./spark-shell --master yarn-client或者./...</span>
</a>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/hechenghai/article/details/49621577,BlogCommendFromQuerySearch_43,index_42"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/hechenghai/article/details/49621577,BlogCommendFromQuerySearch_43,index_42"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/hechenghai/article/details/49621577" target="_blank" title="hadoop2提交到Yarn： Mapreduce执行过程分析1 - hechenghai(现在已经不走技术路线了，可能没法回答大家的问题了，抱歉！)">
<h4 class="text-truncate oneline" style="width: 860px;">
hadoop2提交到<em>Yarn</em>： <em>Mapreduce</em>执行过程分析1 - hechenghai(现在已经不走技术路线了，可能没法回答大家的问题了，抱歉！) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/0/A/9/3_hechenghai.jpg" alt="hechenghai" class="avatar-pic">
<span class="namebox">
<span class="name">hechenghai</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">11-03</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
2011</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/hechenghai/article/details/49621577" target="_blank" title="hadoop2提交到Yarn： Mapreduce执行过程分析1 - hechenghai(现在已经不走技术路线了，可能没法回答大家的问题了，抱歉！)">
<span class="desc oneline">转载网址：http://www.aboutyun.com/forum.php?mod=viewthread&tid=9366&highlight=hadoop2%CC%E1%BD%BB%B5%BDYa...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/hechenghai">来自： <span class="blog_title"> hechenghai</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-ad-box"><div id="kp_box_558" data-pid="67" data-track-view="{"mod":"kp_popu_558","con":",,"}"><iframe style="width: 100%; height: 76px; border: 0px none;" scrolling="no"></iframe>
<div style="float:right;color:#ccc;font-size:10px;margin-top:-20px;">广告</div></div><script>$(function(){csdn.track.viewCheck($("#kp_box_558"));});</script></div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/wisgood/article/details/44001209,BlogCommendFromBaidu_44,index_43"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/wisgood/article/details/44001209,BlogCommendFromBaidu_44,index_43"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/wisgood/article/details/44001209" target="_blank" title="yarn内存参数 - wisgood的专栏">
<h4 class="text-truncate oneline" style="width: 860px;">
<em>yarn</em><em>内存</em><em>参数</em> - wisgood的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/9/6/B/3_wisgood.jpg" alt="wisgood" class="avatar-pic">
<span class="namebox">
<span class="name">wisgood</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">03-01</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
4093</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/wisgood/article/details/44001209" target="_blank" title="yarn内存参数 - wisgood的专栏">
<span class="desc oneline">yarn内存参数</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/wisgood">来自： <span class="blog_title"> wisgood的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/bluishglc/article/details/42436321,BlogCommendFromBaidu_45,index_44"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/bluishglc/article/details/42436321,BlogCommendFromBaidu_45,index_44"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/bluishglc/article/details/42436321" target="_blank" title="Hadoop/Yarn/MapReduce内存分配（配置）方案 - laurence的技术博客(// 风尘仆仆，终于归途)">
<h4 class="text-truncate oneline" style="width: 859px;">
Hadoop/<em>Yarn</em>/<em>MapReduce</em><em>内存</em>分配（配置）方案 - laurence的技术博客(// 风尘仆仆，终于归途) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/6/7/A/3_bluishglc.jpg" alt="bluishglc" class="avatar-pic">
<span class="namebox">
<span class="name">bluishglc</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">01-06</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
2.8万</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/bluishglc/article/details/42436321" target="_blank" title="Hadoop/Yarn/MapReduce内存分配（配置）方案 - laurence的技术博客(// 风尘仆仆，终于归途)">
<span class="desc oneline">以horntonworks给出推荐配置为蓝本，给出一种常见的Hadoop集群上各组件的内存分配方案。方案最右侧一栏是一个8G VM的分配方案，方案预留1-2G的内存给操作系统，分配4G给Yarn/Ma...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/bluishglc">来自： <span class="blog_title"> Laurence的技术博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/rzhzhz/article/details/7660376,BlogCommendFromBaidu_46,index_45"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/rzhzhz/article/details/7660376,BlogCommendFromBaidu_46,index_45"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/rzhzhz/article/details/7660376" target="_blank" title="Hadoop Map/Reduce内存限制 - 厚积而薄发(无数风雨的洗礼才能使你绽放……)">
<h4 class="text-truncate oneline" style="width: 860px;">
Hadoop Map/Reduce<em>内存</em>限制 - 厚积而薄发(无数风雨的洗礼才能使你绽放……) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/F/8/7/3_rzhzhz.jpg" alt="rzhzhz" class="avatar-pic">
<span class="namebox">
<span class="name">rzhzhz</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">06-13</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
6861</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/rzhzhz/article/details/7660376" target="_blank" title="Hadoop Map/Reduce内存限制 - 厚积而薄发(无数风雨的洗礼才能使你绽放……)">
<span class="desc oneline">如何设置hadoop Map/Reduce任务的内存限制？
Parameter
Type
Meaning
mapred.cluster....</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/rzhzhz">来自： <span class="blog_title"> 厚积而薄发</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/u014791046/article/details/51170584,BlogCommendFromBaidu_47,index_46"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/u014791046/article/details/51170584,BlogCommendFromBaidu_47,index_46"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u014791046/article/details/51170584" target="_blank" title="CDH集群调优：内存、Vcores和DRF - 金字塔上的蜗牛的专栏">
<h4 class="text-truncate oneline" style="width: 860px;">
CDH集群调优：<em>内存</em>、Vcores和DRF - 金字塔上的蜗牛的专栏 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/9/1/B/3_u014791046.jpg" alt="u014791046" class="avatar-pic">
<span class="namebox">
<span class="name">u014791046</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">04-16</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
3815</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/u014791046/article/details/51170584" target="_blank" title="CDH集群调优：内存、Vcores和DRF - 金字塔上的蜗牛的专栏">
<span class="desc oneline">原文地址：http://blog.selfup.cn/1631.html?utm_source=tuicool&utm_medium=referral
吐槽
最近“闲”来无事，通过...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/u014791046">来自： <span class="blog_title"> 金字塔上的蜗牛的专栏</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/qq_20641565/article/details/54729059,BlogCommendFromBaidu_48,index_47"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/qq_20641565/article/details/54729059,BlogCommendFromBaidu_48,index_47"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/qq_20641565/article/details/54729059" target="_blank" title="白话hadoop yarn的调度过程以mapreduce程序为例 - allen的博客(it's never too old to learn)">
<h4 class="text-truncate oneline" style="width: 860px;">
白话hadoop <em>yarn</em>的调度过程以<em>mapreduce</em>程序为例 - allen的博客(it's never too old to learn) </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/C/0/7/3_qq_20641565.jpg" alt="qq_20641565" class="avatar-pic">
<span class="namebox">
<span class="name">qq_20641565</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">01-25</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
2031</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/qq_20641565/article/details/54729059" target="_blank" title="白话hadoop yarn的调度过程以mapreduce程序为例 - allen的博客(it's never too old to learn)">
<span class="desc oneline">今天上完班，终于可以回去了，现在在公司没什么事干，就用白话来总结下mapreduce的执行流程吧，如果有错误的地方大家尽管指出。首先在hadoop 1.x中和hadoop 2.x中，mapreduce...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/qq_20641565">来自： <span class="blog_title"> allen的博客</span></a></span>
</p>
</div>
</div>
<div class="recommend-item-box recommend-ad-box"><div id="kp_box_625" data-pid="68" data-track-view="{"mod":"kp_popu_625","con":",,"}"><div id="_kt3ia080j7b"><iframe scrolling="no" src="//pos.baidu.com/s?hei=60&wid=852&di=u3565460&ltu=https%3A%2F%2Fblog.csdn.net%2Fz3935212%2Farticle%2Fdetails%2F78637157&tpr=1541469145114&chi=2&ps=7022x402&col=zh-CN&cja=false&cdo=-1&ari=2&ant=0&cfv=0&cpl=1&drs=1&pss=2345x7153&cmi=2&dis=0&dc=3&cce=true&par=1366x728&exps=111000&psr=1366x768&cec=UTF-8&dri=0&pcs=1349x615&prot=2&dai=4&tlm=1541469149&tcn=1541469149&ti=Yarn%E4%B8%8BMapreduce%E7%9A%84%E5%86%85%E5%AD%98%E5%8F%82%E6%95%B0%E7%90%86%E8%A7%A3%26xml%E5%8F%82%E6%95%B0%E9%85%8D%E7%BD%AE%20-%20z3935212%E7%9A%84%E4%B8%93%E6%A0%8F%20-%20CSDN%E5%8D%9A%E5%AE%A2&pis=-1x-1&ccd=24&dtm=HTML_POST" width="852" height="60" frameborder="0"></iframe></div><script type="text/javascript" src="//rabc1.iteye.com/common/resource/site/workyo.js?hcuzbz=ybi"></script></div><script>$(function(){csdn.track.viewCheck($("#kp_box_625"));});</script></div>
<div class="recommend-item-box recommend-box-ident type_blog clearfix" data-track-view="{"mod":"popu_387","con":",https://blog.csdn.net/xiaoshunzi111/article/details/52173643,BlogCommendFromBaidu_49,index_48"}" data-track-click="{"mod":"popu_387","con":",https://blog.csdn.net/xiaoshunzi111/article/details/52173643,BlogCommendFromBaidu_49,index_48"}" data-flg="true">
<div class="content" style="width: 952px;">
<a href="https://blog.csdn.net/xiaoshunzi111/article/details/52173643" target="_blank" title="hadoop的mapreduce作业中经常出现Java heap space解决方案 - 似水流年">
<h4 class="text-truncate oneline" style="width: 860px;">
hadoop的<em>mapreduce</em>作业中经常出现Java heap space解决方案 - 似水流年 </h4>
<div class="info-box d-flex align-content-center">
<!-- <p class="avatar">
<img src="https://avatar.csdn.net/D/4/2/3_xiaoshunzi111.jpg" alt="xiaoshunzi111" class="avatar-pic">
<span class="namebox">
<span class="name">xiaoshunzi111</span>
<span class="triangle"></span>
</span>
</p> -->
<p class="date-and-readNum">
<span class="date hover-show">08-10</span>
<span class="read-num hover-hide">
<svg class="icon csdnc-yuedushu" aria-hidden="true">
<use xlink:href="#csdnc-m-passwords-visible"></use>
</svg>
4948</span>
</p>
</div>
</a>
<p class="content" style="width: 952px;">
<a href="https://blog.csdn.net/xiaoshunzi111/article/details/52173643" target="_blank" title="hadoop的mapreduce作业中经常出现Java heap space解决方案 - 似水流年">
<span class="desc oneline">我们经常遇到这样的问题，内存溢出，内存溢出的原因是很简单，不够用了，不够用该怎么设置，通过水设置。可以参考下面案例
一、hadoop的mapreduce作业中经常出现Java heap space解...</span>
</a>
<span class="blog_title_box oneline"><a target="_blank" href="https://blog.csdn.net/xiaoshunzi111">来自： <span class="blog_title"> 似水流年</span></a></span>
</p>
</div>
</div>
<div class="recommend-loading-box">
<img src="https://csdnimg.cn/release/phoenix/images/feedLoading.gif">
</div>
<div class="recommend-end-box">
<p class="text-center">没有更多推荐了，<a href="https://blog.csdn.net/" class="c-blue c-blue-hover c-blue-focus">返回首页</a></p>
</div>
</div>
上一篇： 5.带头结点的双向循环链表