我打算做的是限制可以将任务发送到testmonitor
队列的用户或组。
在按如下所示设置AclSubmitApps
之后,我仍然可以在用户testmonitor
中提交配置单元任务,该任务的队列通过set mapred.job.queue.name=root.testmonitor;
设置为monitor:monitor
。我可以确保我的fair-scheduler.xml是由yarn重新加载的(通过更改maxRunningApps
并可以在hadoop监视器网页中看到更改)。
我看不到fair-scheduler.xml
与official document给出的示例之间的本质区别。谁能给我一些想法?非常感谢!
我的fair-scheduler.xml
<allocations>
<queue name="default">
<minResources>100000 mb,64vcores</minResources>
<maxResources>12000000 mb,720vcores</maxResources>
<maxRunningApps>50</maxRunningApps>
<weight>1.0</weight>
<schedulingPolicy>fifo</schedulingPolicy>
<minSharePreemptionTimeout>300</minSharePreemptionTimeout>
</queue>
<queue name="testmonitor">
<minResources>12000000 mb, 600vcores</minResources>
<maxResources>20000000 mb, 720vcores</maxResources>
<maxRunningApps>5</maxRunningApps>
<weight>2.0</weight>
<schedulingPolicy>fair</schedulingPolicy>
<minSharePreemptionTimeout>300</minSharePreemptionTimeout>
<aclSubmitApps>testmonitor</aclSubmitApps>
</queue>
<userMaxAppsDefault>50</userMaxAppsDefault>
<fairSharePreemptionTimeout>6000</fairSharePreemptionTimeout>
<defaultQueueSchedulingPolicy>fifo</defaultQueueSchedulingPolicy>
</allocations>
监视正在运行的任务:
关于公平调度程序的官方文件示例:
<?xml version="1.0"?>
<allocations>
<queue name="sample_queue">
<minResources>10000 mb,0vcores</minResources>
<maxResources>90000 mb,0vcores</maxResources>
<maxRunningApps>50</maxRunningApps>
<weight>2.0</weight>
<schedulingPolicy>fair</schedulingPolicy>
<queue name="sample_sub_queue">
<aclSubmitApps>charlie</aclSubmitApps>
<minResources>5000 mb,0vcores</minResources>
</queue>
</queue>
<user name="sample_user">
<maxRunningApps>30</maxRunningApps>
</user>
<userMaxAppsDefault>5</userMaxAppsDefault>
<queuePlacementPolicy>
<rule name="specified" />
<rule name="primaryGroup" create="false" />
<rule name="default" />
</queuePlacementPolicy>
</allocations>
最佳答案
将根队列添加为父队列。
<queue name="root">
<aclSubmitApps> </aclSubmitApps>
<queue name="queue1">
<minResources>10000mb,10vcores</minResources>
<maxResources>30000mb,30vcores</maxResources>
<maxRunningApps>10</maxRunningApps>
<weight>2.0</weight>
<schedulingMode>fair</schedulingMode>
<aclAdministerApps>User1</aclAdministerApps>
<aclSubmitApps>User1</aclSubmitApps>
</queue>
</queue>
关于hadoop - Hadoop纱:fair-scheduler.xml中的AclSubmitApps无效吗?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/26462082/