r - 有没有办法限制运行的R进程的数量

标签 r ulimit domc setrlimit

我使用 doMC,它使用 multicore 包。当我(在控制台中)调试时,它发生了(好几次)它横向移动并且 fork-bombed .

R 有 setrlimit() 系统调用吗? 在 pyhton 中,我会使用 resource.RLIMIT_NPROC

理想情况下,我想将运行的 R 进程数限制为一个数字

编辑:操作系统是 linux CentOS 6


应该有几个选择。这是来自 Writing R Extensions, Section 的相关部分

   Packages are not standard-alone programs, and an R process could
contain more than one OpenMP-enabled package as well as other components
(for example, an optimized BLAS) making use of OpenMP. So careful
consideration needs to be given to resource usage.  OpenMP works with
parallel regions, and for most implementations the default is to use as
many threads as 'CPUs' for such regions.  Parallel regions can be
nested, although it is common to use only a single thread below the
first level.  The correctness of the detected number of 'CPUs' and the
assumption that the R process is entitled to use them all are both
dubious assumptions.  The best way to limit resources is to limit the
overall number of threads available to OpenMP in the R process: this can
be done via environment variable 'OMP_THREAD_LIMIT', where
implemented.(4)  Alternatively, the number of threads per region can be
limited by the environment variable 'OMP_NUM_THREADS' or API call
'omp_set_num_threads', or, better, for the regions in your code as part
of their specification.  E.g. R uses
     #pragma omp parallel for num_threads(nthreads) ...
That way you only control your own code and not that of other OpenMP

我最喜欢的工具之一是控制它的包:RhpcBLASctl .这是它的描述:

Control the number of threads on 'BLAS' (Aka 'GotoBLAS', 'ACML' and 'MKL'). and possible to control the number of threads in 'OpenMP'. get a number of logical cores and physical cores if feasible.

毕竟您需要控制并行 session 的数量以及分配给每个并行线程的 BLAS 核心数量。并行包默认每个 session 2 个线程是有原因的...

所有这些应该在很大程度上独立于您正在运行的 Linux 或 Unix 的风格。好吧,除了 OS X 当然(仍然 !!)没有为您提供 OpenMP 之外。

你可以从 doMC 和 friend 那里控制最外层。

关于r - 有没有办法限制运行的R进程的数量,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/40135812/


R: 使用 SMOTE 包 "invalid ' 标签时遇到问题'"

r - 无法弄清楚如何使用 if 语句格式化 for 循环

amazon-ec2 - 如何在 Amazon EC2 实例上增加 ulimit?

python - 从 shell 设置 `ulimit -c`

MongoDB:软 rlimits 太低。 rlimits 设置为 27774 个进程,64000 个文件。进程数应至少为 32000:文件数的 0.5 倍

r - 在 R 中使用多核和 doMC 时如何标记我的子进程以进行日志记录

R 多核 mcfork() : Unable to fork: Cannot allocate memory

R read.csv 如何忽略回车?

r - bigmemory 和带有文件支持的 friend 的示例

r - 多次迭代后出现 %dopar% 错误