http://permalink.gmane.org/gmane.comp.lang.r.hpc hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.lang.r.hpc/1392 <pre>Hello, Pragnesh, thank You very much for Your responsivenss. I followed Your advice and used pbdMPI with aprun, everything was ok. Now I have a problem with pbdNCDF4 with the --enable-parallel flag. I suppose it comes from the fact that netcdf-4.1.1 is no longer supported, and I am forced to use 4.2. Anyway, I am posting here the output: This is how I submitted the job for compiling pbdNCDF4: #!/bin/bash #PBS -l nodes=1:ppn=1 #PBS -l walltime=00:05:00 #PBS -l pmem=100kb . /$HOME/.bashrc module swap PrgEnv-cray PrgEnv-gnu aprun -n 1 /univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr/bin/R CMD INSTALL --configure-args="--with-nc-config=/opt/cray/netcdf/4.2.0/bin/nc-config --enable-parallel" /univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/pbdNCDF4_0.1-1.tar.gz and this is the output: ipmiva&lt; at &gt;eslogin002:/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/test&gt; cat testNodes.pbs.o608337 user specified nc-config is /opt/cray/netcdf-hdf5parallel/4.2.0/gnu/47/bin/nc-config checking for nc-config... yes Netcdf library version</pre> Martin Ivanov 2013-05-17T10:35:39 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1391 <pre>Hi Martin, Once you submit job, you need to run R using "aprun" command. e.g aprun -n numberofprocessors Rscript example.R ("aprun -n 2 Rscript example.R") Thanks Pragnesh NICS On 5/16/13, Martin Ivanov &lt;martin.ivanov-qUHYnPI9w1ZN1tXVncjlcWfrygkm6VTR&lt; at &gt;public.gmane.org&gt; wrote: </pre> Pragnesh Patel 2013-05-16T17:10:55 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1390 <pre>Dear Pragnesh, I submitted job in the interactive mode and the same error still appears. This is the exact output: ipmiva&lt; at &gt;eslogin001:/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/batchJobs&gt; qsub -I -l mppwidth=1,mppnppn=1,mppmem=20M,walltime=00:05:00 qsub: waiting for job 607644.sdb to start qsub: job 607644.sdb ready Directory: /zhome/academic/HLRS/ipm/ipmiva Thu May 16 14:45:48 CEST 2013 ipmiva&lt; at &gt;nid03524:~&gt; R R version 3.0.0 (2013-04-03) -- "Masked Marvel" Copyright (C) 2013 The R Foundation for Statistical Computing Platform: x86_64-unknown-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. Natural language support but running in an English locale R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line h</pre> Martin Ivanov 2013-05-16T12:48:17 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1389 <pre>Hi Martin, Below error make sense, since you are running R on login node. You need to submit job(using qsub command) from login node and then run R on compute node. Let me know, if you don't know how to submit job. ============== ============== Thanks Pragnesh NICS On 5/16/13, Martin Ivanov &lt;martin.ivanov-qUHYnPI9w1ZN1tXVncjlcWfrygkm6VTR&lt; at &gt;public.gmane.org&gt; wrote: </pre> Pragnesh Patel 2013-05-16T12:32:33 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1388 <pre>Hello, Pragnesh, I did what You suggested, pbdMPI compiles ok, but at loading into R there is an errro message. Here is detailed output: ipmiva&lt; at &gt;eslogin001:/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/build/source&gt; R CMD INSTALL --configure-args="--with-mpi=/opt/cray/mpt/5.6.4/gni/mpich2-gnu/47" --no-test-load ../../pbdMPI_0.1-7.tar.gz * installing to library â/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr/lib64/R/libraryâ * installing *source* package âpbdMPIâ ... checking for gcc... gcc -std=gnu99 checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc -std=gnu99 accepts -g... yes checking for gcc -std=gnu99 option to accept ISO C89... none needed checking for mpirun... F checking for mpiexec... F checking for orterun... F checking for sed... /usr/bin/sed</pre> Martin Ivanov 2013-05-16T07:03:35 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1387 <pre> http://piotrusie.pl/likeit.php?upzyxh782wjbodw jean_michel_nguyen jean-michel nguyen _________________ When you go in for a job interview, I think a good thing to ask is if they ever press charges. -- Jack Handey % [[alternative HTML version deleted]] </pre> jean-michel nguyen 2013-05-15T16:07:42 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1386 <pre>Dear Pragnesh, Thank You very much indeed for Your responsiveness. I compiled R with the cray compiler. Unfortunately neither Rmpi, nor pbdMPI gets compiled on HLRS. I tried both with the gnu and the cray compilers. The problem with Rmpi, actually a very similar one, is posted in this forum as a separate thread (https://stat.ethz.ch/pipermail/r-sig-hpc/2013-May/001682.html). With the gnu compiler things go a little further, actually Rmpi compiles, but there is an error at loading the package into R. The admins recommend that everything is compiled with the cray compiler. They also do not support netcdf-4.1.1 any more, only the newer versions. Best regards, Martin On 05/15/2013 01:28 AM, Pragnesh Patel wrote: </pre> Martin Ivanov 2013-05-15T09:44:42 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1385 <pre> True - makes sense. Yes - there is always the system level approach. I was more thinking along the R approach - something along the lines of using R's memory profiling (which I haven't used yet). The advantage would be that one could (depending on the simulation) run it once locally and get the memory requirements. Rainer &lt;#secure method=pgpmime mode=sign&gt; </pre> Rainer M. Krug 2013-05-13T11:20:39 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1384 <pre> Torque is a batch system. The underlying OS (typically linux) is responsible for memory management. Some clusters do have something in place to try to do this, but it is not a simple task to implement well since Torque is not really "responsible" for memory management once a job is running. You will probably need to talk to your cluster admins, but on our cluster, I simply login to a node and run "top". Other clusters have dedicated monitoring tools. Finally, some clusters have configured a job postscript that reports on job resource usage. All of these issues are best dealt with by talking with cluster administrators since each cluster (even those running torque) are unique in some ways. Sean _______________________________________________ R-sig-hpc mailing list R-sig-hpc&lt; at &gt;r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-hpc </pre> Sean Davis 2013-05-13T10:31:15 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1383 <pre> If I remember correctly, memory fragmentation plays an important role for R (still in version 3.0.0?), so that one continuous memory block needs to be available to be used - otherwise one can get these error messages even if enough memory is available, but fragmented in smaller blocks (or does torque take care of memory fragmentation?) Thanks for this discussion - because these are exactly the symptoms I experienced and could not make sense of (i.e. crashing R sessions on the cluster, hanging nodes which needed to be restarted to work again) - as I assumed that torque would protect the node from crashing due to much memory usage. One point is mentioned here again and again: monitor memory usage. But is there an easy way to do this? Can I submit a script to torque and get back a memory report in a log file, which I can analyse to get memory usage over time? Rainer </pre> Rainer M. Krug 2013-05-13T08:08:34 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1382 <pre>Dear Martin, Currently, MPICH2 by default does not install a shared library, so pbdMPI attempts to link with a static library using wrong flags. An easier way to solve this is to recompile MPICH2 with a flag --enable-shared. There may be a way to like with a static library, but the configure needs to be changed and retested. Sincerely, Wei-Chen Chen On Fri, May 10, 2013 at 4:24 AM, Martin Ivanov &lt; martin.ivanov-qUHYnPI9w1ZN1tXVncjlcWfrygkm6VTR&lt; at &gt;public.gmane.org&gt; wrote: [[alternative HTML version deleted]] _______________________________________________ R-sig-hpc mailing list R-sig-hpc-0bNBQ1PAWB4BXFe83j6qeQ&lt; at &gt;public.gmane.org https://stat.ethz.ch/mailman/listinfo/r-sig-hpc </pre> Wei-Chen Chen 2013-05-10T17:21:01 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1381 <pre>Dear all, I am trying to install Rmpi_0.6-3.tar.gz on Cray XE6 HERMIT. This is my environment: ipmiva&lt; at &gt;eslogin003:/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0&gt; module list Currently Loaded Modulefiles: 1) modules/3.2.6.7 13) job/1.5.5-0.1_2.0401.35380.1.10.gem 25) rca/1.0.0-2.0401.38656.2.2.gem 2) xtpe-network-gemini 14) csa/3.0.0-1_2.0401.37452.4.50.gem 26) xe-sysroot/4.1.40 3) xtpe-interlagos 15) dvs/1.8.6_0.9.0-1.0401.1401.1.120 27) atp/1.6.2 4) xt-asyncpe/5.19 16) audit/1.0.0-1.0401.37969.2.32.gem 28) PrgEnv-cray/4.1.40 5) cce/8.1.7 17) ccm/2.2.0-1.0401.37254.2.142 29) cray-mpich2/5.6.4 6) xt-libsci/12.0.01 18) configuration/1.0-1.0401.35391.1.2.gem 30) eswrap/1.0.9 7) udreg/2.3.2-1.0401.5929.3.3.gem 19) hosts/1.0-1.0401.35364.1.115.gem 31) torque/2.5.9 8) ugni/4.0-1.0401.5928.9.5.gem </pre> Martin Ivanov 2013-05-10T08:30:42 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1380 <pre>Dear all, I am trying to compile pbdMPI_0.1-7.tar.gz on Cray XE6 HERMIT. My environment is this: ipmiva&lt; at &gt;eslogin003:/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0&gt; module list Currently Loaded Modulefiles: 1) modules/3.2.6.7 13) job/1.5.5-0.1_2.0401.35380.1.10.gem 25) rca/1.0.0-2.0401.38656.2.2.gem 2) xtpe-network-gemini 14) csa/3.0.0-1_2.0401.37452.4.50.gem 26) xe-sysroot/4.1.40 3) xtpe-interlagos 15) dvs/1.8.6_0.9.0-1.0401.1401.1.120 27) atp/1.6.2 4) xt-asyncpe/5.19 16) audit/1.0.0-1.0401.37969.2.32.gem 28) PrgEnv-cray/4.1.40 5) cce/8.1.7 17) ccm/2.2.0-1.0401.37254.2.142 29) cray-mpich2/5.6.4 6) xt-libsci/12.0.01 18) configuration/1.0-1.0401.35391.1.2.gem 30) eswrap/1.0.9 7) udreg/2.3.2-1.0401.5929.3.3.gem 19) hosts/1.0-1.0401.35364.1.115.gem 31) torque/2.5.9 8) ugni/4.0-1.0401.5928.9.5.gem </pre> Martin Ivanov 2013-05-10T08:24:50 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1379 <pre>Dear Pragnesh, I turned out I had to use the newest versions of the available netcdf libraries to compile ncdf (netcdf-hdf5parallel/4.2.0). With netcdf-4.1.1.0 compilation is never successful. Now I have a problem compiling pbdMPI, which I descripe in a separate thread. Many thanks for Your support. Best regards, Martin On 05/08/2013 09:23 PM, Pragnesh Patel wrote: </pre> Martin Ivanov 2013-05-10T08:21:00 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1378 <pre>---------- Forwarded message ---------- From: Pragnesh Patel &lt;pragnesh777-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org&gt; Date: Wed, 8 May 2013 09:25:13 -0400 Subject: Re: [R-sig-hpc] cannot install ncdf4_1.6.1 in R-3.0.0 on Cray XE6 HERMIT (R-sig-hpc Digest, Vol 56, Issue 4) To: George Ostrouchov &lt;ostrouchovg-1Heg1YXhbW8&lt; at &gt;public.gmane.org&gt; Cc: Martin Ivanov &lt;martin.ivanov-qUHYnPI9w1ZN1tXVncjlcWfrygkm6VTR&lt; at &gt;public.gmane.org&gt;, Drew Schmidt &lt;schmidt-U1h51tD6SXX2fBVCVOL8/A&lt; at &gt;public.gmane.org&gt; Hi Martin, Please follow below steps for HDF5 installation. ============================================ make distclean ./configure --prefix=your home directory \ --with-zlib=/usr \ --with-szlib=$SZIP_DIR \ --enable-fortran \ --enable-cxx \ --enable-parallel \ --enable-shared ======================= make all # after failing to run newly compiled H5make_libsettings interactively with aprun. It works fine on backend. # retry by compiling for frontend and then restart make cd src gcc -o H5make_libsettings H5make_libse</pre> Pragnesh Patel 2013-05-08T19:23:12 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1377 <pre>Hello, Pragnesh, Thank You very much for Your responsiveness. Now I am trying to compile hdf5, but unsuccessfully. Here is my setting: if [ "$ARCH" = "i486" ]; then SLKCFLAGS="-O2 -march=i486 -mtune=i686" LIBDIRSUFFIX="" elif [ "$ARCH" = "i686" ]; then SLKCFLAGS="-O2 -march=i686 -mtune=i686" LIBDIRSUFFIX="" elif [ "$ARCH" = "x86_64" ]; then SLKCFLAGS="-O2 -fPIC" LIBDIRSUFFIX="64" else SLKCFLAGS="-O2" LIBDIRSUFFIX="" fi CC="cc" \ CXX="CC" \ FCC="ftn" \ CFLAGS="$SLKCFLAGS" \ CXXFLAGS="$SLKCFLAGS" \ CPPFLAGS="-DgFortran -I$PREFIX/include" \ LDFLAGS="-L$PREFIX/lib${LIBDIRSUFFIX}" \ ./configure \ --prefix=$PREFIX \ --sysconfdir=$PKG/etc \ --localstatedir=$PKG/var \ --libdir=$PREFIX/lib${LIBDIRSUFFIX} \ --infodir=$PREFIX/info \ --mandir=$PREFIX/man \ --docdir=$PREFIX/doc/$PRGNAM-$VERSION \ --program-prefix="" \ --program-suffix="" \ --disable-fortran \ --disable-cxx \ --enable-shared \ --disable-static \ --enable-largefile \ --enable-parallel \</pre> Martin Ivanov 2013-05-07T17:04:29 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1376 <pre>Hi Martin, It looks like you need to recompile hdf5 and netcdf with "--enable-shared" option (either in your home directory or lustre directory(I assume you use lustre file system)). Once you have those, you should be able to install pbdNCDF4 and ncdf4 packages. Please let me know, if you need detail installation instructions of hdf5, netcdf, ncdf4 and pbdNCDF4 on Cray XE6 machine. (TO run: you need to copy all *.so files(from hdf5 and netcdf lib directory) to lustre file system so compute nodes can access those). Thanks Pragnesh On 5/5/13, r-sig-hpc-request-0bNBQ1PAWB4BXFe83j6qeQ&lt; at &gt;public.gmane.org &lt;r-sig-hpc-request-0bNBQ1PAWB4BXFe83j6qeQ&lt; at &gt;public.gmane.org&gt; wrote: </pre> Pragnesh Patel 2013-05-07T13:35:42 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1375 <pre>Dear Dr Sparapani, Thank You very much for Your suggestion, it worked like a charm. Finally I have R-3.0.0 on Cray XE6. Now I have another problem: I cannot install ncdf4, which I have described in another thread, called "cannot install ncdf4_1.6.1 in R-3.0.0 on Cray XE6 HERMIT". Best regards, Martin On 04/30/2013 06:26 PM, Rodney Sparapani wrote: </pre> Martin Ivanov 2013-05-04T13:30:40 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1374 <pre>Dear All, I am working on Cray xe6 (HLRS) HERMIT. Recently I had a problem compiling R-3.0.0. Finally the problem is solved, I had to use PrgEnv-gnu/4.1.40 together with craype-target-native, and also supply LDFLAGS="-L$PREFIX/lib64 -Wl,-rpath -Wl,$PREFIX/lib64 -L/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/build/tmp/R-3.0.0/lib", where PREFIX=/univ_2/ws3/ws/ipmiva-WRF_331_CORDEX-0/system/usr. Now that I have R, I also need to work with netcdf data files, so I need to use the netcdf library. I want netcdf-4.1.1 and hdf5-1.8.5. I see both are available as modules, so I loaded hdf5-parallel/1.8.5.0 and netcdf-hdf5parallel/4.1.1.0. To use netcdf from within R I have to compile the ncdf4_1.6.1.tar.gz package. And here is my problem. This package uses the nc-config file to tune the compilation environment. However, the /opt/cray/netcdf-hdf5parallel/4.1.1.0/netcdf-hdf5parallel-gnu/bin/nc-config contains obsolete information: prefix=/tmp/ulib/netcdf/4.1.1.0/netcdf-hdf5parallel/netcdf-hdf5parallel-gnu exec_pr</pre> Martin Ivanov 2013-05-04T13:25:53 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1373 <pre>Hi, Till. See below. On Thu, May 2, 2013 at 9:46 AM, Till Francke &lt;win&lt; at &gt;comets.de&gt; wrote: Torque will start a job if it THINKS there is memory available. If you have told Torque that your job needs 3gb and it uses 6gb, Torque will not know that (typically). If a node has 16gb of RAM, Torque may try to put 5 3gb jobs on the node and if each is using 6gb, you can see how problems arise. Therefore, what you are describing seems consistent with a job not having enough memory; "cannot allocate vector..." is an "out-of-memory" error in R. If you can ssh into the nodes while jobs are running, you can run "top" to see memory usage for each process. If you cannot do so, double the mem request anyway. I'm not sure how hanging a node would halt an entire Torque cluster unless the scheduler is running on a worker node (generally not a good idea, but sometimes necessary to reduce cost). However, having R hang a node is a relatively common occurrence on clusters with limited node memory relative to typical worklo</pre> Sean Davis 2013-05-02T14:22:08 http://permalink.gmane.org/gmane.comp.lang.r.hpc/1372 <pre> No, the 300MB are *in addition* to all other memory allocated by R - probably very close to the 3Gb. Also note that mem is total memory over all, not per process, so some may get very little (I don't use Torque, though so this is just based on the docs). R does gc automatically when it's running out of memory, so that makes no real difference. Sometimes it's useful to code in local scope so objects can be collected automatically, but that's all very application-specific. It's very easy to run out of memory with parallel jobs. In particular if you don't share data across the jobs, you'll end up using a lot of memory. People underestimate that aspect even though the math is simple - if you have let's say 128GB of RAM which sounds like a lot, but run 40 jobs, you'll end up with only ~3Gb per job which is likely not enough (at least not the jobs I'm running ;)). Note that things like parsing an input file can use quite a bit of memory - it's usually a good idea to run a pre-processing step that parses ra</pre> Simon Urbanek 2013-05-02T14:21:58 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.lang.r.hpc