gmane.comp.lang.r.devel

Re: Assigning NULL to large variables is much faster than rm() - any reason why I should still use rm()?

Henrik Bengtsson — 2013-05-26T00:10:59

On Sat, May 25, 2013 at 4:38 PM, Simon Urbanek
<simon.urbanek< at >r-project.org> wrote:

Thanks for this one.  This is useful - I did try to follow where
.Internal(remove, ...), but got lost in the internal structures.

Of course, I'd love to see such a function in 'base' itself.  Having
such a well defined and narrow function for removing a variable in the
current environment may also be useful for 'codetools'/'R CMD check'
such that code inspection can detect undefined variables in the case
they used to be defined but later have been removed.  Technically rm()
allows for that too, but I can see how such a task quickly gets
complicated when arguments 'list', 'envir' and 'inherits' are
involved.


I didn't mention it, but another reason I use rm() a lot is actually
so R can catch my programming mistakes (I'm maintaining 100,000+ lines
of code), i.e. the opposite to being error prone.  For instance, by
doing rm(tmp) as soon as possible, R will give me the run-time error
"Error: object 'tmp' not found" in case I u

Re: Assigning NULL to large variables is much faster than rm()- any reason why I should still use rm()?

Simon Urbanek — 2013-05-25T23:38:40


Yes, as you probably noticed rm() is a quite complex function because it has to deal with different ways to specify input etc. 
When you remove that overhead (by calling .Internal(remove("a", parent.frame(), FALSE))), you get the same performance as the assignment.
If you really want to go overboard, you can define your own function:

SEXP rm(SEXP x, SEXP rho) { setVar(x, R_UnboundValue, rho); return R_NilValue; }
poof <- function(x) .Call(rm_C, substitute(x), parent.frame())

That will be faster than anything else (mainly because it avoids the trip through strings as it can use the symbol directly).

But as Bill noted - it practice I'd recommend using either local() or functions to control the scope - using rm() or assignments seems too error-prone to me.

Cheers,
Simon

R in the browser ...

Jony Hudson — 2013-05-25T21:58:28

Hi all,

 I hope you'll forgive me - I don't plan to start using this list as my blog - but given the discussion following my last post I thought people on here might be interested to see some progress. This is a minimal build of R, cross-compiled from C/Fortran to javascript with emscripten - to be clear, nothing is running server-side, this is all running in the browser's JS engine. The user experience is rather lacking at the minute, much is missing (see below), and there are no compiler optimisations applied (also below) but still, it kind of works. Have a play here:

http://r-in-the-browser.herokuapp.com/

(WARNING: 20MB HTML file, 3.5MB gzipped  + 7MB data file). It goes without saying that you'll want to use a modern web browser to look at it! It works in the latest chrome, firefox, and safari (although I can't see the session output until after I q() in my safari :-() It also seems to run on my iPad, after a very long start-up wait, but it shares the problem of desktop safari that you have to work bl

Re: Assigning NULL to large variables is much faster than rm() - any reason why I should still use rm()?

William Dunlap — 2013-05-25T21:00:16

Another way to avoid using rm() in loops is to use throw-away
functions.  E.g., 
+   a <- x[,k]
+   colSum <- sum(a)
+   a <- NULL # Not needed anymore
+   b <- x[k,]
+   rowSum <- sum(b)
+   b <- NULL # Not needed anymore
+ })
+     colKSum <- function(k) { a <- x[,k] ; sum(a) }
+     rowKSum <- function(k) { b <- x[k,] ; sum(b) }
+     for(k in 1:ncol(x)) {
+         colSum <- colKSum(k)
+         rowSum <- rowKSum(k)
+ }})
   user  system elapsed 
   7.89    0.02    7.93 
   user  system elapsed 
   7.88    0.02    7.93
I think the code is clearer.  It might make the compiler's job easier.

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com

Assigning NULL to large variables is much faster than rm() - any reason why I should still use rm()?

Henrik Bengtsson — 2013-05-25T19:48:33

Hi,

in my packages/functions/code I tend to remove large temporary
variables as soon as possible, e.g. large intermediate vectors used in
iterations.  I sometimes also have the habit of doing this to make it
explicit in the source code when a temporary object is no longer
needed.  However, I did notice that this can add a noticeable overhead
when the rest of the iteration step does not take that much time.

Trying to speed this up, I first noticed that rm(list="a") is much
faster than rm(a).  While at it, I realized that for the purpose of
keeping the memory footprint small, I can equally well reassign the
variable the value of a small object (e.g. a <- NULL), which is
significantly faster than using rm().

SOME BENCHMARKS:
A toy example imitating an iterative algorithm with "large" temporary objects.

x <- matrix(rnorm(100e6), ncol=10e3)

t1 <- system.time(for (k in 1:ncol(x)) {
  a <- x[,k]
  colSum <- sum(a)
  rm(a) # Not needed anymore
  b <- x[k,]
  rowSum <- sum(b)
  rm(b) # Not needed anymore
})

t2

segfault when using browser() in Rprofile.site

Benjamin Tyner — 2013-05-25T00:41:23

Hi.

It seems that if I put a browser() in my Rprofile.site, I get a
segfault. This happens on several machines, several versions of R.

Here it the valgrind output when using revision 62797:

==31314== Memcheck, a memory error detector
==31314== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==31314== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==31314== Command: /home/btyner/R62797/lib/R/bin/exec/R
==31314==

R version 3.0.1 Patched (2013-05-24 r62797) -- "Good Sport"
Copyright (C) 2013 The R Foundation for Statistical Computing
Platform: x86_64-unknown-linux-gnu (64-bit)

<snip>

Called from: top level
Browse[1]> ls()
==31314== Invalid write of size 1
==31314==    at 0x4CF07D: R_IoBufferPutc (iosupport.c:135)
==31314==    by 0x4D3C5C: Rf_ReplIteration (main.c:222)
==31314==    by 0x4D4047: R_ReplConsole (main.c:307)
==31314==    by 0x4D4353: do_browser (main.c:1137)
==31314==    by 0x4B1E86: Rf_eval (eval.c:639)
==31314==    by 0x4D22F0: R_ReplFile (main.c:101)
==313

Re: Problem with Rboolean in c++ code

Jan van der Laan — 2013-05-24T19:23:50


On 05/24/2013 05:02 PM, Prof Brian Ripley wrote:


Sorry. You're right. Use Rinternals.h; don't use Rdefines.h.

Jan

Re: Problem with Rboolean in c++ code

Prof Brian Ripley — 2013-05-24T15:02:33

That's backwards.  Rinternals.h is the definitive version, used by R itself.

Re: Problem with Rboolean in c++ code

Jan van der Laan — 2013-05-24T14:55:18

Thanks. That does the trick. Although I now have to rewrite some other  
stuff, but I have just started so better now than later.

To be honest, a first and even a second read of the r-extensions  
manual did not really make it clear, to me at least, that Rdefines.h  
is preferred above Rinternals.h. Now that I reread it again, I can see  
a slight preference for Rdefines.h, but you really have to read closely.

Thanks again.

Jan


Prof Brian Ripley <ripley< at >stats.ox.ac.uk> schreef:

Re: Problem with Rboolean in c++ code

Prof Brian Ripley — 2013-05-24T13:36:19

Don't use Rdefines.h: use Rinternals.h.

Rdefines.h was for compatibility with S code from the 1990s: it is not 
kept up to date.

Re: Standalone example to use eval in C (not eval in R)?

Simon Urbanek — 2013-05-24T12:59:53

On May 24, 2013, at 6:00 AM, Peng Yu wrote:


Try R-exts:
"5.11 Evaluating R expressions from C"

Cheers,
S

Standalone example to use eval in C (not eval in R)?

Peng Yu — 2013-05-24T10:00:29

'eval' is used in optim.c, but it is used along with other things. I'm
looking for a standalone example to demonstrate the usage of eval in
C.  Does anybody have some a simple example? Thanks.

Problem with Rboolean in c++ code

Jan van der Laan — 2013-05-24T06:53:15

I am trying to use R_RegisterCFinalizerEx to ensure some c++ object is  
properly deleted after use. However, I run into some problems when  
trying to pass in the third argument which is an Rboolean.  The line  
'R_RegisterCFinalizerEx(p, finalizer, TRUE);' generates the following  
error when trying to compile using R CMD SHLIB:

g++ -I/usr/share/R/include -DNDEBUG      -fpic  -O2 -pipe -g  -c  
rboolean.cpp -o rboolean.o
rboolean.cpp: In function ?SEXPREC* create(SEXP)?:
rboolean.cpp:21:46: error: invalid conversion from ?int? to ?Rboolean?  
[-fpermissive]
In file included from /usr/share/R/include/Rdefines.h:29:0,
                  from rboolean.cpp:4:
/usr/share/R/include/Rinternals.h:764:6: error:   initializing  
argument 3 of ?void R_RegisterCFinalizerEx(SEXP, R_CFinalizer_t,  
Rboolean)? [-fpermissive]
make: *** [rboolean.o] Error 1


I have managed to reduce the problem to the example below:

===== foo.cpp =====
#include <R.h>
#include <Rdefines.h>

void foo() {
   Rboolean b = TRUE;
}
==========

Re: Using a shared object without installing a library required by the object.

Xiao He — 2013-05-24T00:09:40

Oops, Sorry, did not reply-all:

Thanks for the reply Simon,

I have a follow-up question:

So after I run the first code you suggested on a computer, the shared
object will use the Fortran run-time included in R. Does this code change
the shared object itself? Meaning if I send this object to yet another
computer and run it, will it know to link to the Fortran included in R?
Thanks!

-Best,
Xiao


On Thu, May 23, 2013 at 5:06 PM, Xiao He <praguewatermelon< at >gmail.com> wrote:


[[alternative HTML version deleted]]

Re: Using a shared object without installing a libraryrequired by the object.

Simon Urbanek — 2013-05-23T23:54:44


The Fortran run-time is included with R, so you only need to change the path -- e.g.

install_name_tool -change /usr/local/lib/libgfortran.2.dylib /Library/Frameworks/R.framework/Resources/lib/libgfortran.2.dylib /Users/xh/Downloads/foo2.so

You can make that permanent on your build machine by running

install_name_tool -id  /Library/Frameworks/R.framework/Resources/lib/libgfortran.2.dylib /usr/local/lib/libgfortran.2.dylib

If you do that, all code compiled against it subsequently will point to the version inside R instead.

Cheers,
Simon

FWIW: There is R-SIG-Mac for Mac-specific questions.

Re: Minimal build of R ...

peter dalgaard — 2013-05-23T21:55:37

On May 23, 2013, at 23:07 , Jony Hudson wrote:


Looks like SSIZE_MAX is usually <*/limits.h>:

pd$ grep -r SSIZE_MAX /usr/include/
/usr/include/i386/limits.h:#defineSSIZE_MAXLONG_MAX/* max value for a ssize_t */
/usr/include/limits.h:#define_POSIX_SSIZE_MAX32767
/usr/include/ppc/limits.h:#defineSSIZE_MAXLONG_MAX/* max value for a ssize_t */

If R_xlen_t is int, you need to adjust R_XLEN_T_MAX to INT_MAX or so.

The case warnings look like they are bound to happen on systems where int and long have the same size, and should presumably be harmless.

Re: Minimal build of R ...

Jony Hudson — 2013-05-23T21:07:40

Hi,

 I'm making some progress with this, but have hit a sticking point and am looking for a hint. Most of the compilation seems to be going ok, after some liberal use of f2c, but I'm getting compile errors in src/main/connections.c :

connections.c:926:43: error: use of undeclared identifier 'SSIZE_MAX'
    if ((double) size * (double) nitems > SSIZE_MAX)
                                          ^
connections.c:937:43: error: use of undeclared identifier 'SSIZE_MAX'
    if ((double) size * (double) nitems > SSIZE_MAX)
                                          ^
connections.c:3354:21: warning: implicit conversion from 'long long' to
      'R_xlen_t' (aka 'int') changes value from 4503599627370496 to 0
      [-Wconstant-conversion]
    nnn = (n < 0) ? R_XLEN_T_MAX : n;
        ~           ^~~~~~~~~~~~
../../src/include/Rinternals.h:65:23: note: expanded from macro 'R_XLEN_T_MAX'
# define R_XLEN_T_MAX 4503599627370496
                      ^~~~~~~~~~~~~~~~
connections.c:3662:11: error: duplicate case value '4

Using a shared object without installing a library required bythe object.

Xiao He — 2013-05-23T20:18:43

Dear all,

I have a C++ code. To create a shared object from this particular code, I
had to install a Fortran library on my computer (Mac). The compiled code
runs fine on my computer. However,  if I try to dyn.load() said shared
object on a computer that does not have the Fortran library installed, the
object won't load, and instead I get a message below:

usr/local/lib/libgfortran.2.dylib
 Referenced from: /Users/xh/Downloads/foo2.so

I wonder if there is any way to compile the original C++ code such that I
can include the necessary components of the Fortran library without having
to install the library.


Thank you in advance.

Best,
Xiao

[[alternative HTML version deleted]]

Code compilation: Drop certain statements in a function before calling it multiple times?

Henrik Bengtsson — 2013-05-23T19:19:38

Hi,

I make heavy use of verbose statements in my code, verbose output that
can be enabled/disabled via an argument.  Here is a dummy example:

foo <- function(n=10, verbose=FALSE) {
  res <- 0;
  for (k in 1:n) {
     if (verbose) cat("Iteration ", k, "...\n", sep="");
     res <- res + k;
     if (verbose) cat("Iteration ", k, "...done\n", sep="");  }
  }
  res;
}

Even with verbose=FALSE, one pay an noticeable overhead due to it when
calling foo(verbose=FALSE).  Thus, before calling it, i'd like to
pre-compile this function by dropping the verbose-related statements,
e.g.

if (verbose) {
  fooT <- dropVerbose(foo);
}

such that I basically get:

fooT <- function(n=10, verbose=FALSE) {
  res <- 0;
  for (k in 1:n) {
     res <- res + k;
  }
  res;
}

Just to clarify, the immediate use case for this is to compile local
functions, e.g.

bar <- function(..., verbose=FALSE) {
  foo <- function(...) { ... };
  if (verbose) {
    foo <- dropVerbose(foo);
  }
  foo(..., verbose=verbose);
}

Instead of me reinvent

minor typo in docs for 'sort'

Enrico Schumann — 2013-05-23T07:33:01

Dear all, 

on the help page for '?sort':


  'Method "shell" uses Shellsort ([...] from Sedgewick (1996))'


but in the references it is Sedgewick (1986). 1986 seems correct:

http://dx.doi.org/10.1016/0196-6774(86)90001-5


Thank you,
Enrico

Re: Inconsistent results from .C()

Prof Brian Ripley — 2013-05-23T06:08:17

lev2 is malloc-ed in the code you supplied.  The new option Bill refers 
to is about input variables in .C, i.e. lev and not lev2.  Other array 
overruns can be found by other tools: Bill mentioned valgrind, and 
AddressSanitizer finds a different set of overruns.  See 
http://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-gctorture-and-memory-access 
.

On 23/05/2013 01:18, William Dunlap wrote: