gmane.comp.file-systems.pvfs2.users

Re: Executing the Java binary on OrangeFS

Boyd Wilson — 2013-05-15T15:36:00

we will send the patch over...  we take extra special care of paying
customers  ;)

it was not much work, hopefully it works around the problem.

-b


On Wed, May 15, 2013 at 10:11 AM, Scott Roberts <scotty< at >jhu.edu> wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Executing the Java binary on OrangeFS

Scott Roberts — 2013-05-15T14:11:48

That was quick.  Sure, I'll take the patch to see if this Java application works.

Cheers,
--Scott

From: Boyd Wilson <boydw< at >omnibond.com<mailto:boydw< at >omnibond.com>>
Date: Wednesday, May 15, 2013 7:38 AM
To: Scott Roberts <scotty< at >jhu.edu<mailto:scotty< at >jhu.edu>>
Cc: PVFS2-Users <pvfs2-users< at >beowulf-underground.org<mailto:pvfs2-users< at >beowulf-underground.org>>
Subject: Re: [Pvfs2-users] Executing the Java binary on OrangeFS

Scott,

I got the code change I made earlier today tested.  We can now make mmap work if MAP_SHARED is used.  Essentially the call is reverted under the covers to be a MAP_PRIVATE.  To enable the support, all you have to do is load the kmod with the new fake_mmap_shared option.

Example:

insmod pvfs2.ko fake_mmap_shared=1

This will be in future releases, but if you want a patch let us know.

-boyd

On Tue, May 14, 2013 at 3:27 PM, Scott Roberts <scotty< at >jhu.edu<mailto:scotty< at >jhu.edu>> wrote:
Hi Boyd,

The Java application we're trying to run doesn't use shared mmap.  That's a -very– o

Re: Executing the Java binary on OrangeFS

Boyd Wilson — 2013-05-15T11:38:31

Scott,


I got the code change I made earlier today tested.  We can now make mmap
work if MAP_SHARED is used.  Essentially the call is reverted under the
covers to be a MAP_PRIVATE.  To enable the support, all you have to do is
load the kmod with the new fake_mmap_shared option.

Example:

insmod pvfs2.ko fake_mmap_shared=1

This will be in future releases, but if you want a patch let us know.

-boyd



On Tue, May 14, 2013 at 3:27 PM, Scott Roberts <scotty< at >jhu.edu> wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Executing the Java binary on OrangeFS

Scott Roberts — 2013-05-14T19:27:13

Hi Boyd,

The Java application we're trying to run doesn't use shared mmap.  That's a -very– old bug and I am surprised Sun/Oracle didn't fix it; maybe it breaks compatibility with an old application they wanted to support.

If you roll the fake out flag into the next version of OrangeFS we will give it a go.  No rush.  Our temporary workaround is to install the JDK on local disk on each compute node.

Thanks,
--Scott

From: Boyd Wilson <boydw< at >omnibond.com<mailto:boydw< at >omnibond.com>>
Date: Tuesday, May 14, 2013 1:50 PM
To: Scott Roberts <scotty< at >jhu.edu<mailto:scotty< at >jhu.edu>>
Cc: PVFS2-Users <pvfs2-users< at >beowulf-underground.org<mailto:pvfs2-users< at >beowulf-underground.org>>
Subject: Re: [Pvfs2-users] Executing the Java binary on OrangeFS

This is caused by the JVM requiring shared mmap, when the java app does not even need it and the current KM for OrangeFS does not support shared MMAP.  We have talked about implementing it, but that would take a while, we also have discussed just putting in a fake out flag

Re: Executing the Java binary on OrangeFS

Boyd Wilson — 2013-05-14T17:50:09

This is caused by the JVM requiring shared mmap, when the java app does not
even need it and the current KM for OrangeFS does not support shared MMAP.
 We have talked about implementing it, but that would take a while, we also
have discussed just putting in a fake out flag for the KM.   Do you know if
the java app you are trying to use actually uses shared mmap or not?  (if
not we can look at turning around the fake out flag pretty quickly).   This
is not limited to JVM , some other apps do the same thing (very very few
actually use shared mmap).

There is also this older java bug report (requesting the JVM not do this),
its here:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7085890

thanks,
-boyd


On Tue, May 14, 2013 at 12:30 PM, Scott Roberts <scotty< at >jhu.edu> wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Executing the Java binary on OrangeFS

Scott Roberts — 2013-05-14T16:30:39

Good afternoon,

Has anyone encountered issues executing applications directly from an OrangeFS mount?  Most applications like Python, Perl, &c. work fine.  I untarred the 64-bit version of Java 7 to the location /export/ofs/apps/src with no errors.   However, executing Java throws this error:

./java -version
mmap failed for CEN and END part of zip file
Error occurred during initialization of VM
java/lang/ClassNotFoundException: error in opening JAR file /export/ofs/apps/src/jdk1.7.0_21/jre/lib/rt.jar

Any ideas?

Cheers,
--Scott
_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Fast way to copy files to OrangeFS?

Boyd Wilson — 2013-05-13T15:47:18

There isn't a direct link, but if you go to:
http://www.omnibond.com/orangefs/docs/v_2_8_6/index.htm

and select installation instructions, then add Other clients, then select
Direct Interface, this is the lib to pre-load.

-boyd


On Mon, May 13, 2013 at 11:14 AM, Kyle Schochenmaier <kschoche< at >gmail.com>wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Fast way to copy files to OrangeFS?

Kyle Schochenmaier — 2013-05-13T15:14:51

Hi Michael -

I believe you just need to LD_PRELOAD the user libs and then cp will pick
up libpvfs2.

Regards,
~Kyle

Kyle Schochenmaier


On Mon, May 13, 2013 at 10:11 AM, Michael Robbert <mrobbert< at >mines.edu>wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Fast way to copy files to OrangeFS?

Michael Robbert — 2013-05-13T15:11:26

Can somebody explain how to do this or what this means?
Are you saying that there is a way to get the system cp or rsync 
commands to use libpvfs2 or am I completely misunderstanding. In either 
case I would love to know how to get my files copied faster. I'm also 
in the process of filling our new OrangeFS cluster with data from our 
old Lustre and Panasas clusters.

Mike

On 5/12/13 6:46 PM, Boyd Wilson wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Fast way to copy files to OrangeFS?

Boyd Wilson — 2013-05-13T12:07:33

Glad that sped things up!

-b


On Sun, May 12, 2013 at 10:00 PM, Scott Roberts <scotty< at >jhu.edu> wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Fast way to copy files to OrangeFS?

Boyd Wilson — 2013-05-13T00:46:44

Scott
You can pre-load the user lib and then use either cp -r or rsync and it
should help.

-Boyd.

Sent from my iPhone

On Sunday, May 12, 2013, Scott Roberts wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Fast way to copy files to OrangeFS?

Scott Roberts — 2013-05-12T17:37:13

All,

I'm migrating data from our old storage array to our new OrangeFS array.  Using rsync to transfer 14GB of data where the average file size is 70MB takes about 5 minutes with an average transfer speed of 60MB/s.  If I copy the files individually using pvfs2-cp they transfer at  750MB/s.

Does anyone have a script handy that will recursively copy directories using pvfs2-cp?

Cheers,
--Scott
_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

need some advices on load monitoring

xihuang sun — 2013-05-11T07:28:41

Hello all,
    I need to perform some tests on my small cluster running orangeFS2.8.7,
now I want to monitor my cluster's cpu,memory and network with each node's
details while running my test . I want to know how you developers did when
orangeFS was developed and is there any good tools for monitoring? Thanks a
lot..
_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Reg extracting stripe size

Becky Ligon — 2013-05-10T14:28:20

Vish:

Sorry for the delay!

On the client side of the house, we have a function called
get_default_distribution() that will return the user-defined distribution
from the config file or the default, which is simple_stripe.  These
distributions can contain many parameters, one of which is the stripe
size.  Look at sys-create.sm to see how the distribution is obtained and
used.  If you have any further questions, please post.

Thanks,
Becky


On Tue, May 7, 2013 at 5:16 PM, Becky Ligon <ligon< at >omnibond.com> wrote:

Re: Reg extracting stripe size

Becky Ligon — 2013-05-07T21:16:57

Vish:

You can probably get that information from the config file structure once
you have gone through the initialization .  I'll get back to you with a
function call or something.

Becky


On Tue, May 7, 2013 at 4:28 PM, Vishwanath Venkatesan
<vvenkates< at >gmail.com>wrote:

Re: Reg extracting stripe size

Nicholas Mills — 2013-05-07T20:56:23

Oops.

Now that I've actually read your email I see you were looking for a
function. My advice would be to follow the example of the pvfs2-viewdist
command. It looks like the key is to call PVFS_sys_geteattr to get the
value associated with DFILE_KEY and then call PINT_dist_decode to turn the
value buffer into a PINT_dist object that describes the distribution
parameters. However the PINT_ prefix suggests that this is a function
internal to PVFS.

Perhaps someone else can suggest an alternative?

-Nick Mills


On Tue, May 7, 2013 at 4:47 PM, Nicholas Mills <nlmills< at >g.clemson.edu>wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Reg extracting stripe size

Nicholas Mills — 2013-05-07T20:47:06

The "pvfs2-viewdist" command will give output similar to the following:

[nlmills< at >user001 ~]$ /opt/pvfs2/bin/pvfs2-viewdist -f
/newscratch/nlmills/tmp
dist_name = simple_stripe
dist_params:
strip_size:524288

Metadataserver:
tcp://pvfs003-myri0:3335,tcp://pvfs003-ib0:3335,tcp://pvfs003:3335
Number of datafiles/servers = 1
Datafile 0 -
tcp://pvfs003-myri0:3335,tcp://pvfs003-ib0:3335,tcp://pvfs003:3335, handle:
5044031582652437680 (45ffffffffd994b0.bstream)

-Nick Mills


On Tue, May 7, 2013 at 4:28 PM, Vishwanath Venkatesan
<vvenkates< at >gmail.com>wrote:

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Reg extracting stripe size

Vishwanath Venkatesan — 2013-05-07T20:28:22

Hi,

I was looking at programmatically extract the stripe size set for the pvfs2
file system using pvfs-2.8.2. I see that the file system provides functions
like pvfs2-dist-set-param, but there is no pvfs2-dist-get-param available
in pvfs-2.8.2. Am I missing something? or is there a different  function
that needs to be used.

Please let me know,

Thanks
Vish
_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

Re: Problems with 2.8.6-r9510 and kernel 3.2.29

Andrew Savchenko — 2013-04-19T12:15:24

Hello,

On Thu, 18 Apr 2013 16:06:19 -0400 Elaine Quarles wrote:

I can easily copy entire user directory tree, thus file content is
not an issue. It looks like that users hits race condition: bug
likely manifests itself on make -j8 on large source tree located on
OrangeFS. Right now I'm investigating this issue further.

Best regards,
Andrew Savchenko
_______________________________________________
Pvfs2-users mailing list
Pvfs2-users< at >beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

RE: Problems with 2.8.6-r9510 and kernel 3.2.29

Elaine Quarles — 2013-04-18T20:06:19

For the user who hits repeatedly hits the problem, could it be due to some
issue with the particular file(s) being accessed?

Thanks,
Elaine

-----Original Message-----
From: Andrew Savchenko [mailto:bircoph< at >gmail.com] 
Sent: Thursday, April 18, 2013 3:54 PM
To: Elaine Quarles
Cc: pvfs2-users< at >beowulf-underground.org; ofs-support< at >omnibond.com
Subject: Re: Problems with 2.8.6-r9510 and kernel 3.2.29

Hello,

On Thu, 18 Apr 2013 15:02:53 -0400 Elaine Quarles wrote:

Yes, systems are identical (both in hardware and software). Most clients
also serve as servers (I know this is suboptimal, but we can't do otherwise
right now).

It was build on the same node that fails, then it was propagated to all
other nodes in binary form.

The first occurrence seems to be random, but after the first time failures
seem to be permanent. One of our users somehow repeatedly hits this, I'm not
able to reproduce problem from scratch right now.

There is nothing more in the system log. Client log is scarce, because
verbose output

Re: Problems with 2.8.6-r9510 and kernel 3.2.29

Andrew Savchenko — 2013-04-18T19:53:37

Hello,

On Thu, 18 Apr 2013 15:02:53 -0400 Elaine Quarles wrote:

Yes, systems are identical (both in hardware and software). Most
clients also serve as servers (I know this is suboptimal, but we
can't do otherwise right now).


It was build on the same node that fails, then it was propagated to
all other nodes in binary form.


The first occurrence seems to be random, but after the first time
failures seem to be permanent. One of our users somehow repeatedly
hits this, I'm not able to reproduce problem from scratch right now.


There is nothing more in the system log. Client log is scarce,
because verbose output was disabled.

Best regards,
Andrew Savchenko
[E 16:26:41.039781] PVFS Client Daemon Started.  Version 2.8.6-orangefs-2012-10-03-190208
[D 16:26:41.053545] [INFO]: Mapping pointer 0x7fe5979d1000 for I/O.
[D 16:26:41.054344] [INFO]: Mapping pointer 0xe7a000 for I/O.
[E 20:27:01.082860] pvfs2-client-core shutting down.
[E 20:27:01.083872] critical device failure: Bad file descriptor
[E 20:27:01.083902