gmane.comp.cms.fedora-commons.devel

Re: [fcrepo-dev] Question regarding some statistics.

2012-05-25T12:32:16

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Please do, Frank, and thanks for checking into it.

- ---
A. Soroka
Software & Systems Engineering :: Online Library Environment
the University of Virginia Library

On May 25, 2012, at 8:04 AM, Asseg, Frank wrote:


-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQEcBAEBAgAGBQJPv3vRAAoJEATpPYSyaoIkf1cIALZ58EiEuTUBtyRAHo8ZWxVn
DuX+5UqAUqk0z4ukoJZ6cpfVIf2je1n8T1cy7g+nLlEbd47OZmVq3fLT0xYPhtHE
IVeZQD3jEfc9xNmunzctVJMxqIVi209YeCqpqOV5mMaMLKGZjBZ24RLagdyapqYq
LfcFN64EKn4mMe9VmVPRXytlqWvBFhJUH3VliVNlGMcx0tytI/L9S8YxNHyqeaIO
IpjI7vh3psgA25QM2mV38nckP7xINR8JQSq9GvgK7Zm4su0yHEyULFUlCkc5OD60
09XBtpDHddhQ0SowOqX/68+oSkFVJ+TnBARqkvvqiT/32b3E/EqAyxiKrMnrYV4=
=x0wv
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT manag

Re: [fcrepo-dev] Question regarding some statistics.

Asseg, Frank — 2012-05-25T12:04:45

Gola Again!

On 05/25/2012 01:23 PM, Asseg, Frank wrote:

Hmm i just stumbled over the synchronized Set used in the StreamManager, 
which might be the cause for the larger spread using akubra-fs.
We'll investigate this a bit and i'll let you guys know, if you're 
interested.

[fcrepo-dev] Question regarding some statistics.

Asseg, Frank — 2012-05-25T11:23:42

Hola Guys!

We're currently gathering some numbers regarding the performance of 
akubra-hdfs compared to the default akubra-fs implementation.
  All the numbers we gathered show an expected rise in e.g. ingest times 
when using akubra-hdfs.
  But when measuring the retrieve times with a sample set of small files 
(0.15MB - 2.1 MB) of 18000 files the results show a unintuitivly better 
result for akubra-hdfs compared to the default akubra implementation.
  Do you guys have an explanation for this behaviour?
  The only thing we could think of was that the HDFS Cluster does some 
caching and has better network latency than the local Harddisks.

I'll try to attach the images although i dont know if the list policy 
allows it ;)

Thanks!

Frank

[fcrepo-dev] Request for Comment: IIIF Image API Proposal

Tom Cramer — 2012-05-12T01:23:54

The International Image Interoperability Framework (http://lib.stanford.edu/iiif) is an initiative driven by several major research and national libraries to enable the rich and robust delivery of digital images through common interfaces, and to spur the development of open source and commercial software solutions in this space.

The IIIF Working Group invites comment and feedback on a proposed API for the the delivery of images via a standard http request. The full specification can be found at:

http://library.stanford.edu/iiif/image-api

The IIIF Image API specifies a web service that returns an image in response to a standard http or https request. The URL can specify the region, size, rotation, quality characteristics and format of the requested image. A URL can also be constructed to request basic technical information about the image to support client applications.

The IIIF Image API was conceived of to facilitate systematic reuse of image resources, and enable their delivery through a diversity of s

[fcrepo-dev] m2e, generated classes and lifecyclemapping

frank asseg — 2012-05-11T14:45:36

Hola guys,

To get rid of eclipse errors the following additions to poms worked for me:

for CXF's generated classes the following helped:

in fcrepo-common/pom.xml i added the following as stated in the SO 
question at 
http://stackoverflow.com/questions/7160006/m2e-and-having-maven-generated-source-folders-as-eclipse-source-folders

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <executions>
   <execution>
    <id>add-source</id>
    <phase>generate-sources</phase>
    <goals>
     <goal>add-source</goal>
    </goals>
    <configuration>
     <sources>
      <source>${project.build.directory}/generated-sources/cxf/</source>
     </sources>
    </configuration>
   </execution>
  </executions>
</plugin>



for the lifecycle mapping errors changing fcrepo-server/pom.xml 
according to this: 
http://wiki.eclipse.org/M2E_plugin_execution_not_covered#ignore_plugin_goal
did the trick

<pluginManagement>
  <plugins>
   <plugin>
    <groupId>org.eclipse.m2e</gr

Re: [fcrepo-dev] fcrepo-1023 && null nodes

Chris Wilper — 2012-05-11T11:47:50

Hi Frank,

Using bound="false" when a result variable is unbound (null) is what
the old version of the SPARQL results format[1] required.

The latest version of the specification[2] says to omit the element
altogether, which means doing the null check a bit earlier in the
process so the whole element can be skipped.

- Chris

[1] http://www.w3.org/TR/2004/WD-rdf-sparql-XMLres-20041221/
[2] http://www.w3.org/TR/2008/REC-rdf-sparql-XMLres-20080115/

On Fri, May 11, 2012 at 6:39 AM, frank asseg <frank.asseg< at >congrace.de> wrote:

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Fedora-commons-developers mailing list
Fedora-co

[fcrepo-dev] fcrepo-1023 && null nodes

frank asseg — 2012-05-11T10:39:52

Hola guys!

When implementing a SparqleW3CTupleWriter for 
https://jira.duraspace.org/browse/FCREPO-1023 in Trippi, which i 
basically copied from SparqlTupleWriter, i stumbled upon this null check in
https://github.com/fcrepo/trippi/blob/master/trippi-core/src/main/java/org/trippi/io/SparqlTupleWriter.java 
at line 55:

...
Node n = result.get(names[i]);
if ( n == null ) {
     m_out.println(" bound=\"false\"/>");
} else if ( n instanceof URIReference ) {
...

Im not quite sure how to handle this null checks in a Tuple Writer 
conforming to the docuemtnation at http://www.w3.org/TR/rdf-sparql-XMLres/


Can you give me a hint on how to handle these null nodes?

[fcrepo-dev] Search both diacritics and non-diacritics

Frank Feng — 2012-05-08T14:43:51

Hi all,

I'm developing a fedora search interface, and want to search both with and
without diacritics, for instance, if I search 'revesz', it will response
both 'revesz' and 'révész'. I've tried to configure solr to use
ASCIIFoldingFilterFactory <http://t.co/qEWsS9ko> but cannot get it working.
Any suggestions would be appreciated and a working solr config file will be
very helpful.

thanks,

Frank
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Fedora-commons-developers mailing list
Fedora-commons-developers-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/fedora-commons-

Re: [fcrepo-dev] FedoraGSearch locks and hogs file connections

Gert Schmeltz Pedersen — 2012-05-03T11:46:51

Hi Christian

Yes, I think that is most likely the cause.

Regards,
Gert


On 03/05/2012, at 11.39, Christian Rohrer wrote:



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Re: [fcrepo-dev] FedoraGSearch locks and hogs file connections

Christian Rohrer — 2012-05-03T09:39:11

Hi Gert

On 27.04.2012, at 17:47, Gert Schmeltz Pedersen wrote:


Thanks for your hint, I now switched it all back to fedoragsearch and the name of the index to FgsIndex, but I still got the problem. The locking itself occurs, when I update objects, which causes Fedora to create a lot of update messages. This in turn causes GSearch to lock up eventually. 

In the meantime I found another possible reason for the locking behaviour: I had both update mechanisms enabled, ie. Messaging and via REST. I have now disabled the update via REST and I haven't had any further locking errors since. Do you think this could be the cause for my troubles? 

Regards
christian

[fcrepo-dev] ANN: Fedora Office Hours

Chris Wilper — 2012-05-01T18:01:36

Hi all,

I've been thinking a lot lately about how best to serve the Fedora
developer community and have been impressed with the Office Hours
concept that Matt Zumwalt has been spearheading[1] in the Hydra
community. So I've decided to do something similar, starting this
week.

Here's how it will work:

Starting this Thursday, after the Fedora committer call, I'll be
spending the remainder of the business day ('till 5pm US Eastern time)
in the #duraspace IRC channel on Freenode. The main purpose will be to
support the work of Fedora committers, contributors, and integrators.
Got a dev question or just an idea you want to bounce off someone?
Stop by and let's talk about it.  As with all irc traffic in
#duraspace, public logs will be automatically made available at
http://irclogs.duraspace.org/

More specifics (and info about my availability, which will change if
I'm traveling, etc.) can always be found here:

https://wiki.duraspace.org/display/~cwilper/Fedora+Office+Hours

Hope to see you there!

- Chris

[1]

Re: [fcrepo-dev] My foxmlToSolr.xsl is breaking Tomcat

Remi Malessa — 2012-04-30T09:18:32

Hi Gert

I checked in oXygen and it is valid. Also, I've been working to sort out 
this error for a week or two now, so there is no
syntax errors, no worries.

It works when I run the transformation locally from oXygen against a 
Fedora object FOXML file.
It correctly goes to server an picks up all data from other objects  and 
datastreams.

The offending line is:


<xsl:variable name="articleText" 
select="document(concat('http://scifdev.llgc.org.uk:8080/mets_disseminators/gsearchGETDS/coordinates/', 
$pagePID, '/', $alto_articleNoRef))//articleText"/>

where

"http://scifdev.llgc.org.uk:8080/mets_disseminators/gsearchGETDS/coordinates/', 
$pagePID, '/', $alto_articleNoRef" is a call to my servlet.
The server works fine for other datastreams, or if called directly from 
a web browser.

If I uncomment the line , the Tomcat won't start.


Remi

Re: [fcrepo-dev] My foxmlToSolr.xsl is breaking Tomcat

Gert Schmeltz Pedersen — 2012-04-30T07:24:20

You may have syntax errors in your foxmlToSolr.xsl, in which case you will get log lines in the Tomcat log in catalina.out. Is there something there?

Gert


On 27/04/2012, at 13.03, Remi Malessa wrote:



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

[fcrepo-dev] Fwd: FedoraGSearch locks and hogs file connections

Gert Schmeltz Pedersen — 2012-04-27T15:54:33

Sorry, I confused two names, the mail just send was not in reply to Remi, it was to Christian Rohrer:

Begin forwarded message:


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Fedora-commons-developers mailing list
Fedora-commons-developers-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers

Re: [fcrepo-dev] FedoraGSearch locks and hogs file connections

Gert Schmeltz Pedersen — 2012-04-27T15:47:26

Hi Remi,

The variable time after a restart before the problems start indicates that it would be best to look at the operation(s) occurring just before the problems start, what does the log say there? I think that the gfindObjects operations, that you list log lines for, could hardly be responsible.

The requests from search engines, some with the old sort parameter AUTO? You have to explain how these relate to the problem.

I see in your code that you have renamed "fedoragsearch" to "indexer". Are you sure you have done this consistently? You may need to check carefully the fgsconfigFinal files. The problems with segmentation files and locks indicate that your configuration is not consistent.

Best
Gert


On 26/04/2012, at 13.02, Christian Rohrer wrote:



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
w

[fcrepo-dev] My foxmlToSolr.xsl is breaking Tomcat

Remi Malessa — 2012-04-27T11:03:55

Hi

In my current project I keep datastreams (METS, RELS-EXT, ALTO) as 
'Managed' content.
I created a Servlet  that fetches a datastream when requested, so in my 
foxmlToSolr.xsl I call the servlet
something like this:

<xsl:variable name="QueryString" 
select="concat('http://server:8080/servlet_path/datastream_name', $PID)"/>
<xsl:variable name="articleText" 
select="document($QueryString))/servletResponse/content/text()"/>

or:

<xsl:variable name="articleText" 
select="document(concat('http://server:8080/servlet_path/datastream_name', 
$PID)))/servletResponse/content/text()"/>

tried both...and It works fine with METS, RELS-EXT, but when I add a 
request for ALTO to the XSLT, Tomcat fails to start-up.

The last lines in the log are always:

16:27:35.870 [main] DEBUG org.apache.axis.AxisEngine - Exit: 
AxisEngine::init
16:27:35.871 [main] DEBUG o.a.a.s.DefaultAxisServerFactory - Exit: 
DefaultAxisServerFactory::getServer
16:27:35.871 [main] DEBUG o.a.axis.transport.http.AxisServlet - Exit: 
getEngine()

.

[fcrepo-dev] Correction: Islandora Announces 12.1.0 Release

David Wilcox — 2012-04-26T18:32:39

*The previous announcement referred to Islandora 11.3.0 in the subject
line, but the new version is actually 12.1.0. Sorry for the
confusion!*

We are pleased to announce the release of Islandora 12.1.0!

You can check out the release notes and download all available modules
from http://islandora.ca/download, or test drive the release at
http://sandbox.islandora.ca.

Islandora 12.1.0 features a beta version of the new Video Solution
Pack, along with improvements to the Collection Manager, Harvester,
and Solr modules. A number of bugs have also been fixed in this new
version. You can read about all the changes in the release notes:
https://wiki.duraspace.org/display/ISLANDORA6121/Release+Notes+and+Downloads.

The documentation is undergoing updates to match this version of
Islandora. You can find it here:
https://wiki.duraspace.org/display/ISLANDORA6121.

Please report any issues to the google developer
(http://groups.google.com/group/islandora-dev) or users
(http://groups.google.com/group/islandora) lists, o

Re: [fcrepo-dev] FedoraGSearch locks and hogs file connections

Christian Rohrer — 2012-04-26T11:02:18

Hi Gert

sorry for the late rely, I subscribed this list in digest-mode, and your message only got delivered today.



The time after a restart before the problem occurs ist quite variable. From a few minutes to several days. 

Activities are request like this:
INFO 2012-04-26 00:17:02,702 (RESTImpl) request=query=rels-ext.isMemberOfCollection%3A%22LOR%3A1%22+AND+fedora-model.hasModel%3A%22LORmodel%3Aobject%22&hitPageSize=5&hitPageStart=1&sortFields=PID%2CSCORE%2Cfalse%3Bfgs.label%2Cde-CH%2Cfalse&operation=gfindObjects&indexName=ChorIndex&restXslt=copyXml&resultPageXslt=copyXml& timeusedms=83
INFO 2012-04-26 00:17:02,819 (RESTImpl) request=query=rels-ext.isMemberOfCollection%3A%22LOR%3A1%22+AND+fedora-model.hasModel%3A%22LORmodel%3Afolder%22&operation=gfindObjects&hitPageSize=1000&indexName=ChorIndex&restXslt=copyXml&resultPageXslt=copyXml&sortFields=PID%2CSCORE%2Cfalse%3Bfgs.label%2Cde-CH%2Cfalse& timeusedms=101
INFO 2012-04-26 00:17:06,023 (RESTImpl) request=query=dcterms.source%3A%22LOR%3A5806%22+or+PID%

[fcrepo-dev] Islandora preconference at ALA June 22 - spaces still available - ALA registration not required

Kirsta Stapelfeldt — 2012-04-25T18:00:37

* Apologies for Cross Posting*

Interested in learning about the Islandora open-source digital asset
management system?  The Islandora team will be presenting a pre-conference
workshop at ALA suitable for new users and current implementers. Learn how
Islandora is simplifying the process of creating robust digital collections
and workspaces. Spots are limited, so sign up soon.

*Building Digital Collections Using Islandora*
Speaker: Mark Leggott, University of Prince Edward Island; Kirsta
Stapelfeldt, University of Prince Edward Island;

This session will introduce the key concepts underpinning the Islandora
Digital Asset Management system. Participants will learn the basics of
running this rich repository system and how to utilize common “solution
packs” to create and theme collections in Islandora. Participants will
learn how to begin modeling and curating custom data collections and how to
migrate from other repository systems, including ContentDM, DSpace and
more. Textbooks and system access will be p

Re: [fcrepo-dev] FedoraGSearch locks and hogs file connections

Gert Schmeltz Pedersen — 2012-04-20T08:46:11

Hi Christian,

Could you give a little more information, please. After a restart of tomcat, how much activity has happened before problems start? and what was that activity?

Gert


On 19/04/2012, at 09.05, Christian Rohrer wrote:



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2

[fcrepo-dev] FedoraGSearch locks and hogs file connections

Christian Rohrer — 2012-04-19T07:05:10

Hi list

I have recently upgraded our Fedora installation to 3.5 and GSearch (using Lucene) to 2.4.1.

Since this upgrade GSearch causes us problems. It keeps opening connections to the segmentation files, which remain open until the OS limit is reached, after which a restart of Tomcat is needed in order to close all open connection of GSearch. Has anybody else observed this behavior?

The other unusual thing (might be related?) are locks that throw error messages in the form:

dk.defxws.fedoragsearch.server.errors.GenericSearchException: Thu Apr 19 09:02:45 CEST 2012 IndexWriter new error indexName=ChorIndex :
; nested exception is: 
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock< at >/opt/fedora35/indexer/ChorIndex/write.lock
at dk.defxws.fgslucene.OperationsImpl.getIndexWriter(OperationsImpl.java:695)
at dk.defxws.fgslucene.OperationsImpl.browseIndex(OperationsImpl.java:151)
at dk.defxws.fedoragsearch.server.RESTImpl.browseIndex(RESTImpl.java:267)
at dk.defxws.fedor