http://blog.gmane.org/gmane.comp.jakarta.lucene.user hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54737 <pre>It appears that StringField instances are treated as literals, even though my analyzer lower-cases (on both write and read sides).  So, for example, I can match with a term query (e.g. "NEW YORK"), but only if the case matches.  If I use a QueryParser (or MultiFieldQueryParser), it never works because these query values are lowercased and don't match. I've found that using a TextField instead works, presumably because it's tokenized and processed correctly by the write analyzer.  However, I would prefer that queries match against the entire/exact phrase ("NEW YORK"), rather than among the tokens ("NEW" or "YORK"). What's the solution here? Thanks in advance.</pre> Shahak Nagiel 2013-05-21T14:06:14 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54733 <pre>Hello, In my understanding, there is no explicit way to group/facet on tokenized fields. We have older indices and are planning to implement faceting on those indices. All the fields in the indices use a ClassicAnalyzer and are tokenized. Is there a work around, through which I can facet on tokenized fields. Bobo-browse from LinkedIn is doing it, but I guess there is no active development happening over there, and is still compatible with Lucene 3.x. We are on Lucene 4.1. Your advice would help. TIA. </pre> Ramprakash Ramamoorthy 2013-05-21T06:37:44 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54724 <pre>I just upgraded from lucene 4.1 to 4.2.1. We believe we are seeing some different behavior. I'm using org.apache.lucene.queryparser.classic.QueryParser. If I pass the string "20110920/EXPIRED" (w/o quotes) to the parser, I get: org.apache.lucene.queryparser.classic.ParseException: Cannot parse '20110920/EXPIRED': Lexical error at line 1, column 17. Encountered: &lt;EOF&gt; after : "/EXPIRED" at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:131) We believe this used to work. I tried googling for this and found something that said I should use QueryParser.escape() on the string before passing it to the parser. However, that seems to break phrase queries (e.g., "John Smith" - with the quotes; I assume it's escaping the double-quotes and doesn't realize it's a phrase). Since it is a forward slash, I'm confused why it would need escaping of any of the characters in the string with the "/EXPIRED". Has anyone seen this? Scott </pre> Scott Smith 2013-05-19T18:50:42 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54722 <pre>Hi folks: Sorry about the cross-post. Luke is awesome, but sometimes you only have command line access to your production boxes. So I wrote CLue, a command line lucene interface in the likes of Luke: Please take a look and collaborators wanted :) https://github.com/javasoze/clue Thanks -John </pre> John Wang 2013-05-17T19:07:51 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54717 <pre>please,how I can solve this error? Exception in thread "main" java.lang.NoSuchMethodError: org.apache.lucene.search.IndexSearcher.&lt;init&gt;(Ljava/lang/String;)V -- View this message in context: http://lucene.472066.n3.nabble.com/ERROR-help-me-please-org-apache-lucene-search-IndexSearcher-init-Ljava-lang-String-V-tp4064195.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. </pre> fifi 2013-05-17T13:11:17 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54711 <pre>I want to know the max value of a long field. I read lucene api , but don't find any api about this? does someone can supply any hits about how to implement this. thanks very much. </pre> Hu Jing 2013-05-17T09:10:06 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54708 <pre>Hi, We're occasionally seeing a CorruptIndexException when a searcher is opened on a new index. When we see the exception, it looks like what is happening is that the searcher is opening the index after prepareCommit for segments_1 but before the commit is completed. Because there is no prior commit to fall back to, lucene throws the CorruptIndexException. Does this sounds feasible and is it a bug? My expectation is that lucene would treat the index as nonexistent until the first commit is completed. We're running lucene 3.5. Cheers, Geoff </pre> Geoff Cooney 2013-05-16T14:45:11 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54704 <pre>Hi, I was checking on Lucene Faceted search and need some guidance. http://lucene.apache.org/core/4_3_0/facet/org/apache/lucene/facet/doc-files/userguide.html#facet_indexingmentions about DocumentBuilder categoryDocBuilder = new CategoryDocumentBuilder(taxo); But I am unable to find DocumentBuilder and CategoryDocumentBuilder classes in lucene-facet-4.3.0.jar. Most of the online tutorials that I could find also mentions the same steps as in official doc These are present in an old version - http://lucene.apache.org/core/old_versioned_docs/versions/3_4_0/api/contrib-facet/org/apache/lucene/DocumentBuilder.html What are the equivalent ones in 4.3? Or any help in getting this implemented in 4.3? Thanks, Rajesh </pre> raj 2013-05-16T05:44:44 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54695 <pre>Is it possible to have a mix of compressed and uncompressed documents within a single index? That is, can I load an index created with Lucene 4.0 into 4.1 and defer the decision of whether or not to use CompressingStoredFieldsFormat until a later time, or even go back and forth between compressed and uncompressed codecs, if needed? I thought at first the answer would be an unequivocal "no", but then how would one migrate data from 4.0 to 4.1 without a full reindex? </pre> Vitaly Funstein 2013-05-15T20:37:50 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54687 <pre>Hi there, We've been having troubles with performance regarding IndexReader's * document&lt;http://lucene.apache.org/core/4_1_0/core/org/apache/lucene/index/IndexReader.html#document(int)&gt; *(int docID) method. In summary: Why would the *document&lt;http://lucene.apache.org/core/4_1_0/core/org/apache/lucene/index/IndexReader.html#document(int)&gt; *(int docID) take a few seconds? For some docIds it takes a millisecond and for some it takes up to a few seconds whereas it used to consistenly take a millisecond for each doc fetched. In depth: We have a set of 6 app servers and each serve around a million requests per day. Specs for these app servers are: Ubuntu x86_64 GNU/Linux 8GB RAM, Java 6 with an Xmx setting of 4 GB. Using Lucene 4.1. Current index size is 2.1 GB. We've started using Lucene more extensively in the last 6 months, our index size used be ~ 1.5 GB and back then we had no problems. Recently we acquired more data and the index size jumped to 2.1 GB. When we pushed this new data to production first we had Out of Memory org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:248) exception. The recommendation was to increase the memory so we upped the Xmx setting 4GB but then we had the https://issues.apache.org/jira/browse/LUCENE-1566 (Lucene bug caused by the JVM). Increasing the memory of the servers to 8GB seems to resolve this problem. Now, the system is stable and no more of those exceptions but the response times of Lucene searches increased by 50%. Debugging the issue, I realized that IndexReader's doc is taking an insane amount of time. Used to be a ms each but now it can take a few seconds to fetch a single doc. Is readChunkSize setting of FSDirectory have anything to do with it? (now that our index size is larger than the default chunk size?) </pre> G B 2013-05-14T20:19:14 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54683 <pre>Hello all, i was wondering why unindexed fields can't be boosted compare to lucene 3. since these fields are still in the score calculation when i checked the score explanation. Is there any clean way to pass this? Thanks Tamer </pre> Tamer Gür 2013-05-14T14:34:52 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54676 <pre>Hi, I was looking to a way to obtain FieldInfo(s) from the IndexReader; we need in some way to describe the index. Can I do this? -------- AtomicReader ar = &lt;indexReader&gt;.leaves().get(0).reader(); // than call ar.getFieldInfos(); -------- What I mean is, can I suppose every AtomicReader in leaves() contain the same informations about FieldInfos and I do not need to iterate over leaves()? Nicola. </pre> Nicola Buso 2013-05-14T10:35:08 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54672 <pre>Hi team, I'm working with apache lucene 4.2.1 and I would like to store lucene index in a NoSql database. So my questions are, - Can I store the lucene index in a mongodb database ? thanks you team! </pre> Rider Carrion Cleger 2013-05-14T08:35:30 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54666 <pre>All - We have a necessity to check for attributes with null values. There are 2 options that we narrowed down to 1. Either we index null values with 'NULL' string and then check against the string. This has a drawback as Lucene does not have the capability of setting any default value. and each document may have a different set of fields to index. 2. The other approach is to check the negative query as follows, which does fetch me all the documents that has category with null *:* -_category:[* TO *] Question: 1. Is there any impact in the query peformance (execution time) if you do the above query because we fetch all documents and then negate. Has anybody tried checking the Query performance 2. Assume we are talking of 10 million documents in the index. -Vidhya -- View this message in context: http://lucene.472066.n3.nabble.com/Performance-of-NULL-check-category-TO-tp4063021.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. </pre> srividhyau 2013-05-13T18:05:18 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54665 <pre>We are using Lucene 3.0.3. Is there a way to set a default value to all fields being indexed in Lucene? Say, i want to set the default value as "NULL", indexed=NOT_ANALYZED, stored=false. This default value will be used, when a particular document does not have a value set for any field. -Vidhya -- View this message in context: http://lucene.472066.n3.nabble.com/Default-Value-for-All-Indexed-Fields-tp4063007.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. </pre> srividhyau 2013-05-13T17:44:31 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54661 <pre>Team, We have three indices by three different versions of lucene(2.3,3.6 and 4.1). Is there anyway I can identify which index belongs to which version somehow programatically? Thanks in advance. </pre> Ramprakash Ramamoorthy 2013-05-13T12:35:15 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54660 <pre>Hi all, I am using MultiReader for my searches and each reader has different fields. For one of the Reader. i need to add sort option which will be used only that specific reader. is it possible to do this with MultiReader? Thanks for help Tamer </pre> Tamer Gür 2013-05-13T10:24:01 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54656 <pre>My situation is that There are 10,000,000 documents, and I Build index every 5,000 documents. while *in every build*, I follow these steps: IndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_40, analyzer); iwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND); iwc.setRAMBufferSizeMB(ram); int maxDocs = itmParamParser.maxBufferedDocs; iwc.setMaxBufferedDocs(100000); logger.info("build | RAMBufferSizeMB="+ram+" MB, MaxBufferedDocs="+maxDocs); writer = new IndexWriter(directory, iwc);and I found it build slower and slower, so that I couldn't stand the speed !Any suggestion, thank guys. ----- -------------------------- Email: wuqiu.main&lt; at &gt;qq.com -------------------------- -- View this message in context: http://lucene.472066.n3.nabble.com/why-did-I-build-index-slower-and-slower-tp4062798.html Sent from the Lucene - Java Users mailing list archive at Nabble.com.</pre> wgggfiy 2013-05-13T03:05:00 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54653 <pre>We have the following code SegmentInfos segments = new SegmentInfos(); segments.read(luceneDir); for(SegmentInfoPerCommit sipc: segments) { String name = sipc.info.name; SegmentReader reader = new SegmentReader(sipc, 1, new IOContext()); Terms terms = reader.terms("content"); TermsEnum tEnum = terms.iterator(null); tEnum.docFreq(); //VAL=0 tEnum.totalTermFreq(); //VAL=-1 } The field "content" is indexed as DOCS_FREQ_AND_POSITION Why does the docFreq returned as 0 for all terms. Is this expected or am I doing something wrong? -- Ravi </pre> Ravikumar Govindarajan 2013-05-10T12:24:50 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54652 <pre>Hello, After search when im fetching the fragments to show result like google using method: String[] fragments = hi.getBestFragments(tokn, contents,20); im getting the best fragments. When i am iterating through the fragments i get one by one all fragments. These fragments contains search word but not the complete sentence. I want to show user the complete sentence with additional lines ( 2 lines first and 2 line after ) so that user can understand the meaning of fragment. How can i do this in Lucene?? </pre> neeraj shah 2013-05-10T05:34:12 http://comments.gmane.org/gmane.comp.jakarta.lucene.user/54649 <pre>We are totally confused and are wondering if this is a known bug. We are currently using lucene 3.6.0 to index with and for our searches. Our indexing software was compiled under openJdk 1.6.0-b09. Our index was built under HotSpot 1.6.0_43-b01 We are trying to run our index under openJdk 1.6.0_0 Yes, I know these are old jvms, but at this point we are stuck with them. If we run our search software under HotSpot (the machine we built the index on) we get correct answers. If we run our search software under openJdk 1.6.0_0 we consistently get the same (incorrect) answer. Is there a feature/bug in lucene that is causing this discrepancy? Thanks so much Liz Sommers lizsworks&lt; at &gt;gmail.com </pre> Liz Sommers 2013-05-09T16:37:53 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.jakarta.lucene.user