http://permalink.gmane.org/gmane.comp.db.couchdb.user hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.db.couchdb.user/18311 <pre>Hi, We are looking for CouchDB best practices for our upcoming project. We are currently reading a lot about it but I would like to ask group for any interesting hints or sites that can help my search. We are also welcome to here if any CouchDB expert interested to work with us as architecture/consultant for this project too. Thanks in advance for all the replies. Regards, Metin Pasaoglu Sr. Vice President Technology CISSP-ISSAP, ISSMP, CISM, CRISC, CISA EQUENEXT Inc Phone : 800-216-2792 Fax : 888-581-8078 Metin.Pasaoglu-hxCycuVieW5Wk0Htik3J/w&lt; at &gt;public.gmane.org &lt;mailto:Metin.Pasaoglu-hxCycuVieW5Wk0Htik3J/w&lt; at &gt;public.gmane.org&gt; </pre> Metin Pasaoglu 2012-05-16T22:09:19 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18310 <pre>This just means couchdb-lucene has encountered a document that doesn't have a "headers" element. In general you should add guard clauses anyway; function(doc) { if (doc.headers &amp;&amp; doc.headers.core &amp;&amp; doc.headers.core.user) { var res = new Document(); res.add( doc.headers.core.user ); return res; } return null; } On 16 May 2012 21:38, Mike Kimber &lt;mkimber-8PJ4d5d6HqQ&lt; at &gt;public.gmane.org&gt; wrote: </pre> Robert Newson 2012-05-16T21:27:41 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18309 <pre>Hi Mike, I just verified the Index Everything snippet at https://github.com/rnewson/couchdb-lucene locally, it worked fine for me. Perhaps you missed off a character when you pasted it into futon? B. On 16 May 2012 21:20, Mike Kimber &lt;mkimber-8PJ4d5d6HqQ&lt; at &gt;public.gmane.org&gt; wrote: </pre> Robert Newson 2012-05-16T21:24:45 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18308 <pre>Ok found it using http://jsonlint.com/. "attachment" needs to be 'attachment' or "default", 'default' depending on which version of the index all I use (http://wiki.fluidproject.org/display/fluid/Setting+Up+CouchDB+and+Lucene or https://github.com/rnewson/couchdb-lucene). All seems to be working now. Mike -----Original Message----- From: Mike Kimber [mailto:mkimber-8PJ4d5d6HqQ&lt; at &gt;public.gmane.org] Sent: 16 May 2012 21:20 To: user-TecvkXQCxlvWgOoZo4XcuR2eb7JE58TQ&lt; at &gt;public.gmane.org Subject: Couch-lucene - Index all - JSON.parse: expected ',' or '}' after property value in object Ive built/installed couch-lucene and pointed it at my couchdb database (1.2). It works when I index a a individual attribute/key pair, but when I try an use the index all js code on https://github.com/rnewson/couchdb-lucene via futon I get JSON.parse: expected ',' or '}' after property value in object JSON.parse: expected ',' or '}' after property value in object Ive tried formatting it etc and it all looks fine, but the error mes</pre> Mike Kimber 2012-05-16T21:16:40 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18307 <pre>I have couch-lucene working when I index a top level element in my documents. However when I try and reference a sub element as I would with a couchdb view no results are returned. A extract from my document is: { "_id": "00023157-c1f8-4e39-ab9a-d56726739b8d", "_rev": "1-45cd4f7506ccede344d682904556be4b", "type": "maven-build-profile", "schema": "TODO-idOfSchemaDocument", "headers": { "started": 1328795714442, "core": { "user": "buildadm", "ip": "127.0.0.1", "executionId": "a607c818-c568-40a9-adbc-0844e6bf9a02" } } } The following works { "_id": "_design/lucene", "_rev": "19-1c7be47e52f8a65e9bb387e8c3251dd4", "language": "javascript", "fulltext": { "by_type": { "index": "function(doc) { var res = new Document(); res.add( doc. type); return res; }" } } } curl http://127.0.0.1:5999/_fti/local/build-profiles-raw-mk/_design/lucene/by_type?q= maven-build-profile The following does not: { "_id": "_</pre> Mike Kimber 2012-05-16T20:38:13 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18306 <pre>Ive built/installed couch-lucene and pointed it at my couchdb database (1.2). It works when I index a a individual attribute/key pair, but when I try an use the index all js code on https://github.com/rnewson/couchdb-lucene via futon I get JSON.parse: expected ',' or '}' after property value in object JSON.parse: expected ',' or '}' after property value in object Ive tried formatting it etc and it all looks fine, but the error message states that the JSON is not valid. My design document looks like: { "_id": "_design/lucene", "_rev": "18-22fdad425fb9f8cd1d8e39ba4b871103", "language": "javascript", "fulltext": { "all": { "index": "function(doc) {var ret = new Document(); function idx(obj) {for (var key in obj) {switch (typeof obj[key]) {case 'object': idx(obj[key]); break; case 'function': break; default: ret.add(obj[key]); break;}}}; idx(doc); if (doc._attachments) {for (var i in doc._attachments) {ret.attachment("attachment", i); }}return ret; }" } } } The followi</pre> Mike Kimber 2012-05-16T20:20:12 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18305 <pre>Sorry for the repost. I was told user&lt; at &gt; is more appropriate for this kind of question: I'm trying to implement a basic way of displaying comments in the way that Hacker News provides, using CouchDB. Not only ordered hierarchically, but also, each level of the tree should be ordered by a "points" variable. If you're familiar with sites such as Hacker News or Reddit you know that each level of the comment tree is ordered by the amount of points and also the date they were posted - however, for sake of simplicity I'll just talk about a "points" variable. The idea is that I want a view to return it in the order I except, and not make many Ajax calls for example, to retrieve them and make them look like they're ordered correctly. This is what I got so far: Each document is a "comment". Each comment has a property `path` which is an ordered list containing all its parents. So for example, imagine I have 4 comments (with _id `1`, `2`, `3` and `4`). Comment `2` is children of `1`, comment `3` is children of `2`, a</pre> Luca Matteis 2012-05-16T10:11:17 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18304 <pre>Hi, all! I wanna make a couchapp, a little like slideshare to share some cc contents, but it seems that the management of multi-user's accounts is difficult to finish only by couchdb , is there some use case about the multi-user access ? Thanks! -- 文武之道，一张一弛。 </pre> googcheng 2012-05-15T18:00:00 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18303 <pre>Thanks everyone! The final db is 1.85gb. I altered my script to use _bulk_docs and it was much faster with the final not taking as much space. Either way, the final after compacting is much better than I expected! On Mon, May 14, 2012 at 8:03 PM, Paul Davis &lt;paul.joseph.davis-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org&gt;wrote: </pre> Tim Tisdall 2012-05-15T02:25:07 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18302 <pre>On Mon, May 14, 2012 at 3:08 PM, James Marca &lt;jmarca-gxN29gZi1SnrmNeUD2Wsk/Z8FUJU4vz8&lt; at &gt;public.gmane.org&gt; wrote: The data for this view is available using the _active_tasks URL: curl http://127.0.0.1:5984/_active_tasks Will give you a JSON blob that shows progress. </pre> Paul Davis 2012-05-15T00:03:46 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18301 <pre> yes, and if they get too big they get slow to open up with JSON parsers (which generally aren't streaming parsers). There are always trade-offs James </pre> James Marca 2012-05-14T23:58:40 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18300 <pre> You can do a lot with tweaking your _id. See this link : http://wiki.apache.org/couchdb/Performance#File_size Basically : - use shorter _ids - use sequential _ids. If you cannot (eg because you have multiple disconnected parts that will have to merge often and that would cause too many clashes), you can use couchdb's own semi-sequential generated uuids. </pre> Matthieu Rakotojaona 2012-05-14T21:03:14 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18299 <pre> On May 14, 2012, at 1:08 PM, James Marca wrote: For example, I have detector data with one record per 30 seconds. If I combine data into daily docs and save, after compaction the resulting database is much smaller than if I keep one document per observation. Isn’t that just because there are a lot fewer nodes in the b-tree? The disadvantage of large documents is that they’re expensive to update, and they don’t play well with sync (if there are multiple writers) as they become very prone to conflicts. —Jens </pre> Jens Alfke 2012-05-14T20:42:00 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18298 <pre> As the documentation of the compaction code explains, compaction must finish, and then the new smaller db replaces the old bigger db. That way if compaction stops, or breaks, you still have your data. The wiki does a decent job of explaining all this if you can find the right parts. What I said above is found at http://wiki.apache.org/couchdb/Technical%20Overview#Compaction </pre> James Marca 2012-05-14T20:12:34 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18297 <pre> the "status" tool in Futon will show you compaction progress Also, two other things. Insertions of data goes faster if you use the bulk_docs interface. To keep things under control, I like to insert about 100 docs at a time, but it depends on your doc size, really. Second, I have found in my own totally unscientific testing that large documents compact better than many small documents. For example, I have detector data with one record per 30 seconds. If I combine data into daily docs and save, after compaction the resulting database is much smaller than if I keep one document per observation. I ran these tests back around the 1.0.1 generation of CouchDB, but I think the reason compaction doesn't work well for small document is the same gzip doesn't work well for small documents...if there is very little repeated information in a document, then gzip and other compression utilities can't do much. The larger the doc, the more the text will have repeats, and the better the compression algorithms perfo</pre> James Marca 2012-05-14T20:08:24 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18296 <pre>Okay, I see that you can tell that it's running by doing a GET on the database in question and looking for the "compact_running":true . However, I don't seem to see any changes in the db's file size. On Mon, May 14, 2012 at 3:42 PM, Tim Tisdall &lt;tisdall-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org&gt; wrote: </pre> Tim Tisdall 2012-05-14T20:01:25 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18295 <pre>Yes, I did it with a PUT for each id. When you call for compaction, is there a way to see the progress or a way to know if it's done? On Mon, May 14, 2012 at 3:20 PM, Paul Davis &lt;paul.joseph.davis-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org&gt;wrote: </pre> Tim Tisdall 2012-05-14T19:42:01 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18294 <pre>How did you insert them? If you did a PUT per docid you'll still want to compact afterwards. On Mon, May 14, 2012 at 2:13 PM, Tim Tisdall &lt;tisdall-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org&gt; wrote: </pre> Paul Davis 2012-05-14T19:20:51 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18293 <pre>I've got several gigabytes of data that I'm trying to store in a couchdb on a single machine. I've placed a section of the data in an sqlite db and the file is about 5.9gb. I'm currently placing the same data into couchdb and while it hasn't finished yet, the file size is already 10gb and continuing to grow. The sqlite database is essentially a table of ids with a json block of text for each, so I figured the couchdb wouldn't be too much different in size. Does anyone have some recommendations on how to reduce the size of the db? Right now I've only inserted data and have not made any "updates" to documents, so there should be no revision copies to be cleared away. </pre> Tim Tisdall 2012-05-14T19:13:30 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18292 <pre> Hello, I am test driving spine.app ( https://github.com/maccman/spine.app ), and before I get too much time invested, I wonder if it makes sense to use for couchdb. </pre> bsquared 2012-05-14T15:28:57 http://permalink.gmane.org/gmane.comp.db.couchdb.user/18291 <pre> Thank you for the prompt attention. </pre> bsquared 2012-05-14T15:18:50 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.db.couchdb.user