gmane.comp.db.couchdb.user

Re: Couchdb Server Error 500

Alexander Shorin — 2013-06-19T03:35:22

Hi,

Try to increase max_dbs_open parameter's value in [couchdb] section of
the ini to some greater than yours value (100 by default).

If it wouldn't help or helps for some time, try also to increase
system limits for opened files for couchdb user in
/etc/security/limits.conf. Like:
couchdb         soft     nofiles         32000
couchdb         hard     nofiles         40000

--
,,,^..^,,,


On Wed, Jun 19, 2013 at 7:28 AM, dilpreet singh <giggs102-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

Re: Couchdb Server Error 500

dilpreet singh — 2013-06-19T03:28:06

the couchdb server log entry is like this :

[Tue, 18 Jun 2013 19:29:30 GMT] [error] [<0.29253.0>] Uncaught server
error: {error,all_dbs_active}
 43935 [Tue, 18 Jun 2013 19:29:30 GMT] [info] [<0.29253.0>] 127.0.0.1 - -
HEAD /db-2013-3-20 500

Is there a way to resolve this error . To work with with couchdb again , i
have to restart it which is not the correct way as i have to start querying
process all over again .

Thanks in advance


On Mon, Jun 17, 2013 at 5:51 PM, svilen <az-fYFLDUw5NEJ3N0eo3W6VwA< at >public.gmane.org> wrote:

Re: large replication URL

Cliffano Subagio — 2013-06-18T23:19:19

Hi Jens,

Specific version: Blue Coat SG510 Series, Version: SGOS 6.2.10.3, Release
id: 90684 Proxy Edition
HTH.



On Tue, Jun 18, 2013 at 6:00 PM, Cliffano Subagio <cliffano-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>wrote:

CouchDB Parameterized filtered replication Performance issue

Saroj patel — 2013-06-14T15:51:32

Hi,

We have developed an iOS application with replication from: {"couchdb":"Welcome","version":"1.1.1","bigcouch":"0.4.0"}.

We are facing performance issue with the parameterized filter replication. As required we need to filter the latest 6 months of date from the Couch database which contains around 20,000 documents. The filter function written in Javascript works fine but the replication takes around 2.5 to 4 minutes. The filter is based on the data for timeSinceEpoch. 
The filter function source is as follows:
{
"_id": "_design/segmenting",
"filters": {
"by_epoch": "function(doc, req) { var beginTime = req.query.beginTime;var endTime = req.query.endTime;if (doc.timeSinceEpoch >= beginTime && doc.timeSinceEpoch <= endTime) { return true;} else {return false;}}"
}


Also read about the erlang filter function to speed up the data filter process and converted the filter function in Erlang. Tested multiple times and observed that the filter replication takes around same time. Source for Erlang design docume

use cradle to delete db but cache didn't reset

qaqabincs — 2013-06-15T14:54:20

I use node 0.8.18, express3.2.5, cradle0.6.6. my problem is : after I
saved a doc into db, then I delete db to reset my testcase env, but when
I save the doc with the same id, cradle told me that doc is exsited!
source code as follow:

|db.save(id, acct, callback0);
//in the callback0 ...
db.destroy(callback1);
//in the callback1 ...
db.create(callback2);
//in the callback2 ... 
db.get(id, callback);|

then I got the same acct document, it seems still in the cradle cache.

can I clean the cache the same time I delete the db ?



qaqabincs

Re: CouchDB Language-Specific Examples

lenz — 2013-06-16T03:08:09

so based on github stats c# is most verbose and curl is the most compact.
sure we can learn something from that :-)

will write up some perl code in the next few days.

cheers
lenz


On Sat, Jun 15, 2013 at 6:20 AM, Jens Alfke <jens-eZVmTAPG3+5l57MIdRCFDg< at >public.gmane.org> wrote:

Re: large replication URL

Cliffano Subagio — 2013-06-18T08:00:54

Hi Jens,

It's an older Bluecoat SG.

Btw, I found this line on RFC 2616 re proxy and URI length support (party
like it's 1999 :p)

      Note: Servers ought to be cautious about depending on URI lengths
      above 255 bytes, because some older client or proxy
      implementations might not properly support these lengths.



On Tue, Jun 18, 2013 at 12:13 PM, Jens Alfke <jens-eZVmTAPG3+5l57MIdRCFDg< at >public.gmane.org> wrote:

Re: large replication URL

Cliffano Subagio — 2013-06-18T07:50:28

Thanks for the explanation and the hint to MAX_URL_LEN, Adam.

We do build from source, will patch manually for now to get around this
issue.


On Tue, Jun 18, 2013 at 10:26 AM, Adam Kocoloski <kocolosk-1oDqGaOF3Lkdnm+yROfE0A< at >public.gmane.org>wrote:

Re: large replication URL

Jens Alfke — 2013-06-18T02:13:32

On Jun 17, 2013, at 5:12 PM, Cliffano Subagio <cliffano-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:


That URL is only 6273 bytes long; I wouldn’t expect a proxy to reject it. Do you know what type of proxy server this is?

(I have an interest in this as I’m the main author of a CouchDB-compatible database, which has its own heuristics for how long to let such a URL get.)

—Jens

Re: large replication URL

Adam Kocoloski — 2013-06-18T00:26:03

Well, a GET request is the RESTful choice.  I do admit that the limit on the length of a URL is going to be much more stringent than the limit on the size of a POST body.

CouchDB already has code to split the request up if the generated URL is "too large".  Currently the value of "too large" is hard-coded at 7000 bytes:

https://github.com/apache/couchdb/blob/1.3.0/src/couch_replicator/src/couch_replicator_api_wrap.erl#L473

If you happen to be building from source you can lower that value to something your proxy will accept.

For dev< at > -- it would absolutely make sense to patch this so that the user could supply a parameter like max_url_length in the replication spec and override the default.  It may also make sense to turn the default into a config setting.

Adam

On Jun 17, 2013, at 8:12 PM, Cliffano Subagio <cliffano-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

large replication URL

Cliffano Subagio — 2013-06-18T00:12:01

Hi,

I have a CouchDB database which replicates from npm registry and I notice
that the replication URL can get large due to the revisions array in
atts_since parameter, to a point that its too large that the proxy I'm
using doesn't let the GET request through.

Here's a sample of the large URL
https://gist.github.com/cliffano/5726639/raw/ea0e416dbbd8928ade4809c38e5a3214a736c35c/gistfile1.txt

Is there any particular reason why this is a GET instead of a POST?
Is there any way to change the request to use POST?

Cheers,
Cliff.

Re: Couchdb Server Error 500

svilen — 2013-06-17T21:51:09

check couchdb logs to see eventualy what exact request caused it


On Mon, 17 Jun 2013 17:46:35 -0400
dilpreet singh <giggs102-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

Couchdb Server Error 500

dilpreet singh — 2013-06-17T21:46:35

Hi

I have aroung 180 databases in the couchdb , each of about 3Gb . I am using
the permanent views to run some queries in all the databases one by one .
Also i am using python-couchdb to access couchdb using python . I get the
following error after going through few databases :

 db = server[dbname]
  File
"/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/client.py",
line 137, in __getitem__
    db.resource.head() # actually make a request to the database
  File
"/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/http.py",
line 377, in head
    return self._request('HEAD', path, headers=headers, **params)
  File
"/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/http.py",
line 419, in _request
    credentials=self.credentials)
  File
"/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/http.py",
line 310, in request
    raise ServerError((status, error))
couchdb.http.ServerError: (500, '')


I googled it and found it to be some couchd

Re: Large views and concurrent access

mangvlad — 2013-06-15T13:04:31

The data in my db are read only.
Even to get 304 back takes twice as long when I have 2 requests.

I am using JMeter to do my test and getting the same results...
I also have tried several hosted couchdbs to make sure that it is not my 
local setup...

Re: Large views and concurrent access

mangvlad — 2013-06-15T13:00:18

I think my problem is not with the size or seep of parsing.
When I am making a single request, getting the response in 1 sec is ok.
Improving this speed would be a bonus...

My problem is that when I get more than one user, the time is rapidly increasing
with every new concurrent request...

Breaking views into smaller multiple views will be pretty difficult...
What is it that is shared between requests inside couchdb that is causing this?

Re: Large views and concurrent access

Stanley Iriele — 2013-06-14T22:38:48

I agree caching is always going to be faster when done correctly...if it
were pulling across the wire I would try compressing it because its the
transfer of 1.5 MB that's getting you.

But I guess without knowing the extent of dynamism your require with your
query ..is it possible to split your large view into smaller views that you
could query in clever ways?... Also is that is there a high write to read
ratio?.. It could be fixing the index during this
On Jun 14, 2013 1:52 PM, "Jens Alfke" <jens-eZVmTAPG3+5l57MIdRCFDg< at >public.gmane.org> wrote:

Re: Large views and concurrent access

Jens Alfke — 2013-06-14T20:51:28

On Jun 14, 2013, at 12:09 PM, mangvlad <mangvlad-/E1597aS9LQAvxtiuMwx3w< at >public.gmane.org> wrote:


It’s always going to be much faster to cache local objects than to download and parse 1.5MB of JSON, no matter how fast the CouchDB server gets.
Consider using conditional GETs, so you’ll get the updated view results if they’ve changed, and a simple 304 response if they haven’t.

—Jens

Re: Large views and concurrent access

mangvlad — 2013-06-14T20:38:18

Stanley,

I have tried to use lists, but its performance was 10x times worse,
than doing it in node.js.

Thanks,
Vlad

Re: Large views and concurrent access

Stanley Iriele — 2013-06-14T19:45:51

just out of curiosity ...why aren't you using a list function after the
view?...its certainly not the best way for filtering but it gives you the
HTTP request.


On Fri, Jun 14, 2013 at 12:09 PM, mangvlad <mangvlad-/E1597aS9LQAvxtiuMwx3w< at >public.gmane.org> wrote:

Re: Large views and concurrent access

mangvlad — 2013-06-14T19:09:17

Thanks Filippo.

I guess I can introduce some caching in my node.js layer, but ideally, 
I would like to "exhaust" my couchdb options first...

I also have tried using lists feature, but its performance is many times slower 
than doing it using node's V8.

Again, my original problem is in the fact that I am not allowed to use 
multiple keys filtering and group=false together, 
otherwise I can reduce my views inside couchdb and probably avoid
this concurrency issue all together...

Thanks,
Vlad

Re: Large views and concurrent access

Filippo Fadda — 2013-06-14T18:32:23

Maybe you should caching the result.

-Filippo

On Jun 14, 2013, at 6:58 PM, mangvlad wrote: