gmane.comp.java.hadoop.zookeeper.user

simple zookeeper client in C

Pramod Srinivasan — 2013-05-21T23:09:39

Hi ,

I am trying out a simple c client for zookeeper, made a copy of zookeeper-3.4.5/src/c/src/cli.c just to understand how the c APIs works, the program is attached.

I see errors/coredumps being generated, I am surely doing something wrong, any help appreciated.

case 1:

.../zookeeper-3.4.5/src/c> ./myt localhost:2181
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
Segmentation fault (core dumped)

starting it under gdb:
(gdb) r localhost:2181
Starting program: .../zookeeper-3.4.5/src/c/myt localhost:2181
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff70d9700 (LWP 9489)]
[New Thread 0x7ffff68d8700 (LWP 9490)]
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
2013-05-21 15:58:03,210:9485(0x7ffff7fd7700):ZOO_ERROR< at >handle_socket_error_msg< at >1643: Socket [127.0.0.1:2181] zk retcode=-7, errno=110(Connection timed out): connection to 127.0.0.1:2181 timed out (exceeded timeout by 1ms)
2013-05-21 15:58:03,210:9485(0x7ffff70d9700):ZOO_ERROR< at >handle_socket_error_msg< at >1643: Socket [127.0.0.1:2181] zk retcode=-7, errno=110(Connection timed out): connection to 127.0.0.1:2181 timed out (exceeded timeout by 2ms)
Watcher SESSION_EVENT state = CONNECTING_STATE
Watcher SESSION_EVENT state = CONNECTING_STATE
2013-05-21 15:58:13,212:9485(0x7ffff7fd7700):ZOO_WARN< at >zookeeper_interest< at >1557: Exceeded deadline by 10003ms
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...

Program received signal SIGSEGV, Segmentation fault.
queue_buffer (list=0x6030b8, b=0x0, add_to_front=0) at src/zookeeper.c:970
970         b->next = 0;
(gdb) bt
#0  queue_buffer (list=0x6030b8, b=0x0, add_to_front=0) at src/zookeeper.c:970
#1  0x00007ffff7bc939d in check_events (events=<optimized out>, zh=0x603040) at src/zookeeper.c:1727
#2  zookeeper_process (zh=0x603040, events=<optimized out>) at src/zookeeper.c:2170
#3  0x0000000000401448 in main (argc=2, argv=0x7fffffffeab8) at mytest.c:542

case 2:

2013-05-21 15:49:25,811:2493(0x7fae0ec17700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
2013-05-21 15:49:25,812:2493(0x7fae0fb16700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
2013-05-21 15:49:25,813:2493(0x7fae0ec17700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
2013-05-21 15:49:25,814:2493(0x7fae0fb16700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
2013-05-21 15:49:25,814:2493(0x7fae0ec17700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
2013-05-21 15:49:25,815:2493(0x7fae0fb16700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
2013-05-21 15:49:25,816:2493(0x7fae0ec17700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
Segmentation fault (core dumped)

under gdb:
2013-05-21 16:05:13,598:10413(0x7ffff7fd7700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
2013-05-21 16:05:13,598:10413(0x7ffff70d9700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
2013-05-21 16:05:13,599:10413(0x7ffff7fd7700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed
2013-05-21 16:05:13,599:10413(0x7ffff70d9700):ZOO_ERROR< at >handle_socket_error_msg< at >1579: Socket [:0] zk retcode=-1, errno=97(Address family not supported by protocol): socket() call failed

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff70d9700 (LWP 10415)]
zookeeper_interest (zh=0x603040, fd=0x7ffff70d8e38, interest=0x7ffff70d8e3c, tv=0x7ffff70d8e20)
    at src/zookeeper.c:1577
1577                zh->fd = socket(zh->addrs[zh->connect_index].ss_family, SOCK_STREAM, 0);
(gdb) bt
#0  zookeeper_interest (zh=0x603040, fd=0x7ffff70d8e38, interest=0x7ffff70d8e3c, tv=0x7ffff70d8e20)
    at src/zookeeper.c:1577
#1  0x00007ffff7bd1361 in do_io (v=0x603040) at src/mt_adaptor.c:378
#2  0x00007ffff72eee9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#3  0x00007ffff78f3cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x0000000000000000 in ?? ()


case 3
zookeeper-3.4.5/src/c> ./myt localhost:2181
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
2013-05-21 15:54:04,078:7030(0x7f36727cf700):ZOO_ERROR< at >handle_socket_error_msg< at >1643: Socket [127.0.0.1:2181] zk retcode=-7, errno=110(Connection timed out): connection to 127.0.0.1:2181 timed out (exceeded timeout by 7ms)
Watcher SESSION_EVENT state = CONNECTING_STATE
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
2013-05-21 15:54:24,085:7030(0x7f36736ce700):ZOO_ERROR< at >handle_socket_error_msg< at >2245: Socket [127.0.0.1:2181] zk retcode=-2, errno=115(Operation now in progress): unexpected server response: expected 0x14584484, but received 0xfffffffe
Watcher SESSION_EVENT state = CONNECTING_STATE
2013-05-21 15:54:34,088:7030(0x7f36727cf700):ZOO_ERROR< at >handle_socket_error_msg< at >1721: Socket [127.0.0.1:2181] zk retcode=-4, errno=112(Host is down): failed while receiving a server response
2013-05-21 15:54:34,091:7030(0x7f36736ce700):ZOO_ERROR< at >handle_socket_error_msg< at >1721: Socket [127.0.0.1:2181] zk retcode=-4, errno=112(Host is down): failed while receiving a server response
2013-05-21 15:54:44,099:7030(0x7f36727cf700):ZOO_ERROR< at >handle_socket_error_msg< at >1721: Socket [127.0.0.1:2181] zk retcode=-4, errno=112(Host is down): failed while receiving a server response
2013-05-21 15:54:44,099:7030(0x7f36736ce700):ZOO_ERROR< at >handle_socket_error_msg< at >1721: Socket [127.0.0.1:2181] zk retcode=-4, errno=112(Host is down): failed while receiving a server response

The java client worked perfectly fine and the zkCli.sh works fine as well./**
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

#include <zookeeper.h>
#include <proto.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <sys/time.h>
#include <unistd.h>
#include <sys/select.h>

#include <time.h>
#include <errno.h>
#include <assert.h>


#define _LL_CAST_ (long long)

static zhandle_t *zh;
static clientid_t myid;
static const char *clientIdFile = 0;
struct timeval startTime;
static char cmd[1024];
static int batchMode=0;

static int to_send=0;
static int sent=0;
static int recvd=0;

static int shutdownThisThing=0;

static const char *hostPort;
static int verbose = 0;

static __attribute__ ((unused)) void 
printProfileInfo(struct timeval start, struct timeval end, int thres,
                 const char* msg)
{
  int delay=(end.tv_sec*1000+end.tv_usec/1000)-
    (start.tv_sec*1000+start.tv_usec/1000);
  if(delay>thres)
    fprintf(stderr,"%s: execution time=%dms\n",msg,delay);
}

static const char* state2String(int state){
  if (state == 0)
    return "CLOSED_STATE";
  if (state == ZOO_CONNECTING_STATE)
    return "CONNECTING_STATE";
  if (state == ZOO_ASSOCIATING_STATE)
    return "ASSOCIATING_STATE";
  if (state == ZOO_CONNECTED_STATE)
    return "CONNECTED_STATE";
  if (state == ZOO_EXPIRED_SESSION_STATE)
    return "EXPIRED_SESSION_STATE";
  if (state == ZOO_AUTH_FAILED_STATE)
    return "AUTH_FAILED_STATE";

  return "INVALID_STATE";
}

static const char* type2String(int state){
  if (state == ZOO_CREATED_EVENT)
    return "CREATED_EVENT";
  if (state == ZOO_DELETED_EVENT)
    return "DELETED_EVENT";
  if (state == ZOO_CHANGED_EVENT)
    return "CHANGED_EVENT";
  if (state == ZOO_CHILD_EVENT)
    return "CHILD_EVENT";
  if (state == ZOO_SESSION_EVENT)
    return "SESSION_EVENT";
  if (state == ZOO_NOTWATCHING_EVENT)
    return "NOTWATCHING_EVENT";

  return "UNKNOWN_EVENT_TYPE";
}

void watcher(zhandle_t *zzh, int type, int state, const char *path,
             void* context)
{
    /* Be careful using zh here rather than zzh - as this may be mt code
     * the client lib may call the watcher before zookeeper_init returns */

    fprintf(stderr, "Watcher %s state = %s", type2String(type), state2String(state));
    if (path && strlen(path) > 0) {
      fprintf(stderr, " for path %s", path);
    }
    fprintf(stderr, "\n");

    if (type == ZOO_SESSION_EVENT) {
        if (state == ZOO_CONNECTED_STATE) {
            fprintf(stderr, "Connected ...\n");
        } else if (state == ZOO_AUTH_FAILED_STATE) {
            fprintf(stderr, "Authentication failure. Shutting down...\n");
            zookeeper_close(zzh);
            shutdownThisThing=1;
            zh=0;
        } else if (state == ZOO_EXPIRED_SESSION_STATE) {
            fprintf(stderr, "Session expired. Shutting down...\n");
            zookeeper_close(zzh);
            shutdownThisThing=1;
            zh=0;
        }
    }
}

int main(int argc, char **argv) {
    fd_set rfds, wfds, efds;
    int processed=0;
    char buffer[4096];
    char p[2048];
    int bufoff = 0;
    FILE *fh;

    if (argc < 2) {
        fprintf(stderr,
                "USAGE %s hostport\n", argv[0]);
        return 2;
    }
    verbose = 0;
    zoo_set_debug_level(ZOO_LOG_LEVEL_WARN);
    zoo_deterministic_conn_order(1); // enable deterministic order
    hostPort = argv[1];
    zh = zookeeper_init(hostPort, watcher, 30000, &myid, 0, 0);
    if (!zh) {
        return errno;
    }

    FD_ZERO(&rfds);
    FD_ZERO(&wfds);
    FD_ZERO(&efds);
    while (!shutdownThisThing) {
        int fd;
        int interest;
        int events;
        struct timeval tv;
        int rc;
        zookeeper_interest(zh, &fd, &interest, &tv);
        if (fd != -1) {
            if (interest&ZOOKEEPER_READ) {
                FD_SET(fd, &rfds);
            } else {
                FD_CLR(fd, &rfds);
            }
            if (interest&ZOOKEEPER_WRITE) {
                FD_SET(fd, &wfds);
            } else {
                FD_CLR(fd, &wfds);
            }
        } else {
            fd = 0;
        }
        FD_SET(0, &rfds);
        rc = select(fd+1, &rfds, &wfds, &efds, &tv);
        events = 0;
        if (rc > 0) {
            if (FD_ISSET(fd, &rfds)) {
                events |= ZOOKEEPER_READ;
            }
            if (FD_ISSET(fd, &wfds)) {
                events |= ZOOKEEPER_WRITE;
            }
        }
        zookeeper_process(zh, events);
    }
    zookeeper_close(zh);
    return 0;
}

Transaction log: /logs/jhe/nc/apps/zookeeper/version-2/log.1400470027 has invalid magic number 0 != 1514884167

suman p — 2013-05-21T13:03:47

Hello,

 We are running zookeeper ensemble of 3 servers.


Today one of our sever died. When i restarted the sever i am getting the
following error


java.io.IOException: Transaction log:
/logs/jhe/nc/apps/zookeeper/version-2/log.1400470027 has invalid magic
number 0 != 1514884167
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:560)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:577)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:543)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.init(FileTxnLog.java:525)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.<init>(FileTxnLog.java:504)
        at
org.apache.zookeeper.server.persistence.FileTxnLog.read(FileTxnLog.java:341)
        at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:132)
        at
org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
        at
org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:417)
        at
org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:409)
        at
org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
        at
org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
        at
org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
[2013-05-21 00:30:32,496] [ERROR]
[org.apache.zookeeper.server.quorum.QuorumPeerMain] [Unexpected exception,
exiting abnormally]
java.lang.RuntimeException: Unable to run quorum server
        at
org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:454)
        at
org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:409)
        at
org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
        at
org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
        at
org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Caused by: java.io.IOException: Transaction log:
/logs/jhe/nc/apps/zookeeper/version-2/log.1400470027 has invalid magic
number 0 != 1514884167
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:560)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:577)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:543)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.init(FileTxnLog.java:525)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.<init>(FileTxnLog.java:504)
        at
org.apache.zookeeper.server.persistence.FileTxnLog.read(FileTxnLog.java:341)
        at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:132)
        at
org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
        at
org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:417)
        ... 4 more



Here is our configuration

tickTime=2000

initLimit=10

syncLimit=5

dataDir=/data

clientPort=2181

dataLogDir=/logs/zookeeper/

minSessionTimeout=5000
maxSessionTimeout=100000

server.1=host1:3888:4888
server.2=host2:3888:4888
server.3=host3:3888:4888


autopurge.snapRetainCount=3
autopurge.purgeInterval=1


Not sure what went wrong.  Any help is greatly appreciated.


Regards,
Suman

zookeeper

suman p — 2013-05-21T12:46:03

Zookeeper test

Michael Black — 2013-05-16T19:59:32

Installing zookeeper-3.4.5 for the first time.


I was able to compile with "ant" and start up the server with
"bin/zkServer.sh start" on RedHat.

 

But running the test (or the startup) as the docs show doesn't work (Step 6
and Step 7 both don't work).

 

From the zookeeper directory I run (I noted that there is no zookeeper.jar
so I did change that to point to what was built - perhaps the docs should
reflect that too).

java -cp
zookeeper-3.4.5.jar:lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:lib/
log4j-1.2.15.jar:conf:src/java/lib/jline-0.9.94.jar \
org.apache.zookeeper.ZooKeeperMain -server

 

I'm a Java idiot so I don't understand why it can't find ZooKeeperMain -
it's in the jar.

jar tf zookeeper-3.4.5.jar|grep ZooKeeperMain

org/apache/zookeeper/ZooKeeperMain$1.class

org/apache/zookeeper/ZooKeeperMain$2.class

org/apache/zookeeper/ZooKeeperMain$MyCommandOptions.class

org/apache/zookeeper/ZooKeeperMain$MyWatcher.class

org/apache/zookeeper/ZooKeeperMain.class

org/apache/zookeeper/ZooKeeperMain.java

 

Same goes for the java startup on the server too (even though the
zkServer.sh script works just fine).

 

 

Michael Black

[DISCUSS] recipes package

FPJ — 2013-05-15T18:25:13

With Curator, I was wondering if it makes sense to maintain the recipes
package. How do people feel about it? Should we keep it or consider
deprecating it?

 

-Flavio

Use zookeeper as Load Blancer

Neo Qing — 2013-05-15T05:26:17

Hi,There
  I'm new to zookeeper, currently I'm working on a project, I have a
cluster of server, client need to connection server (TCP/IP), and we need
to keep the connection long live.
  To do the fail over and load balance , I was wondering whether zookeeper
can handle this case, especially for long live connection.
  thanks in advance.

Neo

Use zookeeper as Load Blancer and Fail Over

Neo Qing — 2013-05-15T06:48:54

Hi,There
  I'm new to zookeeper, currently I'm working on a project, I have a
cluster of server, client need to connection server (TCP/IP), and we need
to keep the connection long live.
  To do the fail over and load balance , I was wondering whether zookeeper
can handle this case, especially for long live connection.
  thanks in advance.

Neo

[ANNOUNCE] New ZooKeeper committer: Alex Shraer

Benjamin Reed — 2013-05-13T15:56:47

The Apache ZooKeeper PMC recently extended committer karma to Alex and he
has accepted. Alex has made some great contributions and we are looking
forward to even more :) Congratulations and welcome aboard, Alex!

ben

Client host selection?

Owen Kim — 2013-05-02T00:26:15

Is the client configurable on which host it tries to connect to when I give
it the host list? Will it perhaps try deterministically in order?
Specifically, I'd like it to favor ZK hosts in the same DC.

Help understanding the protocols

Arink Verma — 2013-04-28T12:57:55

Hello everyone

I am Arink, computer science student and open source enthusiastic. I am
working on pure zookeeper client on twitter's finagle. But I am not aware
of zookeeper's protocols. How it make connection with server?

can someone provide me any resource and help?

Thanks

Understanding the protocols

Arink Verma — 2013-04-28T13:13:29

Hello everyone

I am Arink, computer science student and open source enthusiastic. I am
working on pure zookeeper client on twitter's finagle. But I am not aware
of zookeeper's protocols. How it make connection with server?

I have read ZooKeeper Internals[1], but that is so brief to implement fresh
client from scratch on scala .


can someone provide me any resource and help?


[1]
http://zookeeper.apache.org/doc/r3.1.2/zookeeperInternals.html#sc_guaranteesPropertiesDefinitions

Thanks

Running a 3-node ensemble on a single machine

Drew Kutcharian — 2013-04-22T22:23:50

Hi Guys,

For development purposes I would like to run a 3 node Zookeeper ensemble on a single machine (Mac OS X). What's the cleanest way to do this?

Thanks,

Drew

Fully Elastic Zookeeper Ensembles

Dave Katz — 2013-04-22T21:44:26

I've been thinking about the implications of running Zookeeper in a fully dynamic distributed system, in which the number of nodes can be as few as one, or can be quite large. This has led to a few questions.

The dynamic server reconfiguration work appears to require a working quorum of servers under the old config in order to distribute the new config. This implies that the mechanism cannot be used if a quorum is lost (a common-mode failure across many servers). This leads to the obvious question, how does one recover from a (semi-)permanent loss of quorum? This would seem to require the HOG (Hand Of God) approach, with an external agent restarting the ZK servers with a new (shorter) server list. Presumably, the loss of quorum means a potential loss of state, since updates may not have made it to any of the surviving servers.

If servers come to the ensemble with disparate contents, how does ZK converge on the new state? From what I've been able to read, it appears that all servers will end up converging to the state of the newly elected leader (and so any divergent contents on other nodes are discarded). Is this the case?

If the system is to be fully dynamic, we have to deal with the two-node problem. How best to do this? In a two-node ensemble, it is guaranteed that if one of the nodes fails, the other node is guaranteed to be consistent, true? So if there is an external mechanism to prevent split brain, it should be possible to restart the surviving node in standalone mode, and once the second node returns, restarting both nodes should still guarantee consistency, yes?

Thanks in advance,

--Dave

timeout per transaction?

German Blanco — 2013-04-16T11:32:23

Hello,

According to the System Administrator's Guide:
syncTime -> Amount of time, in ticks (see
tickTime<http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#id_tickTime>),
to allow followers to sync with ZooKeeper. If followers fall too far behind
a leader, they will be dropped.

In order to check what is the time or number of transactions that a
follower may fall behind, I did some tests. I couldn't find a way to make a
transaction timeout in the Leader. It seems that the Leader adds
transactions to the list of outstanding proposals and they stay there until
a commit arrives or otherwise forever.

As long as Followers respond to the sync request, the quorum keeps running.

Is this really the behaviour? What is then the way in which it is
guaranteed that followers do not "fall too far behind a leader"?

Thanks in advance for your help.

Semantics of close and ephemeral nodes

Rohan Desai — 2013-04-11T21:29:08

What are the exact semantics of a close. Will it return only after the
zookeeper cluster has closed the session associated with the handle being
closed? Can I expect that after close() on the zookeeper handle returns
that any ephemeral nodes associated with that handle are gone? Or, is it
possible for the session to persist even after close returns?

Help，what's the exception of NIOServerCnxn - EndOfStreamException?

吴靖 — 2013-04-11T04:07:51

hi, everyone!
  here i use the storm to do topology, but some errors happend in the zookeeper, i think this error has no relationship  with the storm，
 here is the log:


[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN  org.apache.zookeeper.server.NIOServerCnxn  - EndOfStreamException: Unable to read additional data from client sessionid 0x13df73ff6620002, likely client has closed socket


the zookeeper is local mode，it works well in other applications。 thank you！

Is there a way to get the first 'n' children of a node?

Brian Tarbox — 2013-04-10T14:15:01

We are building up some quite large (100,000) lists of items under a node.
 Each item is small but there are a lot of them.

I need to process these items but certainly don't need to get the full list
at a time.

I don't see any way to get 'some' of the nodes under a node.   When I ask
for the full list zk fails with a "Packet len xzy is out of range!" error.
 I could work around this by adjusting the jute buffer size but that's a
stop gap answer at best.

Thanks.

Announcing KeptCollections 1.0

Anthony Urso — 2013-04-10T00:12:39

I am pleased to announce the 1.0 release of KeptCollections, a library of
drop-in replacements for standard Java Collections that use Apache
ZooKeeper as a backing store.

KeptCollections are designed to make it easy for anyone to write
distributed applications without having to learn the intricacies of
ZooKeeper, or distributed programming in general.

The collections use the well-known JDK APIs, yet any changes made to any of
these collections by one node are seen by all other nodes within
milliseconds, allowing for easy communication between processes in a
computing cluster.


More information here:

https://github.com/anthonyu/KeptCollections/wiki

and all code is available from:

https://github.com/anthonyu/KeptCollections

Please try it out, and let me know any problems you experience via github
issues or comments on this post.


<dependency>
  <groupId>net.killa.kept</groupId>
  <artifactId>KeptCollections</artifactId>
  <version>1.0.0</version>
</dependency>




Cheers,
Anthony

Zookeeper does not clean up deleted nodes

Mathias Hodler — 2013-04-08T10:32:54

Hi,

I made some tests and it seems like zookeeper doesn't clean up the last 500
deleted nodes.

In my test I created nodes and deleted each node after it was created. I
repeated this step 1000 times and then triggered a full gc. These are the
results

Creating 1000 Nodes and deleting 1000 Nodes and each node has...
...1000kb data = 529MB heap used after FullGC
...500kb data = 281MB heap used after FullGC
...256kb data = 140MB heap used after FullGC
...128kb data =  68MB heap used after FullGC

If I'm creating 1000 nodes with each 1000kb data and then deleting the
nodes and after that creating 1000 nodes with 128kb data and deleting the
nodes again, 68MB heap space is used.

So it seems Zookeeper caches / doesn't clean up the last 500 deleted nodes.

Is this a bug or are there configuration parameter to change that behaviour?

to subscribe

Hitesh Khamesra — 2013-04-04T21:46:11

Thanks,.
Hitesh

Simple Question

Nathan Findley — 2013-04-04T03:37:10

How can I overwrite a node in zookeeper?
How can I dump a node to the filesystem?

Regards,
Nate