gmane.comp.java.hadoop.zookeeper.user

New Apache ZooKeeper PMC Member: Michi Mutsuzaki

Flavio Junqueira — 2013-05-24T22:50:06

The Apache ZooKeeper PMC is pleased to announce that we have invited Michi to join the PMC and he has accepted.

Congratulations, Michi!

- The Apache ZooKeeper PMC

New Apache ZooKeeper committer: Thawan Kooburat

Flavio Junqueira — 2013-05-24T22:46:55

The Apache ZooKeeper PMC is pleased to announce that we have extended committer karma to Thawan and he has accepted.

Congratulations, Thawan!

- The Apache ZooKeeper PMC

ssl support in zookeeper

Даниел Симеонов — 2013-05-24T15:00:39

Hello,
   I saw that there is no ssl support in zookeeper for both
client-to-server connection as well as inter-server connections.
Is it possible to inject custom code which will handle at least the client
to server connection? As that netty is used to handle this connection?
When do you plan to introduce native ssl support in zookeeper?
I have one more question, what if a zookeeper client doesn't know the
addresses of all the zookeeper servers, will it learn the host names or ips
of the remaining servers from the ones it knows and are alive? What if a
zookeeper server changes its address, should all the zookeeper clients
change its knowledge of the server prior to this change?
Thank you very much!
Best regards, Daniel.

ssl support in zookeeper

Даниел Симеонов — 2013-05-24T15:04:22

Hello,
   I saw that there is no ssl support in zookeeper for both
client-to-server connection as well as inter-server connections.
Is it possible to inject custom code which will handle at least the client
to server connection? As that netty is used to handle this connection?
When do you plan to introduce native ssl support in zookeeper?
I have one more question, what if a zookeeper client doesn't know the
addresses of all the zookeeper servers, will it learn the host names or ips
of the remaining servers from the ones it knows and are alive? What if a
zookeeper server changes its address, should all the zookeeper clients
change its knowledge of the server prior to this change?
Thank you very much!
Best regards, Daniel.

Re: Binding to an specific IP address for election and broadcast ports

Vinayak Khot — 2013-05-24T09:50:11

You can configure these ports in the Zookeeper configuration file. You can
specify these
ports on the same line in the config file where you list the quoram servers.

eg.
server.1=127.0.0.1:18441:18451
server.2=127.0.0.1:18442:18452
server.3=127.0.0.1:18443:18453

Last two fields in each line above specifies leader election port and
communication port.

- Vinayak

On Fri, May 24, 2013 at 2:12 AM, German Blanco <
german.blanco.blanco-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

Re: Binding to an specific IP address for election and broadcast ports

Vinayak Khot — 2013-05-24T09:50:11

You can configure these ports in the Zookeeper configuration file. You can
specify these
ports on the same line in the config file where you list the quoram servers.

eg.
server.1=127.0.0.1:18441:18451
server.2=127.0.0.1:18442:18452
server.3=127.0.0.1:18443:18453

Last two fields in each line above specifies leader election port and
communication port.

- Vinayak

On Fri, May 24, 2013 at 2:12 AM, German Blanco <
german.blanco.blanco-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

Binding to an specific IP address for election and broadcast ports

German Blanco — 2013-05-24T09:12:50

Hello,

If I am not wrong, it is only possible to specify the specific address for
the port in which the clients connect to the ZooKeeper server, and not for
leader election and broadcast. Is that the case?
Would it make sense to allow the configuration of those two?

Regards,

Germán Blanco.

Re: Understanding event interleaving

Scott Fines — 2013-05-23T16:16:55

Thanks Flavio,Bahman, that was a great help. Turned out to be pretty straightforward, after all.

Scott

RE: Understanding event interleaving

Flavio Junqueira — 2013-05-23T14:04:38

Hi Scott,

The R client could receive either version 1 or version 2. In the case it receives version 1, it will receive the notification for the version 2 change once the server the client is connected to receives the update. If the getData returns version 2 instead, it won't get the watch triggered by the setData of steps 2 and 3 below. It should get it triggered by a future update, though, assuming there is one.

-Flavio

-----Original Message-----
From: Scott Fines [mailto:scottfines-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org] 
Sent: 23 May 2013 14:48
To: user-jljoq6hZe7LTxqt0kkDzDmD2FQJk+8+b< at >public.gmane.org
Subject: Understanding event interleaving

Hello all, 

This is perhaps a silly question, but one I'm not clear on.

Suppose you have two clients (R and W), and they are interacting through znode Z. Now consider the following situation:

0. Z at version 1
1. R begins getData(Z) (attach Watcher)
2. W begins setData(Z) 
3. W completed setData(Z) successfully --e.g. the client call returns with no errors

Re: Understanding event interleaving

Bahman Kalali — 2013-05-23T14:03:59

If I understand well your scenario, R will receive Version 2 and and event
is delivered to caller (R). In this scenario, the thread that has
registered the watch on node Z has to still be alive to receive the event.

BR//Bahman


On Thu, May 23, 2013 at 8:47 AM, Scott Fines <scottfines-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

Understanding event interleaving

Scott Fines — 2013-05-23T13:47:34

Hello all, 

This is perhaps a silly question, but one I'm not clear on.

Suppose you have two clients (R and W), and they are interacting through znode Z. Now consider the following situation:

0. Z at version 1
1. R begins getData(Z) (attach Watcher) 
2. W begins setData(Z)
3. W completed setData(Z) successfully --e.g. the client call returns with no errors
4. R completes getData(Z) successfully -- e.g. the client call returns with no errors

My questions are then:

1. What version of data will R receive? Version 1, or Version 2? My assumption here is that R would receive version 1 of the data--is that correct?

2. Does R's watch fire with the changes made by W? If so, is a WatchEvent delivered all the way to the caller? A comment made on ZOOKEEPER-1505 (https://issues.apache.org/jira/browse/ZOOKEEPER-1505) implies that it would not, but I'm unclear if that is in reference to the proposed changes or not.

Thanks for your help!

Scott Fines

cluster confused...would not delete node until cluster restarted...

Brian Tarbox — 2013-05-23T13:21:06

My 3 node cluster would not let me delete a node saying it was not
empty...but stat showed that it was in fact empty:

[zk: localhost:2181(CONNECTED) 1] delete /ROOT_A/INSTANCES/
10.244.43.240/WORKERS
*Node not empty:* /ROOT_A/INSTANCES/10.244.43.240/WORKERS

[zk: localhost:2181(CONNECTED) 0] stat /ROOT_A/INSTANCES/
10.244.43.240/WORKERS
cZxid = 0xe0015ad31
ctime = Wed May 22 17:20:52 EDT 2013
mZxid = 0xe0015ad31
mtime = Wed May 22 17:20:52 EDT 2013
pZxid = 0xe0015ae3c
cversion = 2
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 24
*numChildren = 0*
*
*
*
*
The debug log showed this on the machine initiating the delete:
2013-05-23 09:05:37,030 [myid:1] - DEBUG
[FollowerRequestProcessor:1:CommitProcessor< at >171] - Processing request::
sessionid:0x13ed17dc4310000 type:delete cxid:0x3 zxid:0xfffffffffffffffe
txntype:unknown reqpath:/ROOT_A/INSTANCES/10.244.43.240/WORKERS

2013-05-23 09:05:37,034 [myid:1] - DEBUG
[QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:CommitProcessor< at >161] - Committing
request:: s

Re: Just One Zookeeper Can Not Work at Quorum at Any Time

kamaci — 2013-05-22T21:21:28

OK, I found the problem. There was a misconfiguration at zookeeper. myid and
server.myid was not maching for some zookeeper servers.



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Just-One-Zookeeper-Can-Not-Work-at-Quorum-at-Any-Time-tp7578692p7578700.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

RE: Just One Zookeeper Can Not Work at Quorum at Any Time

Flavio Junqueira — 2013-05-22T20:06:09

I think we would need some more info from the logs to sort this one out. The
config file you're using might also help.

-Flavio

-----Original Message-----
From: kamaci [mailto:furkankamaci-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org] 
Sent: 18 May 2013 01:11
To: zookeeper-user-7ArZoLwFLBtd/SJB6HiN2Ni2O/JbrIOy< at >public.gmane.org
Subject: Just One Zookeeper Can Not Work at Quorum at Any Time

I have 3 zookeeper ensemble as a quorum at aws. I use them for my SolrCloud
nodes.

I have a weird problem about my Zookeeper instances. *When I start 3 of them
only 2 of them is working.* When I stop one of working Zookeeper and start
the broken one it starts successfully. 

However when I start the one that I have just stopped, this time it gives
error and can not start. It says:

Opening socket connection ... will not attempt to authenticate using SASL
(unknown error) Socket connection established...

Unable to read additional data from server sessionid 0x0, likely server has
closed socket, closing socket connection and att

Just One Zookeeper Can Not Work at Quorum at Any Time

kamaci — 2013-05-18T00:10:52

I have 3 zookeeper ensemble as a quorum at aws. I use them for my SolrCloud
nodes.

I have a weird problem about my Zookeeper instances. *When I start 3 of them
only 2 of them is working.* When I stop one of working Zookeeper and start
the broken one it starts successfully. 

However when I start the one that I have just stopped, this time it gives
error and can not start. It says:

Opening socket connection ... will not attempt to authenticate using SASL
(unknown error)
Socket connection established...

Unable to read additional data from server sessionid 0x0, likely server has
closed socket, closing socket connection and attempting reconnect

When I run ./zkCli.sh it says connecting and when I do something like 

ls / 

it throws Connection loss for / and throws Keeper Exception.

Any ideas?




--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Just-One-Zookeeper-Can-Not-Work-at-Quorum-at-Any-Time-tp7578692.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Just One Zookeeper Can Not Work at Quorum at Any Time

kamaci — 2013-05-18T21:06:45

I have 3 zookeeper ensemble as a quorum at aws. I use them for my SolrCloud
nodes.

I have a weird problem about my Zookeeper instances. *When I start 3 of them
only 2 of them is working.* When I stop one of working Zookeeper and start
the broken one it starts successfully. 

However when I start the one that I have just stopped, this time it gives
error and can not start. It says:

Opening socket connection ... will not attempt to authenticate using SASL
(unknown error)
Socket connection established...

Unable to read additional data from server sessionid 0x0, likely server has
closed socket, closing socket connection and attempting reconnect

When I run ./zkCli.sh it says connecting and when I do something like 

ls / 

it throws Connection loss for / and throws Keeper Exception.

Any ideas?




--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Just-One-Zookeeper-Can-Not-Work-at-Quorum-at-Any-Time-tp7578693.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

RE: simple zookeeper client in C

Pramod Srinivasan — 2013-05-22T01:25:36

Just realized, I was linking to pthread library and my test program was written for single threaded application, this was causing the problem.
________________________________________
From: Pramod Srinivasan
Sent: Tuesday, May 21, 2013 4:09 PM
To: user-jljoq6hZe7LTxqt0kkDzDmD2FQJk+8+b< at >public.gmane.org
Subject: simple zookeeper client in C

Hi ,

I am trying out a simple c client for zookeeper, made a copy of zookeeper-3.4.5/src/c/src/cli.c just to understand how the c APIs works, the program is attached.

I see errors/coredumps being generated, I am surely doing something wrong, any help appreciated.

case 1:

.../zookeeper-3.4.5/src/c> ./myt localhost:2181
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
Segmentation fault (core dumped)

starting it under gdb:
(gdb) r localhost:2181
Starting program: .../zookeeper-3.4.5/src/c/myt localhost:2181
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-lin

simple zookeeper client in C

Pramod Srinivasan — 2013-05-21T23:09:39

Hi ,

I am trying out a simple c client for zookeeper, made a copy of zookeeper-3.4.5/src/c/src/cli.c just to understand how the c APIs works, the program is attached.

I see errors/coredumps being generated, I am surely doing something wrong, any help appreciated.

case 1:

.../zookeeper-3.4.5/src/c> ./myt localhost:2181
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
Segmentation fault (core dumped)

starting it under gdb:
(gdb) r localhost:2181
Starting program: .../zookeeper-3.4.5/src/c/myt localhost:2181
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff70d9700 (LWP 9489)]
[New Thread 0x7ffff68d8700 (LWP 9490)]
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
Watcher SESSION_EVENT state = CONNECTED_STATE
Connected ...
2013-05-21 15:58:03,210:9485(0x7ffff7fd7700):ZOO_ERROR< at >handle_socket_error_msg< at >1643: Socket [127.0.0.1:2181] zk retco

RE: Transaction log: /logs/jhe/nc/apps/zookeeper/version-2/log.1400470027 has invalid magic number 0 != 1514884167

Dilip Kumar — 2013-05-21T14:52:21

Hi Suman,

This should be related to hostname. You have used below

server.1=host1:3888:4888
server.2=host2:3888:4888
server.3=host3:3888:4888

try to use actual hostname (or IP address preferable). I am using following in my Zookeeper and it is working fine.

server.1=10.20.5.129:2888:3888
server.2=10.20.5.194:2888:3888
server.3=10.20.5.195:2888:3888

Thanks,
Dilip Kumar
3K50, One Lake Street (Pearson Education)
Upper Saddle River, NJ- 07458
Office# 201-236-7163
Cell#201-515-0606
dilip.kumar-KnNNNkqOsPQ< at >public.gmane.org

-----Original Message-----
From: suman p [mailto:sumanp0309-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org] 
Sent: Tuesday, May 21, 2013 9:04 AM
To: user-jljoq6hZe7LTxqt0kkDzDmD2FQJk+8+b< at >public.gmane.org
Subject: Transaction log: /logs/jhe/nc/apps/zookeeper/version-2/log.1400470027 has invalid magic number 0 != 1514884167

Hello,

 We are running zookeeper ensemble of 3 servers.


Today one of our sever died. When i restarted the sever i am getting the following error


java.io.IOException: Transa

Transaction log: /logs/jhe/nc/apps/zookeeper/version-2/log.1400470027 has invalid magic number 0 != 1514884167

suman p — 2013-05-21T13:03:47

Hello,

 We are running zookeeper ensemble of 3 servers.


Today one of our sever died. When i restarted the sever i am getting the
following error


java.io.IOException: Transaction log:
/logs/jhe/nc/apps/zookeeper/version-2/log.1400470027 has invalid magic
number 0 != 1514884167
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:560)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:577)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:543)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.init(FileTxnLog.java:525)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.<init>(FileTxnLog.java:504)
        at
org.apache.zookeeper.server.persistence.FileTxnLog.read(FileTxnLog.java:341)
        at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:132)

zookeeper

suman p — 2013-05-21T12:46:03