http://blog.gmane.org/gmane.comp.web.cache.memcached hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://comments.gmane.org/gmane.comp.web.cache.memcached/14252 <pre>Hello, My name is Yuval Meir and I am a M.Sc student in Tel Aviv University. As part of my academic research I have implemented the ARC (Adaptive Replacement Cache) caching policy in Memcached I ran benchmarks using a modified version of memaslap and the results were very good: cache misses were reduced by up to to 30% on some loads, and in terms of run time, memory usage and CPU usage, my implementation performed similar to the original Memcached. I attached the diff (based on Memcached 1.4.13) and the results I have so far in the excel file (note that it has several tabs). ARC was introduced in the following publication by Nimrod Megiddo, Dharmendra Modha: http://www.almaden.ibm.com/cs/people/dmodha/arcfast.pdf which was published in USENIX 2003. I will be very glad to get you reviewes and comments. Regards, Yuval Meir Computer Science department Tel-Aviv University </pre> Yuval M 2012-05-24T19:12:33 http://comments.gmane.org/gmane.comp.web.cache.memcached/14250 <pre>I revised memcached1.4.13 to make cas ID unique between server reboots. But It seemed that two memcached instances on a server has eaten up almost all memory under stressing load. 1. OS: centerOS5.3, 64bit 2. memcached startup command: /opt/memcached1.4.13.0503b/bin/memcached -u nobody -d -m 4096 -c 1024 - l 192.168.8.224 -p 11211 -v &gt; /var/log/memcached-11211.log 2&gt;&amp;1 /opt/memcached1.4.13.0503b/bin/memcached -u nobody -d -m 4096 -c 1024 - l 192.168.8.224 -p 11211 -v &gt; /var/log/memcached-11211.log 2&gt;&amp;1 3. Memory usage: [root&lt; at &gt;dell1-12 ~]# free -m total used free shared buffers cached Mem: 16044 15986 58 0 85 24 -/+ buffers/cache: 15876 168 Swap: 7820 6911 908 Stats on one instance: stats STAT pid 32205 STAT uptime 60090 STAT time 1336177937 STAT version 1.4.13.0503b STAT libevent 1.4.12-stable STAT pointer_size 64 STAT rusage_user 1125.852844 STAT rusage_system 2174.782382 STAT curr_connections 10 STAT total_connections 18 STAT connection_structures 11 STAT reserved_fds 20 STAT cmd_get 249567553 STAT cmd_set 2698828 STAT cmd_flush 0 STAT cmd_touch 0 STAT get_hits 241304080 STAT get_misses 8263473 STAT delete_misses 1249006 STAT delete_hits 1236936 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT touch_hits 0 STAT touch_misses 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 20934057174 STAT bytes_written 32731347667 STAT limit_maxbytes 4294967296 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 77 STAT hash_power_level 20 STAT hash_bytes 8388608 STAT hash_is_expanding 0 STAT expired_unfetched 488121 STAT evicted_unfetched 0 STAT bytes 2909361795 STAT curr_items 803791 STAT total_items 2698828 STAT evictions 0 STAT reclaimed 492946 END I am not sure whether my change caused this fatal issue. This only happens when a large amount of objects are put into memcached. Here is major change code: uint64_t get_cas_id(void) { struct timeval tm; gettimeofday(&amp;tm, NULL); return (uint64_t)tm.tv_sec &lt;&lt; 32 | (uint64_t)tm.tv_usec; } I can put this customized version somewhere if needed. Hope someone can help me. Thanks in advance. </pre> Smiling Ai 2012-05-05T02:36:18 http://comments.gmane.org/gmane.comp.web.cache.memcached/14249 <pre>Hi! Added support for setting options via ENV variable LIBMEMCACHED. Fix corner case on last used result. Bug fixes. http://launchpad.net/libmemcached Cheers, -Brian</pre> Brian Aker 2012-05-23T07:42:50 http://comments.gmane.org/gmane.comp.web.cache.memcached/14243 <pre>Status: New Owner: ---- Labels: Type-Defect Priority-Medium New issue 274 by vlak...&lt; at &gt;gmail.com: typo in memcached wiki http://code.google.com/p/memcached/issues/detail?id=274 http://code.google.com/p/memcached/wiki/NewInstallFromPackage s/rececnt/recent </pre> memcached< at >googlecode.com 2012-05-20T17:31:39 http://comments.gmane.org/gmane.comp.web.cache.memcached/14241 <pre>We are currently having an issue with our memcache server that appears to be flushing data even though nothing in the apps are flushing data. I can set something with a TTL of 2 weeks yet it is only lasting in memcache for a max of something around 30 minutes. Print out of getstats in one of the php applications shows the following: Array ( [pid] =&gt; 24793 [uptime] =&gt; 373575 [time] =&gt; 1336770386 [version] =&gt; 1.4.2 [pointer_size] =&gt; 64 [rusage_user] =&gt; 4077.050000 [rusage_system] =&gt; 12475.330000 [curr_connections] =&gt; 1143 [total_connections] =&gt; 89011605 [connection_structures] =&gt; 1616 [cmd_get] =&gt; 438522590 [cmd_set] =&gt; 101189934 [cmd_flush] =&gt; 863 [get_hits] =&gt; 435589695 [get_misses] =&gt; 188087908 [delete_misses] =&gt; 27221748 [delete_hits] =&gt; 610097 [incr_misses] =&gt; 0 [incr_hits] =&gt; 0 [decr_misses] =&gt; 0 [decr_hits] =&gt; 0 [cas_misses] =&gt; 0 [cas_hits] =&gt; 0 [cas_badval] =&gt; 0 [bytes_read] =&gt; 71199529191 [bytes_written] =&gt; 955401557632 [limit_maxbytes] =&gt; 10737418240 [accepting_conns] =&gt; 1 [listen_disabled_num] =&gt; 0 [threads] =&gt; 4 [conn_yields] =&gt; 157 [bytes] =&gt; 6796086722 [curr_items] =&gt; 12613469 [total_items] =&gt; 100944174 [evictions] =&gt; 0 ) There are no evictions so the data isnt filling up (we have set a 10GB max and there is currently 6GB of data currently). However flushes are rising and are at 863 as of this post. Is there something going on that we are not seeing, or a setting somewhere? Any help would be appreciated. </pre> Mentch 2012-05-11T21:31:16 http://comments.gmane.org/gmane.comp.web.cache.memcached/14230 <pre>When we are under high load (10k+ requests per second) we start getting communication failures (connection timouts,closed connection during header) to memcached. After a small period of time in this state we start getting mangled returns from the cache. Some returns will contain two pages data, some will simply be the wrong entry. We can recover from this by resetting the memcache servers. Is there some connection concurrency limit we may be bumping into under high load that would cause communication channel mismatches, such that request 1 will somehow receive request 2's answer? We are using memcached 1.4.11 with the following config on x86_64 -d -p 11211 -u memcached -m 1024 -c 1024 </pre> Greg Grensteiner 2012-05-10T20:27:35 http://comments.gmane.org/gmane.comp.web.cache.memcached/14229 <pre>hi , I write a client of memcached myselves,I string the command like "get key1\r\nget key2\r\nget key3\r\nget key4\r\n.......get key500\r \n",when I send it to the memcached server by socket,I get only 460~480 values or less.I doubt it's the bug of the libevent! who can tell me. </pre> casa zhang 2012-05-10T11:06:35 http://comments.gmane.org/gmane.comp.web.cache.memcached/14224 <pre>Status: New Owner: ---- Labels: Type-Defect Priority-Medium New issue 273 by DanseL...&lt; at &gt;gmail.com: Compilation issue on v2.0.19 http://code.google.com/p/memcached/issues/detail?id=273 Hello everyone, Here is my little problem about compilation What steps will reproduce the problem? 1. ./configure 2. make &amp;&amp; make test What is the expected output? What do you see instead? The result is as follows: $ make &amp;&amp; make test make all-recursive make[1]: ディレクトリ `/cygdrive/c/Documents and Settings/ksksol/My Documents/libevent-2.0.19-stable' に入ります Making all in . make[2]: ディレクトリ `/cygdrive/c/Documents and Settings/ksksol/My Documents/libevent-2.0.19-stable' に入ります make[2]: ディレクトリ `/cygdrive/c/Documents and Settings/ksksol/My Documents/libevent-2.0.19-stable' から出ます Making all in include make[2]: ディレクトリ `/cygdrive/c/Documents and Settings/ksksol/My Documents/libevent-2.0.19-stable/include' に入ります make[2]: `all' に対して行うべき事はありません. make[2]: ディレクトリ `/cygdrive/c/Documents and Settings/ksksol/My Documents/libevent-2.0.19-stable/include' から出ます Making all in sample make[2]: ディレクトリ `/cygdrive/c/Documents and Settings/ksksol/My Documents/libevent-2.0.19-stable/sample' に入ります /bin/sh ../libtool --tag=CC --mode=link gcc -g -O2 -Wall -fno-strict-aliasing -o le-proxy.exe le-proxy.o ../libevent.la ../libevent_openssl.la -lssl -lcrypto libtool: link: cannot find the library `Documents/libevent-2.0.19-stable/libevent_core.la' or unhandled argument `Documents/libevent-2.0.19-stable/libevent_core.la' Makefile:308: recipe for target `le-proxy.exe' failed make[2]: *** [le-proxy.exe] Error 1 make[2]: ディレクトリ `/cygdrive/c/Documents and Settings/ksksol/My Documents/libevent-2.0.19-stable/sample' から出ます Makefile:821: recipe for target `all-recursive' failed make[1]: *** [all-recursive] Error 1 make[1]: ディレクトリ `/cygdrive/c/Documents and Settings/ksksol/My Documents/libevent-2.0.19-stable' から出ます Makefile:558: recipe for target `all' failed make: *** [all] Error 2 What version of the product are you using? On what operating system? I am using the version 2.0.19, on windows 7, trying to compile with cygwin </pre> memcached< at >googlecode.com 2012-05-09T07:09:56 http://comments.gmane.org/gmane.comp.web.cache.memcached/14216 <pre>Status: New Owner: ---- Labels: Type-Defect Priority-Medium New issue 272 by keyurgov...&lt; at &gt;gmail.com: Memcached "hung" in do_item_alloc http://code.google.com/p/memcached/issues/detail?id=272 What is the problem? One night at Etsy, we started getting a flood of memcached errors. We were seeing out-of-memory errors when trying to set/add any value into slab #4. The slab held data of sizes upto 10 bytes. I did some low-level checking around what happens in do_item_alloc() because we hadn't built debug-symbols :( Details here: https://gist.github.com/2634998 Starting with search = tails[id]; in line 106. The code's here: https://github.com/memcached/memcached/blob/master/items.c The search item has a starting ref count of 2. So calling refcount_incr on it makes it 3 and it entirely skips the block of code between 107 and 152 (that block handles expired items and forcible eviction). In the else block in line 154, since the slab is full, slabs_alloc returns NULL. In line 156, the refcount is decremented back to 2. Now in the if block starting in line 167, the refcount is still 2, so it never forcibly expires the item. And this stays like this forever. This check was changed from a refcount !=0 to refcount !=2 recently (https://github.com/memcached/memcached/commit/f4983b2068d13e5dc71fc075c35a085e904999cf#items.c) When this happens the "leaked" tails item forever blocks any new item from being inserted into the list and we can't do any more operations on the cluster in slab #4. The only way we could fix it was to restart memcached. What steps will reproduce the problem? We don't exactly know how all 7 of our memcached boxes got into this state one night. We still get occasional blips of these errors. What is the expected output? What do you see instead? Set/add should work. What version of the product are you using? On what operating system? memcached 1.4.13 on Cent5 Please provide any additional information below. In our testing environment, I haven't been able to reproduce the issue, but then the volume I could generate definitely couldn't match up to Production. </pre> memcached< at >googlecode.com 2012-05-08T13:37:56 http://comments.gmane.org/gmane.comp.web.cache.memcached/14214 <pre>Hi all, I want make a slight extension to memcached server, and I have several new source files to be added to the build system. What files shall I modify? How to change? Thanks! -sharc </pre> sharc 2012-05-08T04:48:21 http://comments.gmane.org/gmane.comp.web.cache.memcached/14212 <pre>Status: New Owner: ---- Labels: Type-Defect Priority-Medium New issue 271 by zhega...&lt; at &gt;gmail.com: Spontaneous quits on Solaris http://code.google.com/p/memcached/issues/detail?id=271 What steps will reproduce the problem? 1. Get a memcached 2. Build it on Solaris 10 3. Run it. What is the expected output? What do you see instead? Expected output is none. I see instead something like this: rcurr=30eb2b0 ritem=12740d1f rbuf=30e9c10 rlbytes=1070 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=10acb12c ritem=11eea05b rbuf=10aca010 rlbytes=2467 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=10cc6eb0 ritem=11f77c3f rbuf=10cc5810 rlbytes=1403 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=10e08d08 ritem=11ed6ef7 rbuf=10e07c10 rlbytes=2513 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=20b5db7 ritem=11f77746 rbuf=20b4c10 rlbytes=2656 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=10c57508 ritem=11f77696 rbuf=10c56410 rlbytes=3126 rsize=8192 [warn] port_getn: Error 0 (and here the output ends, memcached quits. ETA is about 18 hours from the start) What version of the product are you using? On what operating system? Solaris 10 x86, running in x64 mode. libevent 2.0.18-stable memcached-1.4.13 Please provide any additional information below. memcached is built with gcc, the version is: (GCC) 3.4.3 (csl-sol210-3_4-branch+sol_rpath) Though it's old, this is the main version on Solaris 10. The average load is (according to cacti): - about 200-300 requests per second - about 396K objects in cache - about 200 simultaneous connections - 10-12 megabits/sec traffic from memcached </pre> memcached< at >googlecode.com 2012-05-04T03:41:39 http://comments.gmane.org/gmane.comp.web.cache.memcached/14208 <pre>Hi, I want to know if there is a way to alter the default cache eviction policy used by memcached(LRU). I want to implement an LFU cache(or an SDC Cache replacement policy) and wondering if I can use memcached or not? Regards, Arghyadip </pre> Arghyadip Paul 2012-05-03T07:23:26 http://comments.gmane.org/gmane.comp.web.cache.memcached/14206 <pre>Hi. I have 1.4.13 on Solaris x86_64. Sometimes it just quits. To inversigate this, I launched it with -v in a screen. It works for now, but it constantly shows some errors: ===Cut=== rcurr=283e52c ritem=1afc85b rbuf=283d410 rlbytes=2422 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=29b9508 ritem=1ab5676 rbuf=29b8410 rlbytes=2152 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=20b1508 ritem=1adf076 rbuf=20b0410 rlbytes=2246 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=2896eb0 ritem=1afcddf rbuf=2895810 rlbytes=797 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=261a508 ritem=1aece77 rbuf=2619410 rlbytes=2669 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=29cb548 ritem=1596f28 rbuf=29ca410 rlbytes=345 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=29b4d08 ritem=1afe3f6 rbuf=29b3c10 rlbytes=2246 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=29c96b0 ritem=1afcddf rbuf=29c8010 rlbytes=797 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=29cbab0 ritem=8e44db rbuf=29ca410 rlbytes=88305 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=263ed08 ritem=1afe3f6 rbuf=263dc10 rlbytes=2246 rsize=8192 Failed to read, and not due to blocking: errno: 0 Error 0 rcurr=29d02b0 ritem=1afcddf rbuf=29cec10 rlbytes=786 rsize=8192 ===Cut=== I think these errors are somehow related to it's spontaneous quits. How can this be solved ? Thanks. Eugene. </pre> Eugene M. Zheganin 2012-05-03T04:52:03 http://comments.gmane.org/gmane.comp.web.cache.memcached/14205 <pre>Hi All i have to manage 3 servers with a rails application, i would like to put only one memcached daemon on one of these servers. this cache must be shared from the servers. 2 servers must connect remotly, 1 locally do you see any problem on this solution if i set the same domain in each rails app? Any suggestion? Thanks Mauro Giannandrea </pre> Mauro Giannandrea 2012-05-02T21:16:16 http://comments.gmane.org/gmane.comp.web.cache.memcached/14201 <pre>Hi! Changelog: * Add API call for exist calls. * Update all license files to be BSD. http://launchpad.net/libmemcached Cheers, -Brian</pre> Brian Aker 2012-04-29T03:16:27 http://comments.gmane.org/gmane.comp.web.cache.memcached/14196 <pre>*There are only two hard things in Computer Science: cache invalidation and naming things. </pre> Jakub Łopuszański 2012-04-27T22:14:49 http://comments.gmane.org/gmane.comp.web.cache.memcached/14194 <pre>Hi, will using append() on an item put it to the top of the LRU as an add() does? Cheers Martin </pre> Martin Hellmich 2012-04-27T08:10:49 http://comments.gmane.org/gmane.comp.web.cache.memcached/14178 <pre>Hi all, I compiled and installed Memcached 1.4.3 from source: http://memcached.googlecode.com/files/memcached-1.4.13.tar.gz The configuration is quite simple: ./configure --prefix=/usr/local/ memcached It works if I install it on a dedicated Ubuntu 11 server. However, it does not work on Linode's Ubuntu 11 VM (Xen-based) The source code can be compilable, installable. The server can run but it is not responsive. === Telnet access ====== root&lt; at &gt;spica:/home/dev7# telnet 127.0.0.1 11211 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. get fdsgfdsf f sdfds f dsadfs ] ^] telnet&gt; quit Connection closed. ============================ The strace output shows that it is running ===== root&lt; at &gt;spica:/sample-app/# ps -eaf | grep mem 1003 17289 1 0 14:49 ? 00:00:00 /usr/local/memcached/ bin/memcached -d -l 127.0.0.1 -p 11211 -u myserver root 17317 29816 0 14:51 pts/1 00:00:00 grep --color=auto mem root&lt; at &gt;spica:/sample-app# strace -p 17289 Process 17289 attached - interrupt to quit clock_gettime(CLOCK_MONOTONIC, {100924, 469837720}) = 0 gettimeofday({1335365516, 645872}, NULL) = 0 epoll_wait(3, {}, 32, 130) = 0 clock_gettime(CLOCK_MONOTONIC, {100924, 600565355}) = 0 gettimeofday({1335365516, 776498}, NULL) = 0 clock_gettime(CLOCK_MONOTONIC, {100924, 600798581}) = 0 epoll_wait(3, {}, 32, 1000) = 0 clock_gettime(CLOCK_MONOTONIC, {100925, 602196472}) = 0 gettimeofday({1335365517, 778115}, NULL) = 0 clock_gettime(CLOCK_MONOTONIC, {100925, 602418650}) = 0 epoll_wait(3, {}, 32, 1000) = 0 clock_gettime(CLOCK_MONOTONIC, {100926, 603838881}) = 0 gettimeofday({1335365518, 779837}, NULL) = 0 clock_gettime(CLOCK_MONOTONIC, {100926, 604232123}) = 0 epoll_wait(3, {}, 32, 1000) = 0 clock_gettime(CLOCK_MONOTONIC, {100927, 605720227}) = 0 gettimeofday({1335365519, 781684}, NULL) = 0 clock_gettime(CLOCK_MONOTONIC, {100927, 606013867}) = 0 epoll_wait(3, {}, 32, 1000) = 0 clock_gettime(CLOCK_MONOTONIC, {100928, 607465458}) = 0 gettimeofday({1335365520, 783409}, NULL) = 0 clock_gettime(CLOCK_MONOTONIC, {100928, 607718097}) = 0 epoll_wait(3, {}, 32, 1000) = 0 clock_gettime(CLOCK_MONOTONIC, {100929, 609098816}) = 0 gettimeofday({1335365521, 785043}, NULL) = 0 clock_gettime(CLOCK_MONOTONIC, {100929, 609365664}) = 0 epoll_wait(3, {}, 32, 1000) = 0 clock_gettime(CLOCK_MONOTONIC, {100930, 610825545}) = 0 gettimeofday({1335365522, 786793}, NULL) = 0 clock_gettime(CLOCK_MONOTONIC, {100930, 611180293}) = 0 epoll_wait(3, ^C &lt;unfinished ...&gt; Process 17289 detached ======================= Has anyone experienced this issue? I don't think that I have done anything wrong. Because it works on my dedicated server already. However, when I installed it on a Xen Linux VM provided by Linode, it did not work any more. Is there any special thing that I need to know about Xen-based virtual machine, kernel or Linode VM? Here is some information about our Linode system in which memcached does not work root&lt; at &gt;spica:/home/dev7# lsb_release -a No LSB modules are available. Distributor ID:Ubuntu Description:Ubuntu 11.10 Release:11.10 Codename:oneiric root&lt; at &gt;spica:/home/dev7# free -m total used free shared buffers cached Mem: 488 424 63 0 68 249 -/+ buffers/cache: 107 381 Swap: 511 1 510 Here is some information about my dedicated server (Memcached works here) pcdinh&lt; at &gt;myserver:~$ lsb_release -a No LSB modules are available. Distributor ID:Ubuntu Description:Ubuntu 11.10 Release:11.10 Codename:oneiric pcdinh&lt; at &gt;myserver:~$ free -m total used free shared buffers cached Mem: 2502 2408 93 0 350 1466 -/+ buffers/cache: 591 1910 Swap: 2955 0 2955 Thanks Dinh </pre> pcdinh 2012-04-25T15:32:55 http://comments.gmane.org/gmane.comp.web.cache.memcached/14177 <pre>the memcached main process can't accept connect when use assoc_maintenance_thread function,here is the gdb bt result: we have 4 thread : |-memcached,28059 -d -p 12111 -c 8192 -m 16384 -u root -l 127.0.0.1 | |-{memcached},28060 | |-{memcached},28061 | |-{memcached},28062 | |-{memcached},28063 | `-{memcached},28064 gdb bt result: gdb -p 28060 GNU gdb Fedora (6.8-37.el5) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later &lt;http://gnu.org/licenses/ gpl.html&gt; This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". Attaching to process 28060 warning: process 28060 is a cloned process Reading symbols from /usr/local/sinawap/bin/memcached...done. Reading symbols from /usr/local/sinawap/lib/libevent-1.4.so.2...done. Loaded symbols for /usr/local/sinawap/lib/libevent-1.4.so.2 Reading symbols from /lib64/libpthread.so.0...done. [Thread debugging using libthread_db enabled] [New Thread 0x7f2ec05b86e0 (LWP 28059)] [New Thread 0x434f2940 (LWP 28064)] [New Thread 0x42cf1940 (LWP 28063)] [New Thread 0x424f0940 (LWP 28062)] [New Thread 0x41cef940 (LWP 28061)] [New Thread 0x414ee940 (LWP 28060)] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libc.so.6...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/libnsl.so.1...done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/librt.so.1...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libresolv.so.2...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libnss_files.so.2...done. Loaded symbols for /lib64/libnss_files.so.2 0x0000003095a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0 (gdb) bt #0 0x0000003095a0d524 in __lll_lock_wait () from /lib64/libpthread.so. 0 #1 0x0000003095a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0 #2 0x0000003095a08cdc in pthread_mutex_lock () from /lib64/ libpthread.so.0 #3 0x000000000040d08e in item_get (key=0x7f2cf8ac1424 "ttt_newuser_2272938860", nkey=128) at thread.c:343 #4 0x0000000000403c14 in process_get_command (c=0x7f2d3888f080, tokens=0x414eded0, ntokens=0, return_cas=false) at memcached.c:2542 #5 0x0000000000408256 in process_command (c=0x7f2d3888f080, command=&lt;value optimized out&gt;) at memcached.c:2975 #6 0x0000000000408bfe in try_read_command (c=0x7f2d3888f080) at memcached.c:3185 #7 0x000000000040989d in event_handler (fd=&lt;value optimized out&gt;, which=128, arg=0x7f2d3888f080) at memcached.c:3491 #8 0x00007f2ec05c25f8 in event_base_loop (base=0x63f420, flags=0) at event.c:392 #9 0x000000000040cbb4 in worker_libevent (arg=0x633f70) at thread.c: 245 #10 0x0000003095a0673d in start_thread () from /lib64/libpthread.so.0 #11 0x00000030952d44bd in clone () from /lib64/libc.so.6 gdb -p 28061 GNU gdb Fedora (6.8-37.el5) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later &lt;http://gnu.org/licenses/ gpl.html&gt; This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". Attaching to process 28061 warning: process 28061 is a cloned process Reading symbols from /usr/local/sinawap/bin/memcached...done. Reading symbols from /usr/local/sinawap/lib/libevent-1.4.so.2...done. Loaded symbols for /usr/local/sinawap/lib/libevent-1.4.so.2 Reading symbols from /lib64/libpthread.so.0...done. [Thread debugging using libthread_db enabled] [New Thread 0x7f2ec05b86e0 (LWP 28059)] [New Thread 0x434f2940 (LWP 28064)] [New Thread 0x42cf1940 (LWP 28063)] [New Thread 0x424f0940 (LWP 28062)] [New Thread 0x41cef940 (LWP 28061)] [New Thread 0x414ee940 (LWP 28060)] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libc.so.6...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/libnsl.so.1...done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/librt.so.1...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libresolv.so.2...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libnss_files.so.2...done. Loaded symbols for /lib64/libnss_files.so.2 0x0000003095a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0 (gdb) bt #0 0x0000003095a0d524 in __lll_lock_wait () from /lib64/libpthread.so. 0 #1 0x0000003095a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0 #2 0x0000003095a08cdc in pthread_mutex_lock () from /lib64/ libpthread.so.0 #3 0x000000000040d0f6 in item_alloc (key=0x7f2cd3a2a0a4 "ttt_api_unread_2557924625", nkey=128, flags=0, exptime=4294967295, nbytes=6395456) at thread.c:331 #4 0x0000000000407fa7 in process_update_command (c=0x7f2ebc16e900, tokens=&lt;value optimized out&gt;, ntokens=&lt;value optimized out&gt;, comm=2, handle_cas=false) at memcached.c:2728 #5 0x00000000004083be in process_command (c=0x7f2ebc16e900, command=&lt;value optimized out&gt;) at memcached.c:2984 #6 0x0000000000408bfe in try_read_command (c=0x7f2ebc16e900) at memcached.c:3185 #7 0x000000000040989d in event_handler (fd=&lt;value optimized out&gt;, which=128, arg=0x7f2ebc16e900) at memcached.c:3491 #8 0x00007f2ec05c25f8 in event_base_loop (base=0x658130, flags=0) at event.c:392 #9 0x000000000040cbb4 in worker_libevent (arg=0x636c98) at thread.c: 245 #10 0x0000003095a0673d in start_thread () from /lib64/libpthread.so.0 #11 0x00000030952d44bd in clone () from /lib64/libc.so.6 gdb -p 28062 GNU gdb Fedora (6.8-37.el5) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later &lt;http://gnu.org/licenses/ gpl.html&gt; This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". Attaching to process 28062 warning: process 28062 is a cloned process Reading symbols from /usr/local/sinawap/bin/memcached...done. Reading symbols from /usr/local/sinawap/lib/libevent-1.4.so.2...done. Loaded symbols for /usr/local/sinawap/lib/libevent-1.4.so.2 Reading symbols from /lib64/libpthread.so.0...done. [Thread debugging using libthread_db enabled] [New Thread 0x7f2ec05b86e0 (LWP 28059)] [New Thread 0x434f2940 (LWP 28064)] [New Thread 0x42cf1940 (LWP 28063)] [New Thread 0x424f0940 (LWP 28062)] [New Thread 0x41cef940 (LWP 28061)] [New Thread 0x414ee940 (LWP 28060)] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libc.so.6...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/libnsl.so.1...done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/librt.so.1...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libresolv.so.2...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libnss_files.so.2...done. Loaded symbols for /lib64/libnss_files.so.2 0x0000003095a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0 (gdb) bt #0 0x0000003095a0d524 in __lll_lock_wait () from /lib64/libpthread.so. 0 #1 0x0000003095a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0 #2 0x0000003095a08cdc in pthread_mutex_lock () from /lib64/ libpthread.so.0 #3 0x000000000040d08e in item_get (key=0x7f2cdedceb84 "iphone_2283873065_2012042520", nkey=128) at thread.c:343 #4 0x0000000000403c14 in process_get_command (c=0x7f2d3861ff20, tokens=0x424efed0, ntokens=0, return_cas=false) at memcached.c:2542 #5 0x0000000000408256 in process_command (c=0x7f2d3861ff20, command=&lt;value optimized out&gt;) at memcached.c:2975 #6 0x0000000000408bfe in try_read_command (c=0x7f2d3861ff20) at memcached.c:3185 #7 0x000000000040989d in event_handler (fd=&lt;value optimized out&gt;, which=128, arg=0x7f2d3861ff20) at memcached.c:3491 #8 0x00007f2ec05c25f8 in event_base_loop (base=0x690ed0, flags=0) at event.c:392 #9 0x000000000040cbb4 in worker_libevent (arg=0x6399c0) at thread.c: 245 #10 0x0000003095a0673d in start_thread () from /lib64/libpthread.so.0 #11 0x00000030952d44bd in clone () from /lib64/libc.so.6 gdb -p 28063 GNU gdb Fedora (6.8-37.el5) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later &lt;http://gnu.org/licenses/ gpl.html&gt; This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". Attaching to process 28063 warning: process 28063 is a cloned process Reading symbols from /usr/local/sinawap/bin/memcached...done. Reading symbols from /usr/local/sinawap/lib/libevent-1.4.so.2...done. Loaded symbols for /usr/local/sinawap/lib/libevent-1.4.so.2 Reading symbols from /lib64/libpthread.so.0...done. [Thread debugging using libthread_db enabled] [New Thread 0x7f2ec05b86e0 (LWP 28059)] [New Thread 0x434f2940 (LWP 28064)] [New Thread 0x42cf1940 (LWP 28063)] [New Thread 0x424f0940 (LWP 28062)] [New Thread 0x41cef940 (LWP 28061)] [New Thread 0x414ee940 (LWP 28060)] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libc.so.6...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/libnsl.so.1...done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/librt.so.1...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libresolv.so.2...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libnss_files.so.2...done. Loaded symbols for /lib64/libnss_files.so.2 0x000000000040c2d1 in assoc_find (key=0x7f2d231264f4 "TC_db784e309e27b22873f62c3329d032c9", nkey=35) at assoc.c:85 85 if ((nkey == it-&gt;nkey) &amp;&amp; (memcmp(key, ITEM_key(it), nkey) == 0)) { (gdb) bt #0 0x000000000040c2d1 in assoc_find (key=0x7f2d231264f4 "TC_db784e309e27b22873f62c3329d032c9", nkey=35) at assoc.c:85 #1 0x000000000040ba1e in do_item_get (key=0xbef4bc56 &lt;Address 0xbef4bc56 out of bounds&gt;, nkey=3432596860) at items.c:466 #2 0x000000000040d099 in item_get (key=0x7f2d231264f4 "TC_db784e309e27b22873f62c3329d032c9", nkey=35) at thread.c:344 #3 0x0000000000403c14 in process_get_command (c=0x7f2d6227cb00, tokens=0x42cf0ed0, ntokens=25009664, return_cas=false) at memcached.c:2542 #4 0x0000000000408256 in process_command (c=0x7f2d6227cb00, command=&lt;value optimized out&gt;) at memcached.c:2975 #5 0x0000000000408bfe in try_read_command (c=0x7f2d6227cb00) at memcached.c:3185 #6 0x000000000040989d in event_handler (fd=&lt;value optimized out&gt;, which=17788, arg=0x7f2d6227cb00) at memcached.c:3491 #7 0x00007f2ec05c25f8 in event_base_loop (base=0x6a9c70, flags=0) at event.c:392 #8 0x000000000040cbb4 in worker_libevent (arg=0x63c6e8) at thread.c: 245 #9 0x0000003095a0673d in start_thread () from /lib64/libpthread.so.0 #10 0x00000030952d44bd in clone () from /lib64/libc.so.6 gdb -p 28064 GNU gdb Fedora (6.8-37.el5) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later &lt;http://gnu.org/licenses/ gpl.html&gt; This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". Attaching to process 28064 warning: process 28064 is a cloned process Reading symbols from /usr/local/sinawap/bin/memcached...done. Reading symbols from /usr/local/sinawap/lib/libevent-1.4.so.2...done. Loaded symbols for /usr/local/sinawap/lib/libevent-1.4.so.2 Reading symbols from /lib64/libpthread.so.0...done. [Thread debugging using libthread_db enabled] [New Thread 0x7f2ec05b86e0 (LWP 28059)] [New Thread 0x434f2940 (LWP 28064)] [New Thread 0x42cf1940 (LWP 28063)] [New Thread 0x424f0940 (LWP 28062)] [New Thread 0x41cef940 (LWP 28061)] [New Thread 0x414ee940 (LWP 28060)] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libc.so.6...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/libnsl.so.1...done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/librt.so.1...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libresolv.so.2...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libnss_files.so.2...done. Loaded symbols for /lib64/libnss_files.so.2 0x0000003095a0aee9 in pthread_cond_wait&lt; at &gt;&lt; at &gt;GLIBC_2.3.2 () from /lib64/ libpthread.so.0 (gdb) bt #0 0x0000003095a0aee9 in pthread_cond_wait&lt; at &gt;&lt; at &gt;GLIBC_2.3.2 () from / lib64/libpthread.so.0 #1 0x000000000040c10c in assoc_maintenance_thread (arg=&lt;value optimized out&gt;) at assoc.c:223 #2 0x0000003095a0673d in start_thread () from /lib64/libpthread.so.0 #3 0x00000030952d44bd in clone () from /lib64/libc.so.6 </pre> lee 2012-04-25T14:05:40 http://comments.gmane.org/gmane.comp.web.cache.memcached/14174 <pre>hi,memcached I'm sorry to trouble you! I have one question to ask you. I has started one memcache on port 11211 , if i want to add a new memcache(11212) , there are need to do same thing . So , how can i do that ? Can i want to restart the first memcached on 11211? </pre> 鱼 邹 2012-04-24T09:42:22 http://comments.gmane.org/gmane.comp.web.cache.memcached/14170 <pre>Status: New Owner: ---- Labels: Type-Defect Priority-Medium New issue 270 by malikb...&lt; at &gt;gmail.com: typo on memslap --help http://code.google.com/p/memcached/issues/detail?id=270 What steps will reproduce the problem? memslap --help What is the expected output? What do you see instead? Generates a load against a memcached custer of servers. custer -&gt; cluster What version of the product are you using? On what operating system? memcached1.2.3 </pre> memcached< at >googlecode.com 2012-04-23T15:46:36 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.web.cache.memcached