http://permalink.gmane.org/gmane.comp.web.cache.memcached hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.web.cache.memcached/14246 <pre> Comment #3 on issue 274 by vlak...&lt; at &gt;gmail.com: typo in memcached wiki http://code.google.com/p/memcached/issues/detail?id=274 http://code.google.com/p/memcached/wiki/NewUserInternals s/chukn/chunk </pre> memcached< at >googlecode.com 2012-05-20T17:46:31 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14245 <pre> Comment #2 on issue 274 by vlak...&lt; at &gt;gmail.com: typo in memcached wiki http://code.google.com/p/memcached/issues/detail?id=274 Sorry that's not "512" but "5*12". The asterisk probably tricked the formatting engine. </pre> memcached< at >googlecode.com 2012-05-20T17:40:10 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14244 <pre> Comment #1 on issue 274 by vlak...&lt; at &gt;gmail.com: typo in memcached wiki http://code.google.com/p/memcached/issues/detail?id=274 another typo: http://code.google.com/p/memcached/wiki/NewConfiguringServer s/maxiumum/maximum also noticed this strange formatting: ... you may receive is 5 &lt;b&gt;12. Always leave a few extra slots ... should rather be: ... you may receive is 512. &lt;b&gt;Always leave a few extra slots ... </pre> memcached< at >googlecode.com 2012-05-20T17:38:30 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14243 <pre>Status: New Owner: ---- Labels: Type-Defect Priority-Medium New issue 274 by vlak...&lt; at &gt;gmail.com: typo in memcached wiki http://code.google.com/p/memcached/issues/detail?id=274 http://code.google.com/p/memcached/wiki/NewInstallFromPackage s/rececnt/recent </pre> memcached< at >googlecode.com 2012-05-20T17:31:39 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14242 <pre> cmd_flush increasing means your application, or some cron, or some human, or some internet script (if your instance isn't protected) is issuing the command "flush_all" over and over. Also, your version is very old, but I doubt that's what's causing your flushes to happen :) -Dormando </pre> dormando 2012-05-11T22:35:16 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14241 <pre>We are currently having an issue with our memcache server that appears to be flushing data even though nothing in the apps are flushing data. I can set something with a TTL of 2 weeks yet it is only lasting in memcache for a max of something around 30 minutes. Print out of getstats in one of the php applications shows the following: Array ( [pid] =&gt; 24793 [uptime] =&gt; 373575 [time] =&gt; 1336770386 [version] =&gt; 1.4.2 [pointer_size] =&gt; 64 [rusage_user] =&gt; 4077.050000 [rusage_system] =&gt; 12475.330000 [curr_connections] =&gt; 1143 [total_connections] =&gt; 89011605 [connection_structures] =&gt; 1616 [cmd_get] =&gt; 438522590 [cmd_set] =&gt; 101189934 [cmd_flush] =&gt; 863 [get_hits] =&gt; 435589695 [get_misses] =&gt; 188087908 [delete_misses] =&gt; 27221748 [delete_hits] =&gt; 610097 [incr_misses] =&gt; 0 [incr_hits] =&gt; 0 [decr_misses] =&gt; 0 [decr_hits] =&gt; 0 [cas_misses] =&gt; 0 [cas_hits] =&gt; 0 [cas_badval] =&gt; 0 [bytes_read] =&gt; 71199529191 [bytes_w</pre> Mentch 2012-05-11T21:31:16 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14240 <pre>Sorry everyone, I was following the instructions in http://support.google.com/groups/bin/answer.py?hl=en&amp;answer=46608 and didn't expect "REMOVE ME" to be broadcast to the group like an ordinary message. </pre> Erik Seaberg 2012-05-11T21:03:02 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14239 <pre> </pre> Erik Seaberg 2012-05-11T20:58:32 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14238 <pre> None of these exclude any of the alternate methods of caching that I describe. In fact, every one of them can still be served out of a proper covering index, including queries like "SELECT song_id, count(*) ... GROUP BY song_id" or "SELECT sum(listen_count)". The complexity and sophistication of modern database indexes cannot be overstated. When used properly, they are borderline miraculous when combined with a good query parser and processor. I make no such claim. Further, I could even construct an adversary for my system such that every cached query result references a particular row, and updating that row could invalidate the entirety of my cache (incidentally, I could do the same for the system that you described). Thankfully, reality isn't an adversary, and we have the ability to choose the queries, what we cache, how we cache, etc. (the "how" is what we are discussing here). So are you saying that I'm making stupid assumptions? I claim that my assumptions about database indexes, queries, and cac</pre> Josiah Carlson 2012-05-11T18:19:20 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14237 <pre> This may be a silly question, but have you benchmarked your cached application against just going straight to the database? I've always had the impression that keeping a perfect cache of a database that beats it in performance was not possible because the overhead of cache invalidation (both in the cache and in the application) would ruin the performance gains on reads. Caches usually beat database performance by sacrificing accuracy (e.g. allowing race conditions and non-ACID behavior) and freshness of data. To put it another way, if it was possible to have an up-to-date cache that outperforms an RDBMS with the same data, wouldn't the makers of that RDBMS simply build that cache into their product? - Perrin </pre> Perrin Harkins 2012-05-11T17:57:58 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14236 <pre>Hm. I was expecting you to explain how your algorithm would work for this particular example. The example (if you couldn't tell from stupid column names and use case) was fictional for the ease of presentation. Instead of offsets, and limits, you could think on many other different parameters of a query such as: - constraints on other columns (for example AND type =HEAVY_METAL) - different sorting (for example SORT song_id DESC) - different columns in result (for example SELECT song_id, type) - different aggregation in result (for example SELECT COUNT(*) ) If you claim that each row of database can be returned only by O(1) different queries in your application, then I agree that your solution makes some sense for your application. Unfortunately, this might not be the case for other people, so if you plan to build a framework, that would be a stupid assumption to make. Sadly, this is an assumption found in several ORMs that I've seen, and now (after reading your posts) I understand why this framewo</pre> Jakub Łopuszański 2012-05-11T16:52:21 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14235 <pre> Usually swapped responses has to do with client object handling. What client are you using? is it a threaded server? Can you paste your "stats" and "stats settings" output? http://code.google.com/p/memcached/wiki/Timeouts - make sure to read over this as well. </pre> dormando 2012-05-11T16:48:29 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14234 <pre>Okay! Now this is perfect. A driving use-case with an example. Now I can see where you are coming from, and I can also see how this would be an issue for you. Before I get into the point-by-point, first a little story. A man goes to the doctor and says, "It hurts when I push here." The doctor says, "So don't push there." End of story. This will become relevant momentarily. On Friday, May 11, 2012 6:00:45 AM UTC-7, Jakub Łopuszański wrote: Rule # 3 in building scalable systems: there is no customized query. If you need to paginate on variable-sized pages, you create larger pages, then sub-paginate. Case in point: When Google produces search results for a query, they aren't making a query for the 10-entry page 1 for the first load, followed by another query for the 10-entry page 2 when you click on that link, etc. No. They make a query for the first X documents up front (historically I believe they made X somewhere in the range of 100-1000). They then cache that, and serve sub-results out of t</pre> Josiah Carlson 2012-05-11T16:13:41 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14233 <pre>So let me present you an example so you could present your idea on a real world examples. Suppose you have a web page, where you can view all songs of a given author, sorted by song_id, with pagination size set to 10, 20, or 50 results per page or any other value (say 42 results) if they wish. Therefore a database queries needed are something like: SELECT song_id FROM songs WHERE author_id = &lt; at &gt;A ORDER BY song_id ASC LIMIT &lt; at &gt;L OFFSET &lt; at &gt;O; Since a user can use many values (10,20,50, etc.) for &lt; at &gt;O, and many possible offsets &lt; at &gt;L (actually any natural number) you will get a huge number of different queries all referencing the same row with author_id=7, song_id=3. Let say there are like 100 different queries which relate to this row. Here are some 2 examples of such queries: SELECT song_id FROM songs WHERE author_id = 7 ORDER BY song_id ASC LIMIT 2 OFFSET 8; SELECT song_id FROM songs WHERE author_id = 7 ORDER BY song_id ASC LIMIT 2 OFFSET 9; You suggested to compute a digest for each such query actually perfr</pre> Jakub Łopuszański 2012-05-11T13:00:45 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14232 <pre>I believe we are talking past each other. When you are performing a write query against rows matching (author_id=7, song_id=3), presumably you would get a group of subspaces (7,3),(*,3),(7,*),(*,*). The latter subspace doesn't really help you at all, as it merely references all of your cached data involving author_id,song_id pairs. But here's the thing: the indexes that I described will give you the equivalent of your (*,3) and (7,*) subspaces, the intersection of which is exactly (7,3). So, as long as you index your columns independently, you can invalidate all of the necessary cache entries. It doesn't require invalidating infinitely many rows (unless you've got a magic machine that has infinite memory, and is caching infinitely many rows from an infinite number of queries), and it also doesn't require a huge explosion of space. Worst-case, 2x. But maybe I'm misreading what you are claiming to have built, as discussion about subspaces, bipartite matching, claimed infitities, really just resu</pre> Josiah Carlson 2012-05-11T08:02:52 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14231 <pre>So you suggest : 1. to build a biderectional graph, where the first direction (which you called "cache") maps a query to the rows, and the second one (which you called "INDEX['table_name']") maps a row to queries which depend on it. 2. keep the first part in a fast cache 3. keep the second part in a reliable database What bugs me is: since this is a symetric graph, one should expect that the size of "index" will be at least as large as size of "cache", which means that you will need a lot (potentially infinitely many) of storage to keep track of all queries that the system generated at any point in time. You would need some routine that would check what queries where evicted from cache and thus can be also removed from index, to keep the size of index bounded. Moreover I do not see how exactly would that work in case of queries which modify many rows. It seems to me that you would have to lock, select these rows, find all (potentially infinitely many) queries depednent on these rows in index, i</pre> Jakub Łopuszański 2012-05-11T05:12:43 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14230 <pre>When we are under high load (10k+ requests per second) we start getting communication failures (connection timouts,closed connection during header) to memcached. After a small period of time in this state we start getting mangled returns from the cache. Some returns will contain two pages data, some will simply be the wrong entry. We can recover from this by resetting the memcache servers. Is there some connection concurrency limit we may be bumping into under high load that would cause communication channel mismatches, such that request 1 will somehow receive request 2's answer? We are using memcached 1.4.11 with the following config on x86_64 -d -p 11211 -u memcached -m 1024 -c 1024 </pre> Greg Grensteiner 2012-05-10T20:27:35 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14229 <pre>hi , I write a client of memcached myselves,I string the command like "get key1\r\nget key2\r\nget key3\r\nget key4\r\n.......get key500\r \n",when I send it to the memcached server by socket,I get only 460~480 values or less.I doubt it's the bug of the libevent! who can tell me. </pre> casa zhang 2012-05-10T11:06:35 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14228 <pre>thank you for response ,here is the stats command show: STAT pid 25682 STAT uptime 1110901 STAT time 1336619479 STAT version 1.4.13 STAT libevent 1.4.14b-stable STAT pointer_size 64 STAT rusage_user 77287.380532 STAT rusage_system 189824.220332 STAT curr_connections 183 STAT total_connections 2127367424 STAT connection_structures 4882 STAT reserved_fds 40 STAT cmd_get 1701673335 STAT cmd_set 744414180 STAT cmd_flush 0 STAT cmd_touch 0 STAT get_hits 1462167441 STAT get_misses 239505894 STAT delete_misses 69425540 STAT delete_hits 1279021 STAT incr_misses 332 STAT incr_hits 452567424 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT touch_hits 0 STAT touch_misses 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 253805193105 STAT bytes_written 801603247301 STAT limit_maxbytes 8589934592 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 8 STAT conn_yields 0 STAT hash_power_level 24 STAT hash_bytes 134217728 STAT hash_is_expanding 0 STAT expired_unfetch</pre> lee 2012-05-10T03:18:24 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14227 <pre>thank you for response ,here is the stats command show: STAT pid 25682 STAT uptime 1110901 STAT time 1336619479 STAT version 1.4.13 STAT libevent 1.4.14b-stable STAT pointer_size 64 STAT rusage_user 77287.380532 STAT rusage_system 189824.220332 STAT curr_connections 183 STAT total_connections 2127367424 STAT connection_structures 4882 STAT reserved_fds 40 STAT cmd_get 1701673335 STAT cmd_set 744414180 STAT cmd_flush 0 STAT cmd_touch 0 STAT get_hits 1462167441 STAT get_misses 239505894 STAT delete_misses 69425540 STAT delete_hits 1279021 STAT incr_misses 332 STAT incr_hits 452567424 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT touch_hits 0 STAT touch_misses 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 253805193105 STAT bytes_written 801603247301 STAT limit_maxbytes 8589934592 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 8 STAT conn_yields 0 STAT hash_power_level 24 STAT hash_bytes 134217728 STAT hash_is_expanding 0 STAT expired_unfetch</pre> lee 2012-05-10T03:17:46 http://permalink.gmane.org/gmane.comp.web.cache.memcached/14226 <pre> On 4月26日, 上午10时15分, dormando &lt;dorma...&lt; at &gt;rydia.net&gt; wrote: </pre> lee 2012-05-10T03:08:46 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.web.cache.memcached