http://permalink.gmane.org/gmane.comp.db.riak.user hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.db.riak.user/11295 <pre>Given Ubuntu's approach to packaging Erlang, I attempted to reproduce your issue after installing Erlang R15B01 from source. Using the steps in the gist below, I was able to get a Riak 1.2.1 devrel from source working on Ubuntu 12.04 32bit: https://gist.github.com/hectcastro/55886f19ada97e534e0b -- Hector On Sat, May 25, 2013 at 11:37 AM, Andrew Thompson &lt;andrew&lt; at &gt;hijacked.us&gt; wrote: </pre> Hector Castro 2013-05-25T18:19:08 http://permalink.gmane.org/gmane.comp.db.riak.user/11294 <pre>Ubuntu does some braindead splitting up of the erlang standard library (so the standard library is no longer guranteed to be 'standard'). You'll probably need to install erlang-syntax-tools or something. Andrew </pre> Andrew Thompson 2013-05-25T15:37:40 http://permalink.gmane.org/gmane.comp.db.riak.user/11293 <pre>Hi all, Riak version used is 1.2.1. I tried spinning up 4 nodes with 'make devrel' but it failed at compiling deps and returned some errors about erlang_js trying to use 64bit operations on 32bit machine. Unfortunately I no longer hold the error log. I then erased riak folder and run 'make devrel' from a fresh copy. This time it still failed at compiling deps but instead stops at erlang_protobuffs: ==&gt; cluster_info (get-deps) ==&gt; lager (get-deps) ==&gt; poolboy (get-deps) ==&gt; meck (get-deps) ==&gt; protobuffs (get-deps) ==&gt; basho_stats (get-deps) ==&gt; riak_sysmon (get-deps) ==&gt; mochiweb (get-deps) ==&gt; webmachine (get-deps) ==&gt; bear (get-deps) ==&gt; folsom (get-deps) ==&gt; riak_core (get-deps) ==&gt; luke (get-deps) ==&gt; erlang_js (get-deps) ==&gt; bitcask (get-deps) ==&gt; merge_index (get-deps) ==&gt; ebloom (get-deps) ==&gt; getopt (get-deps) ==&gt; eper (get-deps) ==&gt; eleveldb (get-deps) Cloning into 'leveldb'... Note: checking out '1.2.2p5'. You are in 'detached HEAD' state. You can look around, make experimental changes and commi</pre> khoi pham cao 2013-05-25T00:28:59 http://permalink.gmane.org/gmane.comp.db.riak.user/11292 <pre>I don't think you can do what you are asking.If, however, you are truly viewing your data as being redundant at the node level, rather than the disk level, then you could just stripe it (RAID 0) on each node. We haven't been this brave yet... _______________________________________________ riak-users mailing list riak-users&lt; at &gt;lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com </pre> Brian Akins 2013-05-24T17:32:47 http://permalink.gmane.org/gmane.comp.db.riak.user/11291 <pre>Dmitry, AAE uses leveldb under the hood, which itself has a reliance on mmap. If you attach to one of the Riak nodes (via 'riak attach') what does 'os:cmd("ulimit -v").' say? -Z On Mon, May 6, 2013 at 5:25 PM, Dmitry Aleksandrov &lt;elixmoon&lt; at &gt;gmail.com&gt;wrote: _______________________________________________ riak-users mailing list riak-users&lt; at &gt;lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com </pre> Ryan Zezeski 2013-05-24T16:34:28 http://permalink.gmane.org/gmane.comp.db.riak.user/11290 <pre>FWIW, we forked the bear repo for another change and I removed the `native` flag as well. Riak 1.4 should no longer have these warnings. -Jared On Tue, May 21, 2013 at 11:50 AM, ageemoore &lt;ageemoore&lt; at &gt;gmail.com&gt; wrote: _______________________________________________ riak-users mailing list riak-users&lt; at &gt;lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com </pre> Jared Morrow 2013-05-24T16:33:37 http://permalink.gmane.org/gmane.comp.db.riak.user/11289 <pre>I've been looking at object stores and Riak seems very promising. However, I strongly believe that, to fit the model, object stores need to use commodity disks and do failure protection by replication. RAIDing/striping over multiple disks doesn't seem to be a valid option to me. You're already paying a 3x penalty for replication and then are being expected to pay a 2x penalty for RAID10. I've seen some discussions about multiple disk per server being configured as multiple file systems, but I'm not certain that is supported/encouraged. Is there some documentation that I'm missing that could definitively say whether multiple file systems per server is supported/encouraged? Thanks, Jim Jim Finlayson Storage Guy _______________________________________________ riak-users mailing list riak-users&lt; at &gt;lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com </pre> Jim Finlayson 2013-05-24T13:24:31 http://permalink.gmane.org/gmane.comp.db.riak.user/11288 <pre>Pascal, We've seen similar behavior in testing and have always failed to track down the cause. Nothing is actually running in Riak, but the process still just hangs around. I filed an issue for this https://github.com/basho/riak/issues/330. Coincidentally on the erlang-bugs mailing list someone just reported a similar problem with shutdown. I am going to try the suggestion provided on erlang-bugs and get the change into our upcoming 1.4 release. Thanks for the message and sorry no one got back to you sooner. -Jared On Tue, May 7, 2013 at 2:09 AM, Pascal Schmiel &lt; pascal.schmiel&lt; at &gt;bluelionmobile.com&gt; wrote: _______________________________________________ riak-users mailing list riak-users&lt; at &gt;lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com </pre> Jared Morrow 2013-05-24T15:22:41 http://permalink.gmane.org/gmane.comp.db.riak.user/11287 <pre> Hi, Jimmy. If that map phase is producing a list of [bucket, key] pairs, as you describe, the only thing you need to do to use them as inputs is to add another map phase right after that one. So: inputs: [["user", "mathew"]], query: [ {map:… get a list of [[bucket, friend_key]] }, {map:… do something with a friend} Riak will handle reading each object and passing it into the map function for you. The special thing you may have been thinking of is "link-walking". If instead of storing the list of friends in the object (or in addition to), you stored their names in the Link metadata, you could use: inputs: [["user", "mathew"]], query: [ {link: {bucket:"user", tag:"friend"}}, {map:… do something with a friend} The 'link' phase would do the bucket-key-pair production for you. You still use a map phase right afterward to work with the objects that were linked. Or, if you just wanted the friend/user objects, instead of some piece of them, you could use the URL h</pre> Bryan Fink 2013-05-24T12:30:47 http://permalink.gmane.org/gmane.comp.db.riak.user/11286 <pre>G'day! We're about to start upgrading our Riak clusters from 1.2.1 to 1.3.1 and I have a couple of questions about the upgrade process. Firstly, the new "enable_health_checks" config option. It's not clear what impact this option will have on a busy cluster. The way I read it, if a riak_kv_vnode message queue goes over the "vnode_mailbox_limit" DisableThreshold then the riak_kv service is disabled. Doesn't this effectively take the node down? If you have a busy cluster isn't it possible that all nodes could hit this limit around the same time and bring the whole cluster down? Especially if you're running close to the limit and one node gets taken offline then the other nodes try to take up the slack which puts them over the threshold. Or am I misunderstanding this feature? Is there any way to query the riak_kv_vnode message queue length to get an idea of how close we are to hitting the limit? Secondly, reformatting the indexes to fix the 2i integer encoding problem. Do I need to run the "riak</pre> Shane McEwan 2013-05-24T09:23:38 http://permalink.gmane.org/gmane.comp.db.riak.user/11285 <pre>Idan, I'll investigate this a bit and see if I can replicate similar behavior and hopefully I can get back to you with more information. Thanks for sharing the info. Kelly On Wed, May 22, 2013 at 3:23 AM, Idan Shinberg &lt;idan.shinberg&lt; at &gt;idomoo.com&gt;wrote: _______________________________________________ riak-users mailing list riak-users&lt; at &gt;lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com </pre> Kelly McLaughlin 2013-05-23T23:31:03 http://permalink.gmane.org/gmane.comp.db.riak.user/11284 <pre>On behalf of Basho, I'm ecstatic to announce that RICON West 2013 has been scheduled and tickets are now available for purchase. Here are the details: * Date: October 29-30 * Location: St. Regis Hotel; San Francisco, CA. The official site is here: ricon.io/west.html The Lanyrd page is here: lanyrd.com/2013/riconwest Like we've done with past RICONs, the format will be two days, with two action-packed tracks each day. You can expect to hear from: * Companies, large and small, running Riak and Riak CS in production * Members of the Basho Engineering team discussing features, road map, and Riak app development * Leading thinkers and practitioners in distributed systems from industry and academia * Much more The confirmed speakers as of right now are: * Jeff Dean (Google Fellow) - Closing Keynote * Kate Matsudaira (Founder &amp; CTO, Pop Forms) * Justin Sheehy (CTO, Basho Technologies) * Pete Bailis (UC Berkeley) * Jeff Hodges (Distributed Systems Engineer, Twitter) See the Speakers page for their full bios: </pre> Mark Phillips 2013-05-23T22:10:39 http://permalink.gmane.org/gmane.comp.db.riak.user/11283 <pre>Hi Julien, Changing to LevelDB would probably make sense for your environment as all keys will no longer need to be kept in memory. You should therefore be able to handle larger amount of data. 4GB is however not a lot of RAM, so the default settings will need to be tuned as these generally are optimised for servers with more available RAM. In order to reduce resource consumption with leveldb, I would recommend using a much smaller ring size. For a 5 node cluster we usually recommend a minimum ring size of 64, but in your case 32 may actually be a more suitable option. Riak also runs best without swap, so you should ensure that the 4GB of RAM you have available are backed by real RAM and not swap space as this will negatively affect performance. Also follow the linux tuning guidelines and ensure you do not have swap enabled. As Riak performance also heavily depends on the speed of the disks, ensure these are configured to be as fast and efficient as possible. This is the best advise I have at the moment</pre> Christian Dahlqvist 2013-05-23T19:23:03 http://permalink.gmane.org/gmane.comp.db.riak.user/11282 <pre>Hello, 2013/5/23 kurt campher &lt;campherkurt2&lt; at &gt;gmail.com&gt;: This is the reason I approached the issue using the search features, since indexes anyway exist there. In Riak this would require switching the datastore to LevelDB, and in my use case Bitcask is superior. I will propably make a config file in my app or some temporary variable which will contain 10 keys which I get with a map/reduce than is run daily, and then I'll fetch the predefined set from Riak when I need the "10 first results". Although this will require 10 requests to get 10 results, search would have been ideal since one request can return a big set of results.. It is a pitty the search feature does not support "q=*" as query. </pre> Otto Kekäläinen 2013-05-23T17:54:56 http://permalink.gmane.org/gmane.comp.db.riak.user/11281 <pre>This looks awesome, Jon. Keep us posted on where it goes. And don't be scared to submit a CFP about this for RICON West [1]. :) Mark [1] ricon.io/west.html On Tue, May 21, 2013 at 1:21 AM, Joshua Muzaaya &lt;joshmuza&lt; at &gt;gmail.com&gt; wrote: </pre> Mark Phillips 2013-05-23T17:33:05 http://permalink.gmane.org/gmane.comp.db.riak.user/11280 <pre> Here are the relevant docs: http://docs.basho.com/riak/latest/tutorials/Secondary-Indexes---Examples/#Range-Query Mark </pre> Mark Phillips 2013-05-23T16:41:20 http://permalink.gmane.org/gmane.comp.db.riak.user/11279 <pre>Hello Christian, all So, after thinking a little bit and reading more we decided to switch to leveldb. For now, we use the default config, but we probably will have to change a couple things in the future. We also monitor the whole cluster using collectd and we see some interestng behaviors on which I'd love to have your feedback. We monitor the node_[get|put]_fsm_time_* and we noticed that our 100% percentile behaves particularly bad. I have attached a collectd graph for it. (ignore the 'hole' in data collection, as we had to updates a couple thing on the collecting server). The mean for get is 1259ms, the average is 3334ms, which is in the same order of magnitude, but the 100% percentile is at 401996ms on average, which is 2 orders of magnitude larger. Is that common? We see the exact same thing for put request. I am not sure whether this is the cause, but we have a big difference for the sizes of objects. The median and mean are respectively 1545 and 1951 and 32315 for the 100% percentile. (just 1 or</pre> Julien Genestoux 2013-05-23T08:23:42 http://permalink.gmane.org/gmane.comp.db.riak.user/11278 <pre>This coming Wednesday (5/29) at the Atlanta Tech Village [0], Basho will be holding its second Atlanta Riak meetup [1]. Tom Santero and myself will be presenting on how to properly configure and tune Riak for different use cases. We'll also cover how to monitor and profile Riak, as well as the underlying operating system. Please feel free to contact me if you have any questions. Also, if you have any interest in giving a talk about your use of Riak at a meetup, please drop me a note. Hope to see you there! [0] http://atlantatechvillage.com/ [1] http://www.meetup.com/Atlanta-Riak-Meetup/events/116993012/ -- Hector </pre> Hector Castro 2013-05-23T15:55:20 http://permalink.gmane.org/gmane.comp.db.riak.user/11277 <pre>Hey Otto So a fundamental design attribute of Riak (and other key/value stores) is that data retrieval is done by knowing the key. To pull the first 10 keys is similar to what one would do in a relational DB like MySQL. However with Riak this sort of operation is expensive as a full table scan is done. The keys are not indexed. There some work-arounds/solutions though: - Use the Riak Secondary index. Documentation is available. - Set a key which has as a value all the keys in a list format. ['key1', 'key2', etc...] - This requires you to maintain this properly and keep the "eventual consistency" in mind. Just my 2 cents. Hope it helps. On Thu, May 23, 2013 at 10:07 AM, Otto Kekäläinen &lt;otto&lt; at &gt;kekalainen.net&gt;wrote: _______________________________________________ riak-users mailing list riak-users&lt; at &gt;lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com </pre> kurt campher 2013-05-23T09:17:51 http://permalink.gmane.org/gmane.comp.db.riak.user/11276 <pre>2013/5/23 Simon de Haan &lt;simon&lt; at &gt;praekeltfoundation.org&gt;: Thanks! What about my actual goal, can I in any way retrieve with search or some other feature say the 10 first keys and values from the Riak datastore? Serach with simply "*" would be cool, since search has build in rows and start that allow pagnation. </pre> Otto Kekäläinen 2013-05-23T08:07:43 http://permalink.gmane.org/gmane.comp.db.riak.user/11275 <pre>It has to do with the tokenizer you are using in your schema. There are a few different ones but the one you are using won't index less than two characters. Not sure which is which off the top of my head but it's in the docs somewhere. Also in the riak book. -Alexander On May 23, 2013 3:28 AM, "Otto Kekäläinen" &lt;otto&lt; at &gt;kekalainen.net&gt; wrote: _______________________________________________ riak-users mailing list riak-users&lt; at &gt;lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com </pre> Alexander Sicular 2013-05-23T08:02:09 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.db.riak.user