http://permalink.gmane.org/gmane.comp.lang.ruby.core hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.lang.ruby.core/19601 It would still require changes to autoload so that before autoload had started but after the autoload constant had been encountered another thread would not have any chance to see a totally empty constant. autoload either needs to leave the autoload marker there or put some other marker there. The ability to affect concurrency from within the autoloaded file hinges on the same requirement, as does a callback. - Charlie Charles Oliver Nutter 2008-12-04T04:09:08 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19600 On Dec 3, 2008, at 9:55 PM, James Gray wrote: That's a new thread, so the mutex.lock in its require will hang until any outer level one is done Dave Thomas 2008-12-04T04:08:58 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19599 autoload is a little trickier than that since it needs some intermediate state that isn't totally undefined but doesn't show up as yet-to-be-autoloaded either. But yeah, if we're willing to expect that require and autoload are one-thread-at-a-time, a lot of problems go away. I'd say load could potentially be unsynchronized, since it's more explicit and doesn't make any guarantees about whether it will fire twice, etc. But perhaps it should just be done across the board. - Charlie Charles Oliver Nutter 2008-12-04T04:05:48 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19598 I don't think this is really any different than having a single mutex for all requires. The requiring thread would acquire it for the duration of the require. Any additional requires it runs into it would still have the lock for. Any other threads doing requires would block until that thread had come all the way back out. This implies a few things you should not do: * Don't do long or blocking operations during script loads * Don't launch threads that will do additional requires during script loads (or at least don't depend on them doing their requires until the launching thread has finished its own) This obviously means requiring becomes a single-threaded process, which is probably the only way to make it completely safe. - Charlie Charles Oliver Nutter 2008-12-04T04:01:36 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19597 I loved your let's-move-on post so much it actually pains me to write this, but can we be totally sure of that? Thread.new do require "i_so_hope_im_wrong_about_this" end James Edward Gray II James Gray 2008-12-04T03:55:59 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19596 Hi, At Thu, 4 Dec 2008 12:15:03 +0900, Dae San Hwang wrote in [ruby-core:20284]: Hmmm, they obviously conflict. I guess so. Could you tell how/from where those headers are included? Nobuyoshi Nakada 2008-12-04T03:39:08 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19595 I've checked system header files and found following macro definitions of ISPRINT. /* in "/usr/include/readline/chardefs.h" */ #define ISPRINT(c) (IN_CTYPE_DOMAIN (c) && isprint (c)) /* in "/usr/include/sys/euc.h" */ #define ISPRINT(c, wp) (wp._multibyte && !ISASCII(c) || isprint(c)) I guess the second definition of ISPRINT macro clashes with ruby? daesan On Dec 2, 2008, at 4:42 PM, Nobuyoshi Nakada wrote: Dae San Hwang 2008-12-04T03:15:03 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19594 On Dec 3, 2008, at 8:40 PM, hemant wrote: I don't see how there could be a deadlock: once you've claimed the mutex, your thread is the one that will do requires. Any requires nested inside the original one will be executed in the same thread, so simply rescue and ignore the recursive lock exception. Honestly, folks, I really don't see what all the fuss is about. One mutex, claimed by autoload, require, and load, will do the trick. If autoload claims it, then it gets to do the requires if it needs to, because they'll operate in-thread. If require claims it, then only requires and autoloads in that same thread will run until the original require finishes. Etc etc... No deadlock, and no contention. The current MRI implementation has a bug. Fix it with a mutex, and it all gets better. Then we find the next global thing that's not thread safe... :) Dave Dave Thomas 2008-12-04T02:55:38 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19593 Why not use const_missing for lazy loading and protect "require" by a mutex (there could be a deadlock, if requires are nested, but in practice I was assuming it can be avoided) hemant 2008-12-04T02:40:39 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19592 If the thread autoloading the file required another file, it's still the same thread. Tomas Matousek wrote: Charles Oliver Nutter 2008-12-04T02:40:37 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19591 A stupid idea that just came to mind (maybe someone already mentioned it, I did not read all the messages): Wouldn't being able to have autoload managed by a callback a nice idea? It would allow the frameworks to implement it their own way... Just my 2 cents Vincent Isambart 2008-12-04T01:12:50 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19590 looks like this has been discussed before [on the todo list since 2000 [1]]. Another "band aid" style option might perhaps be to allow the 'load' file to know which autoload constant triggered it, then it could do its own concurrency. ex: autoload_new :ConstantName, 'thread_safe_require' then when autoload kicks in, it sets "something" [$something, whatever] to 'ConstantName' then 'thread_safe_require' would be able to hopefully load it sanely, if so desired. Perhaps create new functions load_thread_safe and/or maybe require_thread_safe :) Thoughts? -=R [1] http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/4589 Roger Pack 2008-12-04T00:56:43 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19589 Loading code right now is a depth-first search of the dependency tree for the required file. Your suggestion would change it to a breadth first search. As a result, the following example would try to inherit from Bar before Bar was defined, something that is not the case now. If you extend that to Kernel#require and not just Kernel#autoload, as you say, then most currently existing Ruby code would break because of this. $ cat a.rb Kernel.autoload(:Foo,'b') Foo.something $ cat b.rb Kernel.autoload(:Bar,'c') class Foo < Bar def self.something puts "FizzBang" end end $cat c.rb class Bar end Ken Bloom 2008-12-04T00:32:30 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19588 Issue #814 has been updated by Aaron Patterson. File better.patch added Here is an even better patch. With tests. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/814 ---------------------------------------- http://redmine.ruby-lang.org Aaron Patterson 2008-12-03T23:57:06 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19587 is it a compiler directive? [or used to be?] -=R On Wed, Dec 3, 2008 at 2:47 PM, Robert Dober <robert.dober< at >gmail.com> wrote: Roger Pack 2008-12-03T23:45:19 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19586 On Tue, Dec 2, 2008 at 11:10 PM, Charles Oliver Nutter <charles.nutter< at >sun.com> wrote: That might be a great idea. The only drawback being if you have file2.rb: sleep run this: Thread.new { sleep 1; require 'file1'} require 'file2' So if one thread is "happily running" within a require file [rails does this?] then other threads would be prohibited from requiring. Or I guess it could be mutex-ified per autoload [like require is currently for 1.9]--i.e. when a second thread loads the same constant, have it wait till the first's auto-load returns [somewhat of a band-aid, but...hmm this is hard]. -=R Roger Pack 2008-12-03T23:44:29 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19585 Yes, I'd forgotten for a moment about the deadlock issue. Thanks for reminding me. One way to avoid the deadlocks might be to dedicate a thread to servicing all requests to load new code. Require and constant resolution with "autoload" enabled would be redefined to queue code load requests. The code "loader" thread would remove such requests from the queue, then load the required ruby files (including other files required by the requested file) while the requesting thread waited remained blocked. The loader thread would release the requesting thread only after its require request had been satisfied. This single code loader thread thus serializes the otherwise ill behaved concurrent Ruby code loading. There are some details I'm (knowingly) omitting. Does anyone see any truck sized holes in this general idea? - brent Jim Weirich-2 wrote: Brent Roman 2008-12-03T21:58:54 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19584 Hi list I was just having some real fun with the 191p2 and somehow I was sure that tail recursion would be optimized away, but alas.... I than searched the list for any information on that and did not find anything. Did I only dream or am I correct in assuming that Ruby1.9 will eventually optimize tail recursions? Thx in advance Cheers Robert Robert Dober 2008-12-03T21:47:20 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19583 Hello, On Wednesday 03 December 2008 19:47:01 Brent Roman wrote: I'll definitely try it out, too. Actually that's not exact, according to my experiments - it optimizes away assignments of NULL to a pointer. VALUE is not a pointer, and it doesn't optimize away neither NULL nor 0 assignments (i tried with gcc 4.3.2) Coincidently, GCC 4.4 is supposed to have an optimization for variables in switch (see http://gcc.gnu.org/gcc-4.4/changes.html), but unfortunately, if i understand it correctly it's for constants only (i wonder if it's impossible for variables, or just nobody has written it yet :) Regards, Michal Babej 2008-12-03T20:42:27 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19582 Because of this problem, we *will* be removing the use of autoload in 1.1. However, this is still a pretty big bug in Ruby itself (since it makes autoload broken). Yehuda Katz 2008-12-03T19:44:25 http://permalink.gmane.org/gmane.comp.lang.ruby.core/19581 Unless the file that’s being auto-loaded requires another file via regular 'require' call. Then you might have two different constants in the system being accessed concurrently by 2 threads each auto-loading a file that requires other files. Tomas -----Original Message----- From: Charles.O.Nutter< at >sun.com [mailto:Charles.O.Nutter< at >sun.com] On Behalf Of Charles Oliver Nutter Sent: Wednesday, December 03, 2008 6:52 AM To: ruby-core< at >ruby-lang.org Subject: [ruby-core:20259] Re: autoload and concurrency Except that my proposed fix for autoload would have all secondary threads either seeing the final loaded constant value or waiting on the in-progress load; there would be no concurrent call to require whatsoever. Tomas Matousek wrote: Tomas Matousek 2008-12-03T19:04:48 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.lang.ruby.core