gmane.comp.lang.smalltalk.exupery

Planning for Exupery 0.15

2008-08-08T21:38:57

The two areas most in need of improvement before a 1.0 now are run time performance and reliability. Hopefully 0.15 will lead to a decent improvement in both. First runtime performance as the end of 0.14 involved a decent round of testing and debugging. Here's some benchmarks: arithmaticLoopBenchmark 417 compiled 94 ratio: 4.436 bytecodeBenchmark 725 compiled 262 ratio: 2.767 sendBenchmark 692 compiled 403 ratio: 1.717 doLoopsBenchmark 389 compiled 385 ratio: 1.010 pointCreation 423 compiled 426 ratio: 0.993 largeExplorers 198 compiled 199 ratio: 0.995 compilerBenchmark 245 compiled 249 ratio: 0.984 Cumulative Time 401 compiled 260 ratio 1.542 The primary goal is to improve the last two benchmarks, the two macro benchmarks. Both benchmarks use a profiler to decide what to compile, the goal is to compile enough methods to make a difference reasonably quickly so the benchmark doesn't take too long to run. Here's the profile fo

[ANN] Exupery 0.14 is released

2008-07-27T21:25:58

The major improvement is to the speed of register allocation. I've fixed a couple of major performance problems so register allocation takes 50% of compilation time even for the largest methods. Register allocation now appears to take roughly linear time. There's still plenty of room to improve compilation time. Every stage copies the entire intermediate graph to produce the input to the next stage which is redundant, most stages only change a few places. The register allocator's liveness analyser still uses Sets to represent which variables are live rather than bit vectors. This release can compile cascades, the last missing core language feature. Exupery still can only compile a handful of the core primitives, it only compiles #at: for pointer objects. Cascades were added because they were used more in 3.10, the choice was either delete 2 system tests or add cascades. I delayed the release to add cascades. There's a few bug fixes of old bugs but this release is not noticeably more reliable than the pre

Exupery update preparing for the 0.14 release

2008-06-15T19:59:50

The Exupery 0.14 release is in final testing now. The major improvement is to register allocator performance. The register allocator should also produce slightly better code due to the changes. There's also now support for cascades, the last major missing language feature. The main gain in this release is register allocation is much faster on large methods. Now it takes about 50% of the compilation time for all methods, it used to take almost all the time for large methods. This means compilation time is now roughly linear with method size. The major improvements for 0.14 were done by mid April. The release has been delayed due to upgrading to 3.10. Exupery's testing uncovered two bugs in 3.10 and four tests were failing due to changes in the base image. Two tests have been deleted as they were trying to compile methods that have been removed. Those methods were involved in bugs found by the stress test. There should be unit tests that cover any changes to the compiler required to fix them. Two tests we

Re: Problem with ColouringRegisterAllocator

Igor Stasenko — 2008-06-05T09:32:57

The bug was in my code was machine ^ machine ifNil: [ MachineX86 new] instead of: machine ^ machine ifNil: [ machine := MachineX86 new] 2008/6/2 kampjes.demon.co.uk>:

Problem with ColouringRegisterAllocator

2008-06-02T20:11:34

> in method addInterferenceEdge: firstRegister to: secondRegister > > is stumbled upon error: key not found when it does: > > secondNode := interferenceGraph at: secondRegister. > > an interferenceGraph keys = a Set(t9 t11 t14 t10 t2 t24 t22 t19 t25 > t3 t8 t15 eax t20 t21 t16 t11 edx t17 t6 t3 t26 t12 t2 t4 t21 ecx t16 > t12 t9 t23 t1 t5 t20 t13 t15 t14 t5 t27 t1 t6 t19 eax t13 t4 t17 t18 > t28 t7 t8 t18 t10 t7) > > and secondRegister = eax. > > It looks like MedMachineRegister needs #= and #hash methods to make > things working correctly with dictionaries. > > I'll try add these methods, lets see if issue will disappear. There should only ever be one instance of each register. The bug is creating the second one. That said, you may find it easier to impleemnt = and hash than track it down. Using identity is a deliberate design decision. Having multiple versions of the same item may cause problems. I think you may be OK with duplicate registers but am not sure. > Btw, i don't know

Problem with ColouringRegisterAllocator

Igor Stasenko — 2008-06-01T18:25:06

in method addInterferenceEdge: firstRegister to: secondRegister is stumbled upon error: key not found when it does: secondNode := interferenceGraph at: secondRegister. an interferenceGraph keys = a Set(t9 t11 t14 t10 t2 t24 t22 t19 t25 t3 t8 t15 eax t20 t21 t16 t11 edx t17 t6 t3 t26 t12 t2 t4 t21 ecx t16 t12 t9 t23 t1 t5 t20 t13 t15 t14 t5 t27 t1 t6 t19 eax t13 t4 t17 t18 t28 t7 t8 t18 t10 t7) and secondRegister = eax. It looks like MedMachineRegister needs #= and #hash methods to make things working correctly with dictionaries. I'll try add these methods, lets see if issue will disappear. Btw, i don't know why or where it creates another instance of same register (MedMachineRegister name: #eax), in my code i using: 'machine registerNamed: #eax', which should create instance only once, and then return same instance for consequent calls.

Re: Register spilling mechanism

2008-04-24T20:16:37

> Don't worry about it. I'm planning to overthrow the rule of C and make > every bit in VM be implemented as native methods > which can be compiled at run time. :) > So, i care little about what calling convention C or other libraries having. > All foreign calls will be handled by FFI class(es) (compiled as > anything else as well), while inside i'll have methods compiled to > machine code and i'm free to choose any calling convention for them, > as well as choose own stack layout which will be convenient for GC and > object memory. > That's why i asked how i can control these aspects with Exupery. If an signal comes in from the OS, then the signal will be executed on the machine (C) stack, so the stack pointer must always be correct. If not you risk having your C stack trashed by the signal. > > > > > Can be there any contracts between Exupery and caller, where i can > > > state that code should compile under certain rules, like: > > > - specify a set of registers, which should be pres

Re: Register spilling mechanism

Igor Stasenko — 2008-04-24T00:08:09

2008/4/24 kampjes.demon.co.uk>: Don't worry about it. I'm planning to overthrow the rule of C and make every bit in VM be implemented as native methods which can be compiled at run time. :) So, i care little about what calling convention C or other libraries having. All foreign calls will be handled by FFI class(es) (compiled as anything else as well), while inside i'll have methods compiled to machine code and i'm free to choose any calling convention for them, as well as choose own stack layout which will be convenient for GC and object memory. That's why i asked how i can control these aspects with Exupery. Thank you for explanation. I will look forward for any updates concerning these features. Maybe, as temporary solution, i can patch register allocator to get some feedback on how many stack space i need, and then re-run it again. Or i can try to measure maximum number of live temps in code by own.

Re: Register spilling mechanism

2008-04-23T21:34:21

> 2008/4/22 kampjes.demon.co.uk>: > > > > Igor Stasenko writes: > > > More broader explanation why i need this.. > > > I want to allow direct stack manipulation for native methods to some extent. > > > > > > In code, one can write: > > > > > > (object1 expression1) push. > > > (object2 expression2) push. > > > address call. > > > > > > Now i need to make sure that Exupery will produce correct stack frame > > > for a call, regardless what code inlined in expression1/expression2 > > > it should not interfere with top stack layout, which should be: > > > > > > > > > > > > > > > > > > when entering routine at (address call). > > > > Are you talking about the C stack or the Smalltalk stack? > > > > If you're talking about the Smalltalk stack then in Exupery it's only > > dealt with in the front half of Exupery. It's handled in the > > ByteCodeReader and IntermediateSimplifier

Re: Register spilling mechanism

Igor Stasenko — 2008-04-22T22:56:53

2008/4/22 kampjes.demon.co.uk>: No, forget about Smalltalk stack, i plan to use Exupery at lower levels to produce machine code fed by my own compiler. I don't know what you mean under 'C stack'. Isn't Exupery compiles directly to machine code? Then why you speaking about C stack at all? ;) As i understand (correct me if i'm wrong) by fixed size stack you mean, that after done register allocation, Exupery determines the exact size of stack, and then it inserts an instruction in method's preamble to allocate it, like: push %ebp mod %ebp, %esp sub %esp, stacksize and at return point it does reverse: add %esp, stacksize pop %ebp ret or, does just 'ret' , in case if caller are responsible from cleaning the stack and can restore %ebp. And by dynamic stack size, you mean that instead of preallocating fixed amount of bytes on stack, in future, it can use pushes/pops in code based on demand? I having nothing against both models, just need a safe way how i can manipulate stack directly without being

Re: Register spilling mechanism

2008-04-22T20:39:13

> More broader explanation why i need this.. > I want to allow direct stack manipulation for native methods to some extent. > > In code, one can write: > > (object1 expression1) push. > (object2 expression2) push. > address call. > > Now i need to make sure that Exupery will produce correct stack frame > for a call, regardless what code inlined in expression1/expression2 > it should not interfere with top stack layout, which should be: > > > > > > when entering routine at (address call). Are you talking about the C stack or the Smalltalk stack? If you're talking about the Smalltalk stack then in Exupery it's only dealt with in the front half of Exupery. It's handled in the ByteCodeReader and IntermediateSimplifier. Exupery uses exactly the same stack locations as the interpreter so doesn't need to check the stack height. It'll break for exactly the same methods that will cause the interpreter to crash. Exupery does mode

Re: Debugging a bug that wasn't.

2008-04-22T19:19:07

Igor Stasenko writes: > Okay. So, the problem that forcing to collect garbage after single > allocation seem too stressing for your tests. > Then why not divide test on parts - one for testing GC, another one > for testing interrupts which causing active process switching? There's no bug hence the title of this thread. For a week I thought there was a bug because with one instance it was appearing and disappearing based on whether I compiled the code or not. That was just a consequence of a very close race. Bryce

Re: Register spilling mechanism

Igor Stasenko — 2008-04-22T13:01:16

More broader explanation why i need this.. I want to allow direct stack manipulation for native methods to some extent. In code, one can write: (object1 expression1) push. (object2 expression2) push. address call. Now i need to make sure that Exupery will produce correct stack frame for a call, regardless what code inlined in expression1/expression2 it should not interfere with top stack layout, which should be: when entering routine at (address call).

Register spilling mechanism

Igor Stasenko — 2008-04-22T12:44:08

I'm currently want to make an inlining for methods of specific format (lets call them 'native' methods), these methods should work as ST methods (same calling convention & stack layout when you entering method), but these methods don't do polymorphic sends unless specified exclusively, instead all sends are inlined statically with compiler. I don't want to describe much details here, just want you to help me how to deal with register spilling when inlining. Suppose i inlining a send like: object message: a+b with: c+d with: a+d with: .... so, all arguments to method now should use machine temporary registers instead of stack, because it's more effective. What i fear of that if register allocator sees that there is not enough registers in CPU to fit them all it starts using stack for storing temp values and there is some situations where i need to know exact stack depth for a method. Is there any way in Exupery to get know, how many registers will be spilled on stack, so this information can be used by my

Re: Debugging a bug that wasn't.

Igor Stasenko — 2008-04-21T22:18:01

2008/4/20 kampjes.demon.co.uk>: Okay. So, the problem that forcing to collect garbage after single allocation seem too stressing for your tests. Then why not divide test on parts - one for testing GC, another one for testing interrupts which causing active process switching?

Re: Debugging a bug that wasn't.

2008-04-20T20:51:08

> GC signals a semaphore after finishing (don't remember - is weak > finalization process using it or different one). > I'm not sure, can it somehow interfere with given case? > > If weak finalization loop creates new objects, it can lead to > exclusive looping in it, because user processes running with lower > priority. > The problem has nothing to do with finalisation. The issue is running a GC for every allocation slows down execution a lot. Enough that there's a race to get through the test code before all 10 profiling threads start profiling. If they all start profiling then they can consume all available CPU. Bryce

Re: Debugging a bug that wasn't.

Igor Stasenko — 2008-04-20T13:25:45

2008/4/20 kampjes.demon.co.uk>: GC signals a semaphore after finishing (don't remember - is weak finalization process using it or different one). I'm not sure, can it somehow interfere with given case? If weak finalization loop creates new objects, it can lead to exclusive looping in it, because user processes running with lower priority. finalizationProcess [true] whileTrue: [FinalizationSemaphore wait. FinalizationLock critical: [FinalizationDependents do: [:weakDependent | weakDependent ifNotNil: [weakDependent finalizeValues. "***Following statement is required to keep weakDependent from holding onto its value as garbage.***" weakDependent := nil]]] ifError: [:msg :rcvr | rcvr error: msg]. ]. see, if FinalizationSemaphore having excess signals after executing cycle once, it continues with execution. I think better to place FinalizationSemaphore initSignals at the end of weak finalization, so it wouldn't be triggered by garbage collection which may

Debugging a bug that wasn't.

2008-04-20T09:38:45

I'm testing and debugging before the 0.14 release. The major benefit is faster compile times and better generated code. There's around a 10% gain or loss for the macro benchmarks depending on what's compiled. The bug was triggered by the below code. Originally it was compiling "^[42] value". The set-up shown below is what generates the fault without needing to compile the code. The fault was odd because it wasn't crashing, and wasn't stuck in an infinite loop but the image had locked up. I was investigating by interrupting execution with gdb and looking at both the Smalltalk stack using "p printCallStack()" and the C stack using "where". Normally, if compiled code was corrupting the object memory it would crash fairly quickly. That it kept executing, which could be seen because the Smalltalk stack kept changing was weird. If the GC's got a corrupt view of the object memory then normally it'll end up really corrupting it when it decides to try to interpret the middle of an object as an object header. Th

Deciding how to fix a bug.

2008-04-12T21:40:47

I've started the final release testing for Exupery 0.14. There are two bugs captured with tests so far. The first has something to do with interrupts, it needs a few running profilers to cause the bug to happen. I'm hoping this one is the source of the unrepeatable bugs that were in the the previous release, if so solving it may allow Exupery to run for much longer as a background compiler. The second bug is caused by the current work. What's happening is the currentContext register is being spilled but is being used by code generated in the register allocator to load the stack. It may also be needed by spill code to spill back into the stack. The simplest fix is to avoid spilling the currentContext. I doubt that spilling the currentContext is a good idea as it'll make spills of stack registers much more complex. Not spilling currentContext is probably the right thing to do unless the spill heuristics are improved to avoid spilling currentContext then spilling many stack registers that will use it to load

Floating point support

2008-03-16T21:37:00

> Hello Bryce, > > i'd like to ask you, how do you envision adding floating point support > to Exupery > to make it able to use FPU. > > The problem i see, is that it would require adding a type to registers > to be able to use float-type values in intermediate code > representation. > > Or else, how it would look like, when i need to encode floating point > operations like: > > temp1 := floatValue1. > temp2 := floatValue2. > temp3 := temp1 + temp2. "here compiler should generate FPU > instructions instead of integer addition" > > Or maybe i'm looking at problem at wrong angle? > Maybe easier to make intermediate in format like: > > unaryFloatOp(argmunentAddress, resultAddress) > binaryFloatOp(argmunent1Address, argument2Address, resultAddress) > compareFloatsOp(arg1address, arg2address) > > then compiler don't have to deal with float registers (in register > spilling code and other optimization patterns). I'll probably create floating point operations and have floating poi

Floating point support

Igor Stasenko — 2008-03-16T13:05:06

Hello Bryce, i'd like to ask you, how do you envision adding floating point support to Exupery to make it able to use FPU. The problem i see, is that it would require adding a type to registers to be able to use float-type values in intermediate code representation. Or else, how it would look like, when i need to encode floating point operations like: temp1 := floatValue1. temp2 := floatValue2. temp3 := temp1 + temp2. "here compiler should generate FPU instructions instead of integer addition" Or maybe i'm looking at problem at wrong angle? Maybe easier to make intermediate in format like: unaryFloatOp(argmunentAddress, resultAddress) binaryFloatOp(argmunent1Address, argument2Address, resultAddress) compareFloatsOp(arg1address, arg2address) then compiler don't have to deal with float registers (in register spilling code and other optimization patterns). Another thing, is support of byte-wide operations, like loading/storing byte at specific address. And of course being able to do some operations with