gmane.comp.lang.felix.general

Re: flx

john skaller — 2013-05-19T11:53:38

Well, to solve the problem with flx and toolchain plugins, I am just
sourcing the common toolchains (i.e. as includes).

Flx is edited now to use toolchains instead of caling the compilers
through the lower level generic compiler library functions.

I have temporarily added a TOOLCHAIN variable to the plat/config.flx
record to specify the toolchain (this will be replaced by using a package
file).

So I hoped to get rid of the compiler stuff from the config record,
and eventually get rid of the config record altogether.

For example 

config.EXT_SHLIB

will be .dylib on OSX or .so on Linux, but the toolchain object has a function
for that so we don't need it. Right?

Sometimes I HATE grep ...

/felix>grep -r EXT_SH src/*
src/lib/std/dynlink.fdoc:       modopen$ lib, modulename+#(Config::config).EXT_SHLIB, modulename;
.....

If we load a plugin toolchain we can find the appropriate extension for shared libraries.
The problem is .. the plugin is a shared library, and so we have to know the extension
for shared

Blog

john skaller — 2013-05-19T08:16:40

Thought I'd publish some rants ...

http://skalrant.blogspot.com.au/2013/05/reentrant-code.html


--
john skaller
skaller-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
http://felix-lang.org




------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

flx

john skaller — 2013-05-18T11:44:27

So, making "flx" use plugins is a bit tricky. To load the plugins,
whether statically or dynamically, they have to be separately 
compiled.

Guess which tool is used to compile them ....

--
john skaller
skaller-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
http://felix-lang.org




------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: [felix] static link plugins (and a longexplanation of my last 3 months activity)

john skaller — 2013-05-16T20:04:20

here is the new way to "freeze" plugins: i will use the Felix
webserver (src/tools/webserver.flx) as an example.

First you write a loader:

//////////////////////////////////////////////////////
// webserver plugin linker

class WebserverPluginSymbols 
{

  // We have to do this dummy requirements because static
  // linking removes the information in Felix.
  requires package "re2";
  requires package "faio";
  requires package "flx_arun";

  open Dynlink;

  // Now add all the symbols.
  proc addsymbols ()
  {
    static-link-plugin 
      fdoc2html,
      flx2html,
      fpc2html,
      py2html,
      ocaml2html,
      cpp2html,
      fdoc_scanner,
      fdoc_slideshow,
      fdoc_heading,
      fdoc_fileseq,
      fdoc_paragraph,
      fdoc_button
    ;

    // webserver
    static-link-symbol webserver_create_thread_frame in "webserver";
    static-link-symbol webserver_flx_start in "webserver";
    
  }
}

// Add the symbols
WebserverPluginSymbols::addsymbols;

// Now invoke the webserver!
println$ "R

static link plugins

john skaller — 2013-05-15T06:08:28

I am modifying the grammar so you should be able to say something like:

static-link-symbol fred in "lib";

which will emit:

header fred_static_tag 'extern "C"  void*fred;';
const fred : address = "&fred" requires fred_static_tag;
add-symbol("lib","fred",fred);

The effect of this is that it will 

(a) force static linkage of some object file containing symbol fred.
(b) load the address which is that symbol into the dynamic loader
table as a member of library "lib".

At present you cannot use this directly. However there will be an extension
of the above:

static-link-plugin "lib";

This is equivalent to

static-link-symbol lib_create_thread_frame in "lib";
static-link-symbol lib_flx_start in "lib";
static-link-symbol lib_setup in "lib";
static-link-symbol lib in "lib";

These symbols will cause this:

var  lib = Dynlink::load-plugin-func2 [bool * string, string, string] (
    dll-name="lib", setup-str=config, entry-point="lib"
  );

to setup the lib library module and call entry point "lib" (same

Re: [felix] bootstrapping Felix build

john skaller — 2013-05-15T05:31:08

On 13/05/2013, at 6:49 AM, john skaller wrote:



This now works on Linux too.
If there's anyone here using Linux. .. if there's anyone here at all . :)

--
john skaller
skaller-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
http://felix-lang.org




------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: felix-lang.org

john skaller — 2013-05-15T05:29:00

On 14/05/2013, at 3:24 PM, john skaller wrote:


I have just set up the DNS Zone records so in a few hours felix-lang.org should
work again. felix-lang.net should also work.

--
john skaller
skaller-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
http://felix-lang.org




------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

felix-lang.org

john skaller — 2013-05-14T05:24:51

println$ """Dear john,

Netregistry would like to confirm the successful transfer/renewal of felix-lang.org.
""";

--
john skaller
skaller-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
http://felix-lang.org




------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

bootstrapping Felix build

john skaller — 2013-05-12T20:49:32

Felix is now bootstrapped in the sense you can now
completely rebuild the current host model. 

PLEASE TRY THIS. And let me know if it works.
It is tested at this point ONLY on OSX using clang 3.3.
It almost certainly will NOT work on Windows.

Just do:

make build

in the top level of the repository to create the build/release
image, then do

make bootstrap

as many times as you like. This process builds a new Felix entirely
from the host configuration the original "make build" created,
then replaces build/release with it.

The first build process uses Python to create a  Felix image I will
call B0. The first bootstrap uses B0 to create B1. So the B0 tools
are used to run the build scripts which generate B1 tools.

So now we have a set of tools and libraries built by Felix,
but the Felix used to build them was built by Python.

So we need to run the bootstrap again to create B2 tools.
These get made by B1, which was built entirely by Felix,
but by a Felix built by Python.

To understand the concepts here, s

new functions

john skaller — 2013-05-10T08:53:16

A small mod now allows this:

Shell::system("ls","-lasp");
Shell::system("ls");
Shell::system ("ls","-lasp","*.flx");

What's the big deal?

An apparent variable number of arguments!

Really? 

Heck NO, Felix doesn't support that. No, what we have is:

  gen system[T with Streamable[T,string]] (args:T) : int =
  {
    var lst = Empty[string];
    for arg in args do 
      lst = lst + arg; 
    done
    return system args;
  }

So we're using an iterator method (implicit in the for loop) to access
an arbitrary Streamable (any data structure with an iterator method).

Lists are Steamable, and we're making one, but the return statement
calls

 // list of string argument
  gen system (args:list[string]) : int =>
    args.create_system_command.system
  ;

because it is more specialised. That quotes the arguments
and in turn calls:

 // string argument
  gen system (cmd:string) = {
    if Env::getenv "FLX_SHELL_ECHO" != "" do
      eprintln$ "[system] " + cmd;
    done
    return basic_system cmd;
  }

Note "st

flx_build

john skaller — 2013-05-07T15:02:27

The tool flx_build_rtl can now be used to bootstrap Felix,
or build a cross compilation target.

At present a couple of the steps cannot be performed other
than by copying. Any combination of the steps can be performed.

make rtlbuild

is an example (see the Makefile).

What this program can do is

C++ compile the rtl
Felix compile all the plugins
Felix compile all the tools

using a specified OS/C++ compiler pair. By default,

Darwin -> clang
Linux -> gcc
NT -> msvc

detected by "uname". Here is the help page:
===================================================
# locations

  --repo=repo               default: src
  --pkg=bootpkg
  --target-dir=target_dir     default: build/trial
  --target-bin=target_bin     default: host
  --source-dir=source_dir     default: build/release
  --source-bin=source_bin     default: host

# cleaning options

  --clean-target-dir          delete entire target directory
  --clean-target-bin-dir      delete target sub-directory
  --clean-target-bin-binaries delete binarie

product location

john skaller — 2013-05-06T06:19:28

At present, if you say 

flx fred
flx --static fred

you will get a DLL/executable called fred.dll fred.exe in the directory
containing the Felix file "fred".

There is a bug: if fred.dll/exe exists it is run immediately, but without
properly setting LD_LIBRARY_PATH.

however there's a problem: if you run 

flx /usr/local/lib/felix/felix-latest/share/src/tools/webserver

it will fail because it cannot write "webserver" into the shared
tools directory. Felix caches the intermediate files in

$HOME/.felix/cache/(text/binary)

but not the final product. You can of course name the output location

flx -ox $HOME/bin/webserver.exe .....

to work around this, but it destroys the convenient fiction Felix is
a scripting language and can just "run script" with caching used to
optimise performance (but not otherwise interfere with semantics).

Note the option

flx --output_dir= ...

can be used to change the C++ output cache location,  and the whole
cache can be changed by

flx --cache_dir

but since a final pro

Re: bootstrap status

john skaller — 2013-05-06T05:48:00

Ok, so the current Felix builder is progressing.

At present the layout is:


ROOT/share/src # repository source
ROOT/share/lib # reorganised as library
ROOT/share/lib/rtl # shared C++ headers

ROOT/target/config # some config
ROOT/target/lib # platform dependent Felix
ROOT/target/lib/rtl # platform dependent C++ headers,
object files, libraries, binary plugins, etc.
ROOT/target/bin # executables

The standard target is "host". There should be three options I'm working
towards:

(0) Build a new image using current directory as repository
This is a bootstrapping step.

(1) Build a new image, which means a new ROOT
This copies the whole "share" directory and
builds some target, usually target=host

(2) Add a new target to given image.

Note step 0 is basically due to a combination of a repository
pollution problem (naught me: I work in the repository directory,
which adds a lot of crap, sometimes not at the top level, eg
"flx src/tools/webserver" makes an executable file in the src/tools).
Th

Re: Library structure

john skaller — 2013-05-06T04:38:42

On 05/05/2013, at 11:31 PM, john skaller wrote:



Hmm .. one way to do this is:

(a) make the library modules with stupid class names:

_List
_silly_List

etc. At least the library should be fairly self contained,
and not use many globals apart from these names.
A few are needed eg "int".

(b) Make a class structure in one file and inherit the
contents:

class Data { class List { inherit _List; }}

This doesn't solve the problem of interaction of files
in user codes. Of course C has the same problem, though
it is restricted to extern symbols.

Although Felix has "private" that only works inside a class:
the top level class encloses everything.


--
john skaller
skaller-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
http://felix-lang.org




------------------------------------------------------------------------------
Introducing AppDynamics Lite, a free troubleshooting tool

Tool debugging

john skaller — 2013-05-05T23:36:59

At the moment flx has a switch "--echo" which is used to try to make
it echo shell commands. However flx could be used in a shell script
or program, and there are other tools that do shell callouts.

So I am hijacking the system and popen_in functions and inserting
a test based on an environment variable, which is set, will cause
the shell commands to be echoed to stderr. This can be applied
outside a process script. i.e. non-invasively. Seems a better idea than
passing switches all over the place, even though the latter is
more selective. The variable is:

FLX_SHELL_ECHO

and if set to anything other than "" will cause all shell callouts
to be echoed to standard error.

I should document this and other such environment variables which
can be used for monitoring, eg you can monitor the garbage collector
already, service calls, etc.


--
john skaller
skaller-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
http://felix-lang.org




----------------------------------------------------------------------------

Library structure

john skaller — 2013-05-05T13:31:58

I need some help on this...

At present, Felix has classes to make qualified names.

However, the

include "file";

feature causes the file to be parsed and the parse tree (actually a sequence
of statements) to be inserted at the beginning of the including file.

The loss of ordering and scoping is natural, because Felix also has
a rule a file can only be included once. The feature is meant for
hauling in dependencies not structuring.

So given the loss of order you can cut files up at the top
level as you please, but you cannot cut classes up, because
a class is a single statement. Although classes can be nested
this can only be done within a single file.

On the other hand I've stuck with the rule that the filename
is irrelevant: unlike say Python or Ocaml, it isn't a module 
or class name. I don't like that because if you get a file
that starts to get fat, if you split it up you have to change
all the references to it.

This means lots of stuff ends up in global scope resulting
in hijacking. For example

Weird bug

john skaller — 2013-05-04T08:18:39

noinline proc build_felix (xargs:list[string])
{
  if xargs.len.int < 2 do
    println$ "Usage: flx_build [debug]";
    println$ "Execute from directory containing repository";
    println$ "Uses build/release to create build/trial";
    System::exit(1);
  done

  var v = varray xargs;
  var boot_package = xargs . 1;  // <-----------
  var debug = xargs.len.int == 3;
  flx_build (boot_package, debug);
}

This pretends a list is an array, and gives an error:


In assignment (_ctor_size<5863>[size] (_ctor_size<5863>[uint] literal[uint](1))) = (-<6127>[size] (/*proj*/(case 1 of bool)(unsafe_get_mv_1660<99804>), literal[size](1)));
Expected primitive _ctor_size<5863> to have property lvalue
In /Users/johnskaller/felix/build/release/share/lib/std/datatype/list.fdoc: line 191, cols 6 to 5
190:       | Cons (_,?t), ?i => unsafe_get (t, i - 1uz)
191:     ;
          *
192:

The actual function indicated is:

    //$ get n'th element
    pure fun unsafe_get: list[T] * size -> T =
      | Cons (?h,_), 0uz => h

bootstrap status

john skaller — 2013-05-03T07:33:04

I am currently able to do this:

# build the Python based Felix
make build 

# OSX bootstrap
make bootstrap

# upgrade
rm -rf build/release
mv build/trial build/release

# Again
make bootstrap

And so on, as many times as you like.

Running the bootstrap process takes the Python based build and uses it
to create a Felix driven build, however the conversion is incomplete
because the tools used to actually make the new RTL etc are from
the Python build.

So we repeat the process, now using Felix built tools to rebuild
everything.

There's no assurance exactly how many cycles this takes to reach a fixpoint,
however it does appear to. The result of several iterations of the above
can be installed and used (with a couple of restrictions since not quite
everything is built at the moment). However the webserver works,
and that's a pretty major program.

Limitations
========

(1) At present the Makefile hard codes toolchain_osx_clang because I'm not
sure how to make it parametric. You'll have to edit it to duplicate

DLL hell

john skaller — 2013-05-03T00:39:34

In bootstrapping experiments with Felix I built a new Felix with Felix in
a directory called "build/trial".

Then I deleted "build/release" and moved "build/trial" to "build/release"
hoping to perform a second stage bootstrap.

But I got this:

"""
LD_LIBRARY_PATH=build/release/host/lib/rtl build/release/host/bin/flx --test=build/release --static -o build/trial/host/bin/flx_build_rtl_demo build/trial/share/src/tools/flx_build_rtl_demo build/trial/host/config build_flx_rtl_clang_osx build/trial . trial-tmp
flx_librtl: configdir=build/trial/host/config, bootpkg=build_flx_rtl_clang_osx, librtldir=build/trial/host/lib/rtl, repository=.
compiler     : clang
os           : OSX

.....

Dynamic linkage error opening DLL toolchain_clang_osx
Dynamic linkage error
filename: toolchain_clang_osx.dylib
operation: dlopen
what: dlopen(toolchain_clang_osx.dylib, 2): Library not loaded: build/trial/host/lib/rtl/libflx_pthread_dynamic.dylib
 Referenced from: /Users/johnskaller/felix/build/release/host/lib/rtl/toolchain_clang_o

unused param bug

john skaller — 2013-05-02T06:54:06

I have a routine:

noinline proc handle_switch
(
  config:&config_type,
  control:&control_type, 
  zloopctl:&loopctl_type,
  arg:string
)
{
  C_hack::ignore(zloopctl);
  if arg == "--nostdimport" do
......

If I remove the C_hack::ignore, the routine fails with:

SYSTEM FAILURE
[cpp_instance_name] unable to find instance zloopctl<38042>[]

I think I understand this: Felix removes unused variables. I think it
can sometimes remove unused parameters too. When it does this
calls to the function are adjusted to not pass the missing parameter.

However this only works in special cases: it can't work if the function
is turned into a closure (the signature would be wrong). because of this
I think I disabled this feature: parameters are always considered used.
More precisely, a parameter is always considered used even if it isn't.

Well as you see I'm confused because this works:

noinline proc f(x:int, y:&int, z:int) { println$ x+z; }
noinline proc g(a:int,var b:int, c:int) { f (a, &b, c); }
proc aa() {
  g (1,2,3)

Re: [felix] Felix based build now appears correct.

john skaller — 2013-05-01T15:59:17

On 01/05/2013, at 10:15 PM, Michael Maul wrote:


The core
=======


The way it works **for the core** is:

The files in share/src/config are just read only data,
they're not an actual platform configuration.

flx etc will take an option --target=platform
which means (at present) the files in

INSTALL_ROOT/platform

are processed, in particular

INSTALL_ROOT/platform/config # fpc
INSTALL_ROOT/platform/lib # felix
INSTALL_ROOT/platform/lib/rtl #C++

The default value is

--target=host

i.e. "host" is the name of the default platform.

No actual configuration should ever be in

INSTALL_ROOT/share

anywhere, that's for universally read-only sharable data.
It's basically a copy of the repository reorganised.

Note that the config only contains *.fpc files, C++ headers
which are configured (generated) go in

INSTALL_ROOT/platform/lib/rtl

at the moment. (Not sure I like that but anyhow).

These files are #included from the shared C++ headers
to configure them.  I'm not sure I like that either. 
Maybe it s