gmane.os.capros.devel

Re: Cross compilers

Matt Rice — 2013-03-17T22:52:32

yeah, I suppose my main reason for a complete vm was its simple to
host a single image containing everything from bit torrent/file
locker/whatever, but if you don't mind hosting again that point is
moot.


Basically I just wanted to evaluate if llvm suffered the same fate as
a gcc port (requiring posix emulation), If a native port of llvm were
possible and able to compile domains, To me it seemed worthwhile to
then spend the effort in switching the cross toolchain to use it as a
first step in a longer term effort.
If llvm-on-capros were a dead end like GCC
It'd seem like a waste of time/effort to me.
As it is I didn't see any serious roadblocks
Just alot of time/effort.

The other thing was that the llvm toolchain stance to cross compiling
is fairly different than
what GCC does.  By default a llvm/clang is built to target all the
targets and so they are all cross compilers by default.  As such it
seemed like there were some possibly maintenance burden reduction
possibilities there, e.g. if we could provide

Re: Cross compilers

Jonathan S. Shapiro — 2013-03-17T22:05:57

Matt:

There are several topics in your note, so I'm a bit confused. Let me try to
take them in turn.

I think a centos vm is closest to what I was advocating, and the path


If the goal is to help people get set up quickly, I think that setting up a
reference VM image isn't helpful in practice. It is *much* faster to
net-install a CentOS image (or any other major distribution) by pulling
from the major repositories than it is to copy a reference virtual machine
image.

Once you have the base image installed, the Coyotos/CapROS host-xenv
package exists to ensure that any additional packages you need get
installed.


On Sun, Mar 17, 2013 at 1:56 PM, Matt Rice <ratmice< at >gmail.com> wrote:


Yes. These tools are designed to live in a POSIX world. There is no
realistic hope of separating them from that world.




I'm not clear why we would look at LLVM, unless perhaps for performance
reasons. There are some compiler features that LLVM is missing for kernel
compiles, but it might be worth a look.

But if the goal i

Re: Cross compilers

Matt Rice — 2013-03-17T20:56:39


hmm, I see I mentioned the 'everybody builds a toolchain' model,
I didn't intend to advocate it, I don't really like it for all the
reasons you have given, I only really mentioned it for the sake of
completeness.

I think a centos vm is closest to what I was advocating, and the path
of least resistance from where are.  So I am definitely willing to go
alog with that route.  What follows is just a little update of what I
was looking into before this decision just for the record...

---

 I spent a little time evaluating small self hosting distros, and
kinda settled on www.baserock.org as the ideal VM for me at least.  in
that It's minimal and uses sources directly from version control (git)
which helps manage the capros/coyotos specific patches to the
toolchain,
 It has a way (with the trebuchet tool) that we can provide a delta
containing the cross compiler tools that gets applied to the standard
image.  The main issue is it is currently undergoing a rather furious
development pace I was hoping to attack i

Cross compilers

Jonathan S. Shapiro — 2013-03-17T06:12:50

Back in January there was a discussion about cross compilers. For various
reasons I seem to be back in the mode of maintaining cross compilers, and
I'm refreshing the Coyotos/CapROS cross tools in order to get my head back
into the cross tool build middens. I'm not resuming work on Coyotos, so I'm
not going to have time for much ongoing maintenance, but we all want to
have a set of working tools. After talking with Charlie Landau, we have
settled on a course of action.

I want to describe what is being done, but I also want to explain why we
are *not* adopting the "builld it yourself" approach that Matt Rice
suggested back in January.


Building a cross-tool chain is a seriously hairy process. Some steps have
to be done multiple times (e.g. GCC). It's delicate, finicky, error-prone,
cranky, not friendly to tool version updates, and generally a pain in the
ass. By the time I stopped, I had 20 virtual machines running different OS
versions and architectures, each dedicated to building the tools for one of
the

Re: cross compiler binaries situation

Matt Rice — 2013-01-30T06:23:00

well, the GPL section 3 is supposed to work like so:

3a(1):
distributor: gives binaries + source to person A
person A: gives binaries + source to person B
person B: has source + binaries

OR
3a(2):
distributor: gives binaries + source to person A
person A: gives binaries + 3b offer to person B
person B: has binaries + offer

3b:
distributor gives binaries + offer to person A
person A: gives binaries + 3c offer to person B
person B: has binaries + ability to get source.

(theres of course more variations)...

In the 3a(1) case, 'distributor' has fulfilled his obligations of the
license via
"provided that you also do one of the following" 3a, 3b, or 3c
and has no requirement to give the sources to person B.

If you take a look at the format of the repository:
http://www.eros-os.com/YUM/coyotos/Fedora/12/

you will see:
SRPMS/
i386/

under the following paragraph, this satisfies the obligations of 3a
with the key piece being 'from the same place'.

"If distribution of executable or object code is made by offe

Re: cross compiler binaries situation

Charles Landau — 2013-01-30T04:55:49

If you "distribute" the cross-compilation environment, and the sources 
for the cross compilers, by giving the URL of a server with that data 
(e.g. SourceForge), then why can't you "distribute" the source of the 
rest of Fedora by giving the URL of the Fedora servers?

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_jan

Re: cross compiler binaries situation

Lorens Kockum — 2013-01-29T18:41:48

I'm annoyed at their interpretation that "three years" means
three years from the day somebody else gives a physical copy to
yet somebody else. I'd have interpreted it to mean three years
from the date they stop distributing binaries from their central
site.


Is Scientific Linux (www.scientificlinux.org) very much
different from Fedora? I haven't found any License on their site
other than a quote of the GPL, no comments like Fedora has but
they still have their first version from 2004 on line.

Otherwise I'd go the Debian way.

FWIW, I have a Virtualbox VDI of a Fedora-thirteenish that
successfully compiled CapROS, it's 3.8 GB ready-to-boot and 1.1
GB gzipped.

------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http

Re: cross compiler binaries situation

Matt Rice — 2013-01-29T17:33:46

Unfortunately some legalities have gotten in the way,

https://fedoraproject.org/wiki/Legal:Distribution?rd=Legal/Distribution

so not only would we/I have to distribute the sources for our cross compilers,
but the sources for anything originating from fedora as well

given the general purpose nature of a distribution like fedora, it
contains a ton of stuff which is not necessarily essential to our
purposes, so we are on the hook twice for any of that...

at that point it makes more sense to me, to roll a custom distribution
or a different distribution which might better fit our purposes (and
all of the work that that entails/maintaining 2 os's instead of one).

I'll keep looking into how small I can make fedora with the overall
footprint in mind.

------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video

Re: cross compiler binaries situation

Charles Landau — 2013-01-26T04:29:15

So IIUC you are saying we (that is, you) can still update the tools to 
newer versions (using a mechanism that I don't understand but that works 
for you) and release them to others (using a mechanism that you are 
working out now), but you can do it on a schedule that is independent of 
(and less frequent than updates to) the distribution and version of the 
Linux on the host PC.

That seems like the best plan.


The work I know about doesn't seem like a lot, but the work we don't 
know about could be a tar pit.


I don't recall exactly how crt0 gets loaded in linux nor in CapROS; this 
would need to be figured out. DOMCRT0 in src/build/make/makevars.mk is 
no longer used, but was once used to explicitly load crt0, and we could 
go back to that.


src/base/domain/openssl is one of those "alien" things, and instead of 
using configure, the Makefile explicitly builds for the CapROS targets.



------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL

Re: cross compiler binaries situation

Matt Rice — 2013-01-26T03:29:23

I was actually wanting to update them to a newer fedora,

I'm currently running them patched on fc16, but willing to update the
cross-compilers for whatever version we decide upon, and upload a fork
of the existing tools including the patches and scripts to build the
image to a repository.

In other words, I'm already maintaining them locally, and trying to
figure out a decent way to distribute them that doesn't require me to
compile them for a bunch of different fedora versions, something i can
preferably sign and stick on a file locker.

In fact updating them will make it a lot easier given the newer fedora
image generation tools.

I suppose it comes down to: i'm far too lazy to generate them the way shap did,
the excess compiles every time the tools change or fedora releases a
new version would
make me dread rather than enjoy doing it.

I suppose what I should probably do, is just try it once
it see if it works for everybody or not, and then we can decide?


right, I also do a 3rd VM running CapROS-arm r

Re: cross compiler binaries situation

Charles Landau — 2013-01-25T22:10:47

Matt, IIUC, you are proposing:

1, The developer has a PC running his favorite flavor and version of Linux.
2. Under that system there is a VM running Fedora Core 10 (the latest 
version that I know has working cross-compilers). The developer compiles 
and perhaps develops on that system.
3. There is probably a second VM for running x86 CapROS.

I don't understand what you are proposing with a loopback device.

#2 would be essentially the same as the Linux development system I use, 
except mine is on a real PC.

One downside of this is that CapROS builds would remain stuck on the 
FC10 tools and wouldn't be able to take advantage of any subsequent 
progress.

What about this alternative: Use the (non-cross x86) tools (compiler and 
libraries) that come with the current version of Fedora Core (might work 
on other Linuxes too). We would lose the features that CapROS adds to 
the current cross tools. There are two that I know of:

1. The regular C libraries have procedures such as read() and open() 
that make

cross compiler binaries situation

Matt Rice — 2013-01-25T20:17:44

I've been wanting to get back in to capros/coyotos recently,

there are 2 distinct issues:
a) the main issue is my lack of ability to host a yum repository for
the binaries
b) the frequency of fedora release cycles (6 months) multiplied by the
number of architectures/OS's means a lot of compiling/maintenance, because
everybody has their own uprgrade schedule for which they upgrade their
fedora installs
even if we managed to limit ourselves to a single OS.

with regard to a)
the yum repositories require a specific URL format which they are
required to comply
makes it difficult to find free hosting, I suppose I could talk to the
'rpmfusion.org' guys, and see if I could host as part of that project.
(seems like a weird solution to me).

a different mechanism entirely has come to mind which would try and
attack these 2 problems directly.
it has its own unique set of issues... I wanted to see what your opinion was

the alternative mechanism would be to host a virtual machine image
which has a linux kernel & all o

Re: Problem building cparos tools.

Matt Rice — 2012-10-23T21:46:49

I just tested building/running capros under qemu on fedora 16 with the
patch series i sent in ccs-xenv.tar.gz I can test fedora 17 later or
if you'd like or 18 beta

built using 'make build' in ccs-xenv/SPECS directory

until recently there were some hacks you could do to run the old RPM's
on newer fedoras, those don't seem to work anymore (or i can't
remember how)

I don't really have any way to host a yum repository though.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct

Re: Problem building cparos tools.

Charles Landau — 2012-10-22T02:37:28

My apologies, but I have just now gotten time to review this thread, 
which started out with Kitty Guy trying to build CapROS. Unfortunately I 
don't have any knowledge to contribute on tools issues.

CapROS relies on tools that are built under the Coyotos project. If 
anyone succeeds in building a current version of the tools, in a way 
that would work for the CapROS tools too, PLEASE let us know on the 
CapROS list. I am stuck on Fedora Core 10 for my CapROS work.

-Charlie Landau

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct

Re: Problem building cparos tools.

Matt Rice — 2012-09-09T10:47:50


FWIW this appears to be a qemu issue (didn't happen with older qemu's,
doesn't happen with newer ones),  It's sending out a fault which is
reserved by intel... I haven't been able to track down the origin of
this in the affected qemu versions.  qemu doesn't even have this one
in their list of exceptions.  thus given this dubious nature i'm a bit
skeptical of working around a qemu bug and if this is appropriate
should we get it on hardware

reports on actual hardware:
http://news-posts.aplawrence.com/970.html

that said, I shall refrain from further hijacking the capros list for
coyotos stuff/patches apologies
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_________

Re: Problem building cparos tools.

Matt Rice — 2012-09-08T12:34:07


attached is a series of patches including some of kitty guy's patches
(those didn't work here, still unused variables guessing we built for
different targets),

rpm builds don't seem to require the symlink to ldscripts that non-rpm
ccs-xenv builds do, haven't looked into it further yet.

with that everything compiles, but doesn't function as expected due to
an exception after enabling interrupts.

I had a go at trying to finish up the capidl change, but for now it
was just easier to revert

not sure if we need to worry about older boost's in the
file_string->string thing, i don't have one around to test.
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/______________

Re: Problem building cparos tools.

Matt Rice — 2012-09-05T16:15:43



FWIW in light of recent events in 'glibc-land' i've been contemplating
the idea of reinvestigating the glibc port, (or eglibc should glibc
not adopt eglibc's 'option groups' mechanism, which seems to at least
be infrastructure for what shap originally desired from glibc, though
there afaict isn't currently a way to expunge all posix stuff) anyhow
such an endeavor would seem to require a toolchain update, thus i
don't mind spending time on it if there is interest/maybe otherwise.

I seem to recall getting coyotos working sometime after that capidl
issue arrived, only vaguely remember  anyhow i will have to look
around for patches and test them.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw

Re: Problem building cparos tools.

Kitty Guy — 2012-09-05T15:12:27


"Jonathan S. Shapiro" <shap< at >eros-os.org> wrote:

I expect some of the issues would be fixed by updating to current versions of binutils and gdb.


Yes, I expected that there would be some library to provide the symbols but did not find any.


The files are what is in the tree. 
Since capidl is broken the input is bogus. I am not surprised building the image fails.

Cheers

kg
-----------------------------------------------------
Mail.be, WebMail and Virtual Office
http://www.mail.be
"Jonathan S. Shapiro" <shap< at >eros-os.org> wrote:On Wed, Sep 5, 2012 at 7:16 AM, Kitty Guy <kittyguy< at >mail.be> wrote:
Hello,

Thanks for  your quick reply.

"Jonathan S. Shapiro" <shap< at >eros-os.org> wrote:

That could take care of the symlinks. The build errors will obviously remain even when built as RPM.

The right solution, really, is to refresh the entire cross-tool chain. That is likely a week-long effort, and I don't have the time to do it. > 2. The coyotos and capros tool chains really are not the same. The coyotos tools i

Re: Problem building cparos tools.

Kitty Guy — 2012-09-05T14:16:28

Hello,

Thanks for  your quick reply.

"Jonathan S. Shapiro" <shap< at >eros-os.org> wrote:

That could take care of the symlinks. The build errors will obviously remain even when built as RPM.


Then hardcoding the tool prefix is not that much of an issue I guess.

I was able to run the hello world demo of capros (built with capros tools) but coyotos fails linking any native binaries.

The first problem is that the coyotos binaries have unresolved symbols to the kernel and other binaries which the linker does not like.

I tried adding the -r flag like this:

 $(BUILDDIR)/Constructor: $(OBJECTS)
-       $(GCC) -small-space $(GPLUSFLAGS) $(OBJECTS) $(LIBS) $(STDLIBDIRS) -o $< at >
+       $(GCC) -small-space $(GPLUSFLAGS) $(OBJECTS) $(LIBS) $(STDLIBDIRS) -r -o $< at >

The other problem is that the image builder does not like the image descriptions which is something I cannot work around because I have no idea about these. Maybe looking at the spec in more detail would help. Anyway, the error:

make[6]: Entering directory `

Re: Problem building cparos tools.

Jonathan S. Shapiro — 2012-09-05T13:29:12

Kitty:

1. The process for building these tools is delicate, and (at this time)
really only works if you proceed by rebuilding the RPMs.
2. The coyotos and capros tool chains really are not the same. The coyotos
tools implement compilation models that the capros tools do not.
3. Trying to build capros using the coyotos tool chain won't work.


Jonathan

On Wed, Sep 5, 2012 at 3:57 AM, Kitty Guy <kittyguy< at >mail.be> wrote:

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
CapROS-devel mailing list
CapROS-devel< at >lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/capros-devel

Problem building cparos tools.

Kitty Guy — 2012-09-05T10:57:48

Hello,

I tried building capros but the current cross-tools won't build.

I applied some patches to get the tools built: https://bitbucket.org/kittyguy/ccs-xenv/changesets

There are still some problems. Once the configure in coytools would fail all tests with sed complaining it cannot open conftest.c. After poking around to find what the problem was it went away and I cannot reproduce it anymore.

Another issue is that the coyotos linker scripts are installed in a wrong place and ld does not link any of the capros binaries because it would not find them. Needed some symlinks to make the script usable:

$ readlink /capros/host/lib/ldscripts/elf_i386_coyotos_small.xc
../../i386-unknown-capros/lib/ldscripts/elf_i386_coyotos_small.xc

Finally it is quite stupid that the coyotos and capros build systems insist on having each a copy of the same tools under a different name. I tried to use tools build with the default coyotos prefix but the SSL binary would not build. Settling on one name would save quite a bit of