gmane.org.netlabs.libc.user

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-17T12:58:34

** Reply to message from "Dave Yeo" telus.net> on Sun, 16 Nov 2008 14:55:10 -0700 No matter what? For example, the grep uses fopen (without "b") for the "-f" file and then uses fread's to read the file into a buffer. What I find is: #ifdef HAVE_DOS_FILE_CONTENTS # include # ifdef HAVE_SETMODE # define SET_BINARY(fd) setmode (fd, O_BINARY) # else # define SET_BINARY(fd) _setmode (fd, O_BINARY) # endif #endif in a header file and #if defined(SET_BINARY) /* Set input to binary mode. Pipes are simulated with files on DOS, so this includes the case of "foo | grep bar". */ if (!isatty (desc)) SET_BINARY (desc); #endif in the source. Since this code handles the case where files are opened to be searched and since the code for opening a file to read patterns ("-f") works without "-Zbin-files" and without any setmode's (it uses fopen and fread), then I am led to think that I may be covered. What do you think? No you were correct. I changed back to the non-debug options whi

Re: Built grep 2.5.3 successfully, or not?

Dave Yeo — 2008-11-16T21:55:10

Yes that is correct and is usually a good default. Lots of porting problems can be caused by the line endings issue. See http://svn.netlabs.org/libc/wiki/kOptions for OS/2 specific flags Yes, it is a libc problem, not GCC problem. Note that it works fine for most files. See http://svn.netlabs.org/libc/ticket/18 for Knuts take on it. It still works fine for normal files, just questionable for stdin, stdout and stderr. Test with and without the flag. I think it is needed for the files.sh test to pass but I might be wrong. I have it here in my config.site and I also pass the files.sh test. Yes that is the best solution. Often programs already have the setmode call in a #ifdef for windows so can just add __EMX__ to the #ifdef. Other programs have a configure check, perhaps using check_headers to check for io.h and wrap the function in a #ifdef HAVE_IO_H wrapper. I like the second solution for portability but the first solution is simpler and quicker to implement. (Note I might be misremembering some o

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-16T22:27:11

** Reply to message from "John Small" os2world.net> on Sun, 16 Nov 2008 07:52:46 +0500 Update: I have looked at the CVS repository for grep 2.5.4 at gnu.org and they have commented out these three tests because they "are known to fail". So I don't think it is worth looking into them any further. I have commented them out in my 2.5.3 script. So the only remaining issue to my knowledge is the "-Zbin-files" issue (discussed in a separate thread) which breaks the "-f" option for grep.

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-16T21:05:37

** Reply to message from "Dave Yeo" telus.net> on Sat, 15 Nov 2008 19:19:52 -0700 As I understand it "-Zbin-files", when it works, causes files to be opened, by default, in binary mode. Is this correct? If it worked, is it a good default? Other questions: - Is "-Zbin-files" is broken in all 3.3.5 (and up) versions of GCC for OS/2? - If yes, then why is it included LDFLAGS in u:\moztools\config.site-gcc335b4 in Paul Smedley's build environment? - How does one know when "-Zbin-files" is needed? More specifically to the case at hand, how do I know if "-Zbin-files" is needed for grep to function properly? It seems to pass the "make check" tests without it but how can I tell if something untested would be broken? Should I add setmode calls to the source to compensate for the missing "-Zbin-files"? Thanks for your help!

Re: Built grep 2.5.3 successfully, or not?

Dave Yeo — 2008-11-16T02:19:52

It is the -Zbin-files that is allowing you to pass file.sh. Using -Zbin-files is equal to setmode (file, O_BINARY). note also that there is a ticket filed about -Zbin-files not (always?) working with stdin, stdout etc. Also look at the -u and -U grep options .... Dave

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-16T02:52:46

** Reply to message from "Steven Levine" earthlink.net> on Sat, 15 Nov 2008 07:47:33 -0800 Now I have replaced the above with just two very simple changes to the script. And the output and the byte-offset numbers match (for all but 3 of the 32 tests)! I went by the example in the README file: ./configure CC=c99 CFLAGS=-g LIBS=-lposix In any case, I think I have found the problem with building a debug version. I am using Paul Smedley's build environment. And when the configure script runs it reads one of his files: u:\moztools\config.site-gcc335b4. In this file are settings for CFLAGS and LDFLAGS. My earlier efforts simply added -g to the other options which are set in this file. I also noticed that there were some commented out debug settings for CFLAGS and LDFLAGS. The two sets of settings are: debug: CFLAGS='-D__EMX__ -DOS2 -D__ST_MT_ERRNO__ -g' LDFLAGS='-Zexe -Zomf -Zmap -Zargs-wild -D__ST_MT_ERRNO__' non-debug: CFLAGS='-D__EMX__ -DOS2 -D__ST_MT_ERRNO__ -O3 -mcpu=pentium3' LDFLAG

Re: Built grep 2.5.3 successfully, or not?

Steven Levine — 2008-11-15T15:47:33

In <200811150706.1l1jUN6mG3Nl36J0< at >mx-various.atl.sa.earthlink.net>, on 11/15/08 at 05:06 PM, "John Small" os2world.net> said: OK. For our purposes, memchr is logically equivalent to strchr. Not everyone has infinite time. That's why I originally recommended that you upload the test results. > I'll look at this later today. FWIW, I recommend that all uploads of this kind contain a datestamp in the name. This is OK for now, but it's not a good long term solution IMO. I say this because it makes you mods harder to use on the next grep release. The way most of us do this is once the port is done, we do something like diff -u -r grep.2.5.3.org grep2.5.3.ported > grep.2.5.3.patches.diff When the next verison of grep arrives, we apply the patches to the new sources with patch -p0

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-15T12:06:06

** Reply to message from "Steven Levine" earthlink.net> on Fri, 14 Nov 2008 20:01:15 -0800 There are no \0 terminators in the data. They end up using: memchr (motif, '\n', total); I was hoping you would copy and paste it into an editor and then delete the line endings to get the 4 lines per test I described. Since the tested command is also very long and, I think, important in figuring out what is wrong (the actual or the expected output), I have uploaded the text in a zip file: There are 32 tests with multiple offsets for each test. Instead of changing all those offsets I 1) Edited the searched file and removed the \r's 2) Made the searched file read-only so the test could not rewrite it 3) Edited the script to remove (tr -d "\'r") the \r's from grep's output This is how I got the number of failures down to just 4. I have tried both sh ./configure CFLAGS=-g LDFLAGS=-g sh ./configure CFLAGS=-g and make CFLAGS=-g (an error occurs if I us

Re: Built grep 2.5.3 successfully, or not?

Steven Levine — 2008-11-15T04:01:15

In <200811142209.1l1bxw5Y13Nl36v2< at >mx-harris.atl.sa.earthlink.net>, on 11/15/08 at 08:09 AM, "John Small" os2world.net> said: Hi, The --help option is supposed to show all of the available options. That's definitely not going to handle line endings in a platform specific manner. If suspect they are using strchr or strtok to find the line terminators. Interesting. It's still a garbled mess because your mail clint is wrapping the output. What I would do is line up the columns and figure out which one is wrong. That says you could just adjust the offsets, assuming the output is really correct. Add -g to CFLAGS and LDFLAGS. Yes. Link with -Zomf. Have fun, Steven

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-15T03:09:30

** Reply to message from "Steven Levine" earthlink.net> on Fri, 14 Nov 2008 17:17:23 -0800 How does one learn of the available configuration variables? Does ./configure --help just provide a general list of options or would it also include options specific to the software? Will do. The file is read with the fread function into a single block of memory. It will take me a while to find where and how it is parsed. I should have been more specific. The problem is when -i is used in conjunction with certain other options (e.g. -o and --color). Since my earlier message I have checked out gnu.org and found that this is a problem for many platforms, not just OS/2. [snip] Sorry about that. I meant to explain that the text for each test is exactly four lines each: Test # ... output: ... expect ... FAIL So I hope you won;t mind if I try it again: Test #11: { ../src/grep -F -n -b -m 5 -C 1 yes 2>/dev/null; echo ?0; sed 's!^!X!'; } < yesno.txt | tr '\n' ' '/' | tr -d '\r' output: "2-

Re: Built grep 2.5.3 successfully, or not?

Steven Levine — 2008-11-15T01:17:23

In <200811141100.1l115zrL3Nl36r0< at >mx-ailes.atl.sa.earthlink.net>, on 11/14/08 at 08:59 PM, "John Small" os2world.net> said: Hi, I would fix both. It's much easier to be comfortable with a port that passes all the test. Also, if the fixes are not too intrusive, the grep maintainers might accept them. In the case of line endings there may already be a configuration variable to handle this. IMO, yes. Check for how the pattern file is opened or how the code strips line endings. I have to suspect there are some spurious CRs to be accounted for. There are both checking for a match failure. It's likely. This is not necessarily a flaw on on unix box. That's odd. There must a problem with case mapping. This is hard to visualize with the line wrap. It might be better to log to results to a file and attach it. An good eCS/OS2 port should be insensitive to the line endings since both styles occur. According to the man page the format is prefix line(s) matched line suffix lines(s) --

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-14T15:59:54

** Reply to message from "Steven Levine" earthlink.net> on Tue, 11 Nov 2008 08:34:06 -0800 After looking at the scripts I believe that many of the failures are due to line endings. There are two places where the line endings may be mishandled. One is in the grep executable and the other is in the test scripts themselves. Do they both need to be fixed? IOW, if grep.exe works correctly is it important to fix test scripts which incorrectly indicate a failure? As I indicated earlier 3 scripts report failure. One script, file.sh, tests the "-f" option. Are the following equivalent? Command line (single line, watch wrapping): (echo "miles" | ..\src\grep -i -E -f patfile > /dev/null 2>&1) || echo "File_pattern: Wrong status code, test \#1 failed" Script (4 lines) echo "miles" | ../src/grep -i -E -f patfile > /dev/null 2>&1 if test $? -ne 0 ; then echo "File_pattern: Wrong status code, test \#1 failed" fi If these are equivalent then the grep I built fails this test (and oth

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-14T04:47:59

** Reply to message from "Robert Blair" listemail.net> on Tue, 11 Nov 2008 13:43:52 -0800 How do you know that it "works so far"? Have you built anything successfully? If yes, please tell me where I can get the source code so I can try that one too.

Re: Built grep 2.5.3 successfully, or not?

Robert Blair — 2008-11-14T04:09:36

** Reply to message from "John Small" os2world.net> on Fri, 14 Nov 2008 04:26:43 +0500 The only program I compiled runs. I have not tried any other program yet. Yes. What I wanted was a program that could write to a X10 "firecracker" I searched and found the source so I downloaded "flipit-0.3.5.tar.GZ". When you install Paul's build environment two files are put in the root directory. One is gcc335.cmd which you need to modify to match your system. The other file is readme.os2, which explains the very basics of compiling a unix program. From the file readme.os2: "Many things will compile with a simple ash ./configure, make" This did not work for me as the "ash" shell did like the configure file included with the flipit program. This is why I needed to install the "sh" shell. After installing "sh" I did "sh ./configure" followed by "make" and compiled the flipit program. Then I had to make a small change to the flipit source for OS/2 and recompiled (I also had to fix a problem becaus

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-13T23:26:43

Re: Built grep 2.5.3 successfully, or not?

John Small — 2008-11-13T23:13:47

Re: Built grep 2.5.3 successfully, or not?

Peter Weilbacher — 2008-11-12T08:53:44

GLIBIDL is a special library for Mozilla builds, and I doubt that it is used by any other package that you are trying to build. Indeed, it would confuse things in case you were trying to build GNOME. Peter.

Re: Built grep 2.5.3 successfully, or not?

Robert Blair — 2008-11-11T21:43:52

** Reply to message from "John Small" os2world.net> on Tue, 11 Nov 2008 18:45:01 +0500 I have also recently installed Smedley's build environment. I don't know. I installed both AUTOCONF and GLIBIDL335 from the sCS CD number 2. I appears that AUTOCONF is not needed because it is included with Smedley's build environment but GLIBIDL335 is not. GLIBIDL335 appeared to make a difference in what I was doing but it may have been something else I did. Smedley's build environment comes with "ash" but it did not always work for me. Dave Yeo recommended "sh" which works here. I used Smedley's gcc335.cmd and changed the disk assignments to match with my system and that works so far. I did not use "make check" so I do not know what my results would have been. I just started by diong the "make" and going from there. I know how you feel. Are my problems because I don't know what I am doing or are they real problems.

Re: Built grep 2.5.3 successfully, or not?

Steven Levine — 2008-11-11T16:34:06

In <200811110845.1kZTyl7093Nl34i0< at >mx-bracke.atl.sa.earthlink.net>, on 11/11/08 at 06:45 PM, "John Small" os2world.net> said: Hi, That's not atypical. You should be able to look at the makefiles and the scripts to verify this. They are just trying to make you feel good. You figure this out by looking at the failing test scripts. Some scripts are better than others at identifying the failing tests. I doubt it is anything you did. It's possible that the tests don't take line endings into account or there is something not quite right in the handling of path names. These are typical things we need to fix up for a production quality eCs/OS2 build. As always, general questions will get you general answers. You would have been well served to post the log for the first couple of tests that failed. Zip is your friend. How small is the zipped log file? If you feel this is still too large, you can always upload to Netlabs incoming and post a link. Recall that we often do this for fm/2 test

Built grep 2.5.3 successfully, or not?

John Small — 2008-11-11T13:45:01

In my continuing efforts to figure out how to port software I keep trying new source code packages to see if I can get anything to build. Most end in failure and I have not been able to figure out if the problem is - my build environment (Smedley's) - am I missing something? - do I have the wrong version of something? - which shell should I be using? - do I have all the necessary env. vars. set correctly? - my configuration - which options, if any, should I be passing to the configure scripts? - which "can't find" messages during configure are acceptable and which ones indicate an error which needs to be fixed. - which "checking for/whether xyz... no" messages during configure are acceptable and which ones indicate an error which needs to be fixed. - the source code itself - are there changes necessary to make it build under/for OS/2? I finally ran across some source which may have built successfully, grep-2.5.3 (): - running the c

Re: buildenv problem

Robert Blair — 2008-11-01T07:57:14

** Reply to message from "Robert Blair" listemail.net> on Fri, 31 Oct 2008 22:34:13 -0700 I found what the problem is. It appears that something was changed in the newer versions of the GCC compiler. Searching google for "does not give a valid preprocessing token" got the answer. Removing the ## in conf.h line 33 #define SYSCONFFILE SYSCONFDIR ## "/flipit.conf" solved that problem.