Discussion:
[UnixOS2] libc
Stefan Neis
2003-06-17 16:49:52 UTC
Permalink
Maybe moving that discussion to UnixOS2 or EMX list would
be more appropriate...
I vote for UnixOS2 list.
In view of Ilya's recent post to EMX list
Can't we find a new volunteer for maintainance of EMX.DLL and
EMXCRTM.DLL? The job is to *collect* contributed patches, weed out
the weeds, and decide when we have a snapshot good enough to be named
golded.
I decided to also involve emx mailing list into this discussion...
Just some comments from Achim (taken from #netlabs irc, btw hint hint,
IRC is a *good* way to communicate :-)
I like the asynchronity (??) of mail. Everyone can read/reply when he can
spare some minutes. ;-)
<achimha> well, our libc is done to fulfill our needs
<achimha> it is incompatible with emx (note by ktk: due to licensing
problems with EMX)
Huh? Which ones? I always had the impression that EMX's licence is rather
permissive...
<achimha> our GCC is targeted to be a VAC/Watcom replacement not a Unix
porting environment
<achimha> just wait for our first distribution then we can discuss how
to combine the various efforts
To be honest, that doesn't sound to promising from the porting point of
view. There are some things that _have_ to be deep inside libc, otherwise
e.g. closing sockets by close instead of closesocket or pthread support
won't work... :-(
For Unixos2-only people: Henry proposed to create a CVS for libc, now
the question is what do we want to take as base (EMX, LIBEMU and
Innotek-GCC libc is in dicussion, there is also one from IBM btw)
I think we should start with EMX, add Posix/2, some link support,
pthreads, libintl and maybe some other nice little extensions and then
start making it all work together and fixing bugs (e.g. pthread library
and library for symlink support (via EA) both modify certain functions
for accessing files - BTW, large file support is the third candidate which
"wants" specific changes in that same area).

Whether the CVS is rooted at netlabs or sourceforge (we could start by
abusing the Posix/2 CVS) is rather irrelevant to me, however it would be
great to automatically send all CVS commits (as a diff) to some dedicated
mailing list (I know that wxWindows did such a thing also while the CVS
was hosted by sourceforge, so it's possible to set it up like this even
there - but I don't know, how to do it), so whoever is interested has a
chance to check things easily.

The biggest problem for starting however is probably the build system:
EMX libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want
I don't know what etc. Unifying this and "porting" the Makefiles to some
current GNUmake is going to require much time for not much to gain... :-(

Ideally, we would also stay backward compatible, which implies keeping the
right ordering for the already existing symbols in the DLL's. (though I'd
suggest to make the import libraries for the new DLL's using "import by
name"). OTOH, I _suppose_ large file support if put into the "normal" C
routines is going to cause incompatibilities anyway (file positions no
longer fit into an int, do they?), so maybe it would even be a good idea
to be intentionally incompatible (old EMX DLL's handle small files, in the
new runtime, everything would be large file enabled - Hmm, I'll have to
get a new harddisk to create a large enough JFS partition for testing...).
It would be nice if we really could discuss that all together and agree
on a sollution that is fine for everyone and allows enhancements in the
future (no dead end).
What I don't know at all is, how this could fit into Holger's long term
plans. Ideally, replacing a couple of low level C routines by his more
complete/efficient implementations - once they are ready - would be
sufficient, but for this, it would be a big plus to have a rather
detailed list of what functions those are going to be, I suppose.
E.g. an improved link implementation is going to affect some other file
related functions ((f)open, (f)stat, ...), but if we know this in advance,
we can have some internal function like __dereference_link that's called
by all those, so the changes from a temporary E/A based link emulation to
"real" links are minimized...

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Adrian Gschwend
2003-06-17 18:39:16 UTC
Permalink
Post by Stefan Neis
In view of Ilya's recent post to EMX list
Can't we find a new volunteer for maintainance of EMX.DLL and
EMXCRTM.DLL? The job is to *collect* contributed patches, weed out
the weeds, and decide when we have a snapshot good enough to be named
golded.
I decided to also involve emx mailing list into this discussion...
ok, I'm not subscribed to the EMX list so it's possible that my posting
won't make it there.
Post by Stefan Neis
I like the asynchronity (??) of mail. Everyone can read/reply when he can
spare some minutes. ;-)
yes, fine for a lot of things but it would still be nice if at least
some people could show up on our #unixos2 channel. Would help in some
cases too :) As a reminder: Server is irc.anduin.net, Mozilla got an
IRC client built in.
Post by Stefan Neis
<achimha> well, our libc is done to fulfill our needs
<achimha> it is incompatible with emx (note by ktk: due to licensing
problems with EMX)
Huh? Which ones? I always had the impression that EMX's licence is rather
permissive...
Well I won't go into details, some Innotek people should answer that (
Knut promised me to go through the mails later)
Post by Stefan Neis
To be honest, that doesn't sound to promising from the porting point of
view. There are some things that _have_ to be deep inside libc, otherwise
e.g. closing sockets by close instead of closesocket or pthread support
won't work... :-(
I'm sure they will help us if we talk to each other.
Post by Stefan Neis
I think we should start with EMX, add Posix/2, some link support,
pthreads, libintl and maybe some other nice little extensions and then
start making it all work together and fixing bugs (e.g. pthread library
and library for symlink support (via EA) both modify certain functions
for accessing files - BTW, large file support is the third candidate which
"wants" specific changes in that same area).
fine for me.
Post by Stefan Neis
Whether the CVS is rooted at netlabs or sourceforge (we could start by
abusing the Posix/2 CVS) is rather irrelevant to me, however it would be
great to automatically send all CVS commits (as a diff) to some dedicated
mailing list (I know that wxWindows did such a thing also while the CVS
was hosted by sourceforge, so it's possible to set it up like this even
there - but I don't know, how to do it), so whoever is interested has a
chance to check things easily.
I can't provide the diff, so if you want that sourceforge is the
server. But to be honest I don't see the sense in that if we use CVS
anyway. But the cvs server is not really an important question as long
as it works :)
Post by Stefan Neis
EMX libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want
I don't know what etc. Unifying this and "porting" the Makefiles to some
current GNUmake is going to require much time for not much to gain... :-(
ok, can't comment that due the lack of know how in this area. But IMHO
it needs to be done, will help everyone at the end.
Post by Stefan Neis
What I don't know at all is, how this could fit into Holger's long term
plans. Ideally, replacing a couple of low level C routines by his more
complete/efficient implementations - once they are ready - would be
sufficient, but for this, it would be a big plus to have a rather
detailed list of what functions those are going to be, I suppose.
ACK, probably Holger can give some details about that?

cu

Adrian
--
Adrian Gschwend
@ netlabs.org

ktk [a t] netlabs.org
-------
Free Software for OS/2 and eCS
http://www.netlabs.org
Holger Veit
2003-06-17 19:08:14 UTC
Permalink
[...]
Post by Stefan Neis
I like the asynchronity (??) of mail. Everyone can read/reply when he can
spare some minutes. ;-)
I prefer mailinglists over IRC as well for the reason of asynchronicity.
Post by Stefan Neis
<achimha> well, our libc is done to fulfill our needs
<achimha> it is incompatible with emx (note by ktk: due to licensing
problems with EMX)
Huh? Which ones? I always had the impression that EMX's licence is rather
permissive...
The old GPL debate. GPL is not free, it is rather restricted. Too restricted
to be acceptable for people who need tailored version of the software.
Post by Stefan Neis
<achimha> our GCC is targeted to be a VAC/Watcom replacement not a Unix
porting environment
<achimha> just wait for our first distribution then we can discuss how
to combine the various efforts
To be honest, that doesn't sound to promising from the porting point of
view. There are some things that _have_ to be deep inside libc, otherwise
e.g. closing sockets by close instead of closesocket or pthread support
won't work... :-(
I don't see a problem so far, unless "VAC/Watcom" compatibility is meant
in a way to copy all of their quirks and bugs and anachronisms one by one.
In such a case one would have to separate close and closesocket, for instance.
OTOH, Unix compatibility also means to copy numerous broken concepts of
the Unix API which are better solved in the OS/2 API. EMX tried some
balance there with the effect that some Unix things work, and some OS/2
things work, and special cases don't, in both areas.
Post by Stefan Neis
For Unixos2-only people: Henry proposed to create a CVS for libc, now
the question is what do we want to take as base (EMX, LIBEMU and
Innotek-GCC libc is in dicussion, there is also one from IBM btw)
I think we should start with EMX, add Posix/2, some link support,
pthreads, libintl and maybe some other nice little extensions and then
start making it all work together and fixing bugs (e.g. pthread library
and library for symlink support (via EA) both modify certain functions
for accessing files - BTW, large file support is the third candidate which
"wants" specific changes in that same area).
Whether the CVS is rooted at netlabs or sourceforge (we could start by
abusing the Posix/2 CVS) is rather irrelevant to me, however it would be
great to automatically send all CVS commits (as a diff) to some dedicated
mailing list (I know that wxWindows did such a thing also while the CVS
was hosted by sourceforge, so it's possible to set it up like this even
there - but I don't know, how to do it), so whoever is interested has a
chance to check things easily.
Meanwhile, with rather limited time, I second that fully.
Post by Stefan Neis
EMX libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want
I don't know what etc. Unifying this and "porting" the Makefiles to some
current GNUmake is going to require much time for not much to gain... :-(
I had already started to move EMX to GNUmake (don't ask me which version,
I am typically off the recent porting attempt), but eventually messed this
up by trying, at the same time, to improve the EMX environment, e.g. by
throwing out DOSisms and the syscall interface through the INT21 like
register calling mechanism. What make the dmakefile build mechanism
complicated was the IMHO too complex interwoven system of dependencies
in order to build the various variants of linraries from a single source
tree. It needs some time to untangle this, but this is something that has
to be done before any further work.
Post by Stefan Neis
Ideally, we would also stay backward compatible, which implies keeping the
right ordering for the already existing symbols in the DLL's. (though I'd
suggest to make the import libraries for the new DLL's using "import by
You'll have to give up full compatibility eventually, no doubt about
this. Leave the old EMX*.DLLs in the system, and name the new ones EMU*.DLL
or anything else, and don't attempt to keep old code where you have better
new code. You do know that if you want to integrate libemu, you'll get a
clash on certain APIs. And then there is no need to keep the DLL ordinals
consistent with old EMX. They may be compatible in the first release, but
will diverge sooner or later. So might it right from the beginning (means:
name imports).
Post by Stefan Neis
name"). OTOH, I _suppose_ large file support if put into the "normal" C
routines is going to cause incompatibilities anyway (file positions no
You can move file I/O into a separate DLL, and provide large file I/O
by DosLoadModule where present, and normal I/O where not. This is a similar
thing like EM's TCP/IP support (in those days it was not clear whether
every OS/2 system had SO32DLL.DLL, now you would rely on this).
Post by Stefan Neis
longer fit into an int, do they?), so maybe it would even be a good idea
to be intentionally incompatible (old EMX DLL's handle small files, in the
new runtime, everything would be large file enabled - Hmm, I'll have to
get a new harddisk to create a large enough JFS partition for testing...).
It would be nice if we really could discuss that all together and agree
on a sollution that is fine for everyone and allows enhancements in the
future (no dead end).
What I don't know at all is, how this could fit into Holger's long term
plans. Ideally, replacing a couple of low level C routines by his more
complete/efficient implementations - once they are ready - would be
sufficient, but for this, it would be a big plus to have a rather
detailed list of what functions those are going to be, I suppose.
The API I am targeting is the man(2) API. This means there should be still
a separation of man(2) and man(3). This is more or less already implemented
in EMX, however with the mentioned syscall interface.
Post by Stefan Neis
E.g. an improved link implementation is going to affect some other file
related functions ((f)open, (f)stat, ...), but if we know this in advance,
we can have some internal function like __dereference_link that's called
by all those, so the changes from a temporary E/A based link emulation to
"real" links are minimized...
Look at the way it is done in "real" Unix. If the "link" code touches
any stream-I/O function, e.g. fstat() or fopen(), it is doing something
wrong. The link code should be based on lstat()/stat() and open(), and
in particular as close as possible to DosOpen and friends, because it
is very likely that a future DosOpen will already understand and handle
links. Implementing it in this way is IMHO the main challenge of
cleaning up EMX code. And don't worry about EA-based vs. "real" links;
once the symlink code is stabilized, there is a way to do the same in
kernel mode (with the side effect that these EAs become restricted
and protected unless handled by the correct functions).

Holger
Stefan Neis
2003-06-18 09:12:44 UTC
Permalink
Post by Holger Veit
Post by Stefan Neis
Huh? Which ones? I always had the impression that EMX's licence is rather
permissive...
The old GPL debate. GPL is not free, it is rather restricted. Too restricted
to be acceptable for people who need tailored version of the software.
Is it GPL for the library? And even if the C runtime DLL is GPL'ed, what's
the effect on executables that happen to be usable together with that
specific DLL?
Post by Holger Veit
EMX tried some
balance there with the effect that some Unix things work, and some OS/2
things work, and special cases don't, in both areas.
I'm under the impression that our aim would be to move that "balance"
closer to Unix, while InnoTek seems to try to move it closer to Win and
"native" OS/2.
Post by Holger Veit
Post by Stefan Neis
Whether the CVS is rooted at netlabs or sourceforge (we could start by
abusing the Posix/2 CVS) is rather irrelevant to me, however it would be
great to automatically send all CVS commits (as a diff) to some dedicated
mailing list (I know that wxWindows did such a thing also while the CVS
was hosted by sourceforge, so it's possible to set it up like this even
there - but I don't know, how to do it), so whoever is interested has a
chance to check things easily.
Meanwhile, with rather limited time, I second that fully.
By now, I wonder if "cvs diff" is that much more uncomfortable for a
"small" project like this...
Post by Holger Veit
complicated was the IMHO too complex interwoven system of dependencies
in order to build the various variants of linraries from a single source
tree. It needs some time to untangle this, but this is something that has
to be done before any further work.
BTW, we should also determine what libraries we do want to build. I'd
suggest to drop at least single-threaded libraries and maybe even static
ones...
Post by Holger Veit
Look at the way it is done in "real" Unix.
Well, I don't really know that much about "real" Unix libc internals...

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Holger Veit
2003-06-18 10:06:16 UTC
Permalink
Post by Stefan Neis
Post by Holger Veit
Post by Stefan Neis
Huh? Which ones? I always had the impression that EMX's licence is rather
permissive...
The old GPL debate. GPL is not free, it is rather restricted. Too restricted
to be acceptable for people who need tailored version of the software.
Is it GPL for the library? And even if the C runtime DLL is GPL'ed, what's
the effect on executables that happen to be usable together with that
specific DLL?
Don't try to debate on this; I do know differences between GPL and LGPL
and their implications. The point is: if some commercial customer says
NO to GPL, for whatever reason (even because of ignorance), then this
customer is away. Period. And this is IMHO exactly the motivation
why Innotek tries to avoid the buzzword GPL like the devil avoids holy
water. Understandable.
Post by Stefan Neis
Post by Holger Veit
EMX tried some
balance there with the effect that some Unix things work, and some OS/2
things work, and special cases don't, in both areas.
I'm under the impression that our aim would be to move that "balance"
closer to Unix, while InnoTek seems to try to move it closer to Win and
"native" OS/2.
My impression as well.
Post by Stefan Neis
Post by Holger Veit
complicated was the IMHO too complex interwoven system of dependencies
in order to build the various variants of linraries from a single source
tree. It needs some time to untangle this, but this is something that has
to be done before any further work.
BTW, we should also determine what libraries we do want to build. I'd
suggest to drop at least single-threaded libraries and maybe even static
ones...
Multithreaded, in flavours "optimized" and "debuggable". Everything else
is stuff to consider for later (read: if some complains about missing
stuff, then why shouldn't he/she add this to the project - YGWYPF).
Post by Stefan Neis
Post by Holger Veit
Look at the way it is done in "real" Unix.
Well, I don't really know that much about "real" Unix libc internals...
"Real" Unix as in *BSD and Linux ;-) There is a lot of stuff to see.

Holger
Andreas Buening
2003-06-17 19:48:05 UTC
Permalink
[snip]
Post by Stefan Neis
<achimha> our GCC is targeted to be a VAC/Watcom replacement not a Unix
porting environment
<achimha> just wait for our first distribution then we can discuss how
to combine the various efforts
To be honest, that doesn't sound to promising from the porting point of
view. There are some things that _have_ to be deep inside libc, otherwise
e.g. closing sockets by close instead of closesocket or pthread support
won't work... :-(
You're right. That sounds ... strange.
Post by Stefan Neis
For Unixos2-only people: Henry proposed to create a CVS for libc, now
the question is what do we want to take as base (EMX, LIBEMU and
Innotek-GCC libc is in dicussion, there is also one from IBM btw)
I think we should start with EMX, add Posix/2, some link support,
pthreads, libintl and maybe some other nice little extensions and then
start making it all work together and fixing bugs (e.g. pthread library
and library for symlink support (via EA) both modify certain functions
I think we'll have to start with more basic tasks like updating the
emx headers/libs for new standard functions. ;-)
I'd like to keep the standalone libs like intl and pthreads out of emx.
Lots of Unix system also have standalone versions of them so it's
more important to get them working than to integrate them into emx.
However, the details have to be discussed.
Post by Stefan Neis
for accessing files - BTW, large file support is the third candidate which
"wants" specific changes in that same area).
How do other systems solve this problem? Do they use an internal
fseek_64() replacement which is used by default instead of fseek()?


[snip]
Post by Stefan Neis
EMX libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want
I don't know what etc. Unifying this and "porting" the Makefiles to some
current GNUmake is going to require much time for not much to gain... :-(
Yes, this could become a serious problem. Maybe we have to port emx
to a new build system first. ;-)

[snip]
Post by Stefan Neis
It would be nice if we really could discuss that all together and agree
on a sollution that is fine for everyone and allows enhancements in the
future (no dead end).
What I don't know at all is, how this could fit into Holger's long term
plans.
Nobody knows, I guess.

[snip]


Bye,
Andreas
Henry Sobotka
2003-06-18 04:47:33 UTC
Permalink
I agree with Stefan's proposal that we start with EMX, i.e. create a
repository with 0.9d, fix 04: emx/include and emx/src is probably all we
need; though what about emx/bsd? With a vanilla EMX repository, people
who are already running patched versions can check out the tree, diff it
against their own EMX, patch their working directory and submit the
diffs for check-in.

It might also be worth creating a development branch from the outset,
and applying the patches to it. Then, whenever people feel there's
enough substance to warrant a release, merge with the trunk and deliver
a stable runtime.

For the build system, I suggest sticking with it as is for now, and
converting to GNU make in two stages: 1) translate the dmakese into
gmakese; 2) hack the GNU makefiles into the build system we want. This
way any lack of progress with the build system won't impede development
of the library's contents.

h~
Stefan Neis
2003-06-18 09:51:59 UTC
Permalink
Post by Henry Sobotka
I agree with Stefan's proposal that we start with EMX
I wonder if waiting for InnoTek's libc and starting with that wouldn't
be an easier approach. Some of the things I hear sound rather good, though
others don't really, but to form an opinion we'd probably really have to
have a look at it...
Post by Henry Sobotka
It might also be worth creating a development branch from the outset,
and applying the patches to it. Then, whenever people feel there's
enough substance to warrant a release, merge with the trunk and deliver
a stable runtime.
Sounds like a good idea, though it seems a bit unusual (to me at least) to
do it that way round.
Post by Henry Sobotka
For the build system, I suggest sticking with it as is for now, and
converting to GNU make in two stages: 1) translate the dmakese into
gmakese; 2) hack the GNU makefiles into the build system we want. This
way any lack of progress with the build system won't impede development
of the library's contents.
Sounds very good. Also that enables a nice separation of tasks, i.e. if
somebody is working on makefiles, he won't necessarily be bothered with
bugs in the evolving C code...

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Adrian Gschwend
2003-06-18 10:04:06 UTC
Permalink
Post by Stefan Neis
I wonder if waiting for InnoTek's libc and starting with that wouldn't
be an easier approach. Some of the things I hear sound rather good, though
others don't really, but to form an opinion we'd probably really have to
have a look at it...
Achim recommends to wait, he told there might be a release before the
end of month.

cu

Adrian
--
Adrian Gschwend
@ netlabs.org

ktk [a t] netlabs.org
-------
Free Software for OS/2 and eCS
http://www.netlabs.org
Stefan Neis
2003-06-18 09:22:51 UTC
Permalink
Post by Andreas Buening
I'd like to keep the standalone libs like intl and pthreads out of emx.
Lots of Unix system also have standalone versions of them so it's
more important to get them working than to integrate them into emx.
However, the details have to be discussed.
Well, the problem is that pthreads is not really a standalone library,
currently it redefines a couple of file related functions to its own
extended version, which causes some trouble if you are using stuff like
some_class::open. If one tries to clean this up, one ends up with more
dependencies on linking order (i.e. you have to link pthreads library
prior to other libraries defining file I/O functions).
Post by Andreas Buening
How do other systems solve this problem? Do they use an internal
fseek_64() replacement which is used by default instead of fseek()?
There typically is fseek() (32-bit version) and fseek_64().
If -D_FILE_OFFSET_BITS=64 is given on compilation, fseek() is typically
"somehow" mapped to fseek_64(). And you better avoid mixing the two in
the same binary. (i.e. either compile _everything_ (including every helper
library) with that library, or nothing). Similar to the problems
with single- and multi-threaded libc in EMX. That's why I suggested to
have only the large file enabled version in a new library. It would
eliminate all the problems of unintentionally mixing the two variants.

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Holger Veit
2003-06-18 10:21:46 UTC
Permalink
[...]
Post by Stefan Neis
Post by Andreas Buening
How do other systems solve this problem? Do they use an internal
fseek_64() replacement which is used by default instead of fseek()?
There typically is fseek() (32-bit version) and fseek_64().
If -D_FILE_OFFSET_BITS=64 is given on compilation, fseek() is typically
"somehow" mapped to fseek_64(). And you better avoid mixing the two in
the same binary. (i.e. either compile _everything_ (including every helper
library) with that library, or nothing). Similar to the problems
with single- and multi-threaded libc in EMX. That's why I suggested to
have only the large file enabled version in a new library. It would
eliminate all the problems of unintentionally mixing the two variants.
The fseek vs fseek64 is yet another example of broken Unix API design.
Effectively, this revolves around the problem whether off_t has been
designed as 32 bit (as usual) or as 64 bit right from the beginning.

Since most Unixes started broken by concept with the old 'long' offset
type, this had to be "fixed" later when designers "suddenly" detected that
their Unix filesystems were designed for Terabytes of storage and
disks meanwhile approached the size of 2GB (which is ridiculously small
nowadays). Result was this awkward *_64() patches which required specific
programming and compilation.

We only have to preserve compatibility in that we need some fseek() and
fseek_64() at all, not with binary compatibility with existing code
(this can still access the old EMX DLLs and will be happy).

If we need to touch the header files already then my proposal is to
typedef longlong off_t;
and prototype
int fseek(FILE*,off_t,int);
and implement
int fseek_64(FILE* f,off_t off,int whence) { return fseek(f,off,whence); }

The fseek code will itself fall down to a 64 bit compliant lseek(), and
this will, depending on the presence of Dos*L routines or not (loadable
DLL module) decide whether to accept the call or not if an offset >2GB is
specified.

Holger
Stefan Neis
2003-06-18 13:53:37 UTC
Permalink
Post by Holger Veit
If we need to touch the header files already then my proposal is to
typedef longlong off_t;
and prototype
int fseek(FILE*,off_t,int);
and implement
int fseek_64(FILE* f,off_t off,int whence) { return fseek(f,off,whence); }
The fseek code will itself fall down to a 64 bit compliant lseek(), and
this will, depending on the presence of Dos*L routines or not (loadable
DLL module) decide whether to accept the call or not if an offset >2GB is
specified.
That's what I had in mind as well. And maybe it's also what InnoTek did
do (though I suppose they would have completely ommited the fseek_64).
BTW, is there a mail loop? I keep getting my own mail about GPL again and
again ...

Regards,
Stefan
Andreas Buening
2003-06-18 17:48:49 UTC
Permalink
Holger Veit wrote:

[snip]
Post by Holger Veit
We only have to preserve compatibility in that we need some fseek() and
fseek_64() at all, not with binary compatibility with existing code
(this can still access the old EMX DLLs and will be happy).
I can understand why you'd like to break binary compatibility.
However, I have a bad feeling about doing this. Some libraries
do this sometimes and this leads to a set of incompatible
binary files which can become quite nasty. I'd like to keep also
binary compatibility.

[snip]


Bye,
Andreas
Holger Veit
2003-06-19 11:43:39 UTC
Permalink
Post by Andreas Buening
[snip]
Post by Holger Veit
We only have to preserve compatibility in that we need some fseek() and
fseek_64() at all, not with binary compatibility with existing code
(this can still access the old EMX DLLs and will be happy).
I can understand why you'd like to break binary compatibility.
Sure? I am curious...

To be precise: there are several definitions of binary (in-)compatibility,
so it is important to negotiate what we are talking about.

One is the question of ELF vs. a.out.
Another is ordinals vs. named DLL entries.
A third is g++ 2.8.x vs. 2.95 vs. 3.x.
The fourth is about API compatibility, as above the example of whether
off_t is 32 bit or 64 bit.
A fifth is the question whether some EMX successor should preserve all
older entry points.

And there are probably more stumbling blocks in the way that make the
compatibility issue a minefield.

Each of the above has the potential to break something old. The real
question behind the "binary compatibility" killer argument is:

Do we really want an improvement of the free OS/2 libc (call it EMX or
different)?

If the answer is yes, then we *must* break something in order to
build something better, faster,more powerful (whatever the design goal
is). This is the unavoidable price to pay.

If the answer is no...well, German proverb tells us the problem:
Wash my fur, but don't make it wet. It is impossible, and then there is
no use in any further discussion on the topic (which is almost as old
as the whole UnixOS2 movement).
Post by Andreas Buening
However, I have a bad feeling about doing this. Some libraries
do this sometimes and this leads to a set of incompatible
binary files which can become quite nasty. I'd like to keep also
binary compatibility.
So, I'll put up the clear question above again:

Do you want an improvement? YES or NO?

Holger
Andreas Buening
2003-06-19 15:15:08 UTC
Permalink
Post by Holger Veit
Post by Andreas Buening
[snip]
Post by Holger Veit
We only have to preserve compatibility in that we need some fseek() and
fseek_64() at all, not with binary compatibility with existing code
(this can still access the old EMX DLLs and will be happy).
I can understand why you'd like to break binary compatibility.
Sure? I am curious...
To be precise: there are several definitions of binary (in-)compatibility,
so it is important to negotiate what we are talking about.
One is the question of ELF vs. a.out.
Another is ordinals vs. named DLL entries.
A third is g++ 2.8.x vs. 2.95 vs. 3.x.
The fourth is about API compatibility, as above the example of whether
off_t is 32 bit or 64 bit.
A fifth is the question whether some EMX successor should preserve all
older entry points.
Binary compatibility means for me you can replace a DLL and your
programs still work. This includes API compatibility and the
preservation of the ordinal numbers (if you've ever used them).
C++ is another story. We can't do anything about the fact that the
g++ people change name mangling with every release (and that's
the reason why C is used for the libc and why the libc is called
lib_C_).

[snip]
Post by Holger Veit
Do we really want an improvement of the free OS/2 libc (call it EMX or
different)?
Yes.
Post by Holger Veit
If the answer is yes, then we *must* break something in order to
build something better, faster,more powerful (whatever the design goal
is). This is the unavoidable price to pay.
This is the point I'm not totally convinced of. I agree that you have
much more knowlegde of the emx and OS/2 internals than me and, most
likely, than most people on this list. I do not say we must not
do an incompatible change but I'm very sceptical about it.

Sometimes I get the impression (I'm not talking about you, I'm just
generally speaking) that some people are going through the world
wearing a huge axe and taking no care of compatibility problems other
people might have. Nobody ever needs this feature so let's kill it.
Switching arguments of function foo() or adding some members to
structure bar would perform much better so let's change it.
xyz is a much better solution to that problem so let's remove that
option. If people really need it they can easily modify their bla
files. Who cares? This solution is much more beautyful. And, if
in doubt, they recompile everything.
Result? A huge bunch of incompatible stuff.

If we have to introduce an incompatibility into emx then we may do
it only once and never(*) again. And if anybody thinks six months
or a year later that xyz could be solved better by foo or nobody
needs option bar then the answer must be: Keep compatibility!
Otherwise we'll just get a bunch of (more or less) useless libraries.


(*) Never in this sense means never within this computer age which
is equivalent to 10 years in real time. ;-)


[snip]


Bye,
Andreas
Ilya Zakharevich
2003-06-20 09:29:57 UTC
Permalink
Post by Andreas Buening
If we have to introduce an incompatibility into emx then we may do
it only once and never(*) again. And if anybody thinks six months
or a year later that xyz could be solved better by foo or nobody
needs option bar then the answer must be: Keep compatibility!
Otherwise we'll just get a bunch of (more or less) useless libraries.
This depends. If the feature requires only a small amount of code, we
can just "duplicate" the code, and re#define the function to different
name.

Then a recompile of your application gives you a better mousetrap.
[Redefinition is needed since some of the libraries which use the old
mousetrap come as static libraries; with just changing an ordinal they
need a recompile too.]

But I do not see why new functions should come into the picture at
all. The situation is simple:

a) EMX needs to be *fixed*; it has many problems, and most of these
problems are quite easy to fix.

[Actually, IMO fixing these problems is *much* less
time-consuming than participation in endless arguments whether
one wants to fix these problems. This is why I insist that EMX
needs a new maintainer.]

b) There is no need to *extend* EMX/EMXCRTM other than features
*allowing* extension of EMX by third-parties software - and
support for new builtins of new gcc's.

[It would be very nice if these pieces were merged into as few
DLLs as possible. But this is an *eventual* target; one can add
a wait period to avoid the scenario you mention - and merge
things into CRT DLL after the wait expires.]

Hope this helps,
Ilya
nickk
2003-06-20 06:04:56 UTC
Permalink
Post by Holger Veit
Do you want an improvement? YES or NO?
Well, we all DID want an improvement for several years. But now all we have is a bitching bettween os/2 programmers and no other libc besides old
EMX. I dont beleive that we will ever have smth better :(
Holger Veit
2003-06-20 08:10:14 UTC
Permalink
Post by nickk
Post by Holger Veit
Do you want an improvement? YES or NO?
Well, we all DID want an improvement for several years. But now all we have is a bitching bettween os/2 programmers and no other libc besides old
EMX. I dont beleive that we will ever have smth better :(
This brings it close to the point. Besides, this thread is again starting to
shift, in a similar way as many other threads, namely on the issue of
"what make should we use?" This comes up and up again, without any result
at all. Guys, this is a non-debate actually, which is only wasting time
and already has wasted very long time. There is already some make.exe which
apparently does quite well with automake and also with Imake. This one
should be in the UnixOS2 archive, so there is no reason that not everyone
can obtain it and use it. This versioning mess (only the latest greatest
number is the best - neglecting that each version is ported again to OS/2
resulting in new "features" and bugs) is really not necessary.

I think the whole problem we have over the years is that noone really wants
to adopt any responsibility any more: responsibility for some major
design decision like the build system, even the name of a successor library,
the question of compatibility (which to me is the strongest indicator
of -negative- conservativism: it shoud be better, but never touch anything
that might break antediluvian code), and now, even about the decision,
whether make-3.76.1 or 3.77.2 or 3.79.3.141592653589.... is appropriate.
Maybe we should dig deeper into the make source code - there are lots
of lines of code (and even more worthwhile: lines of comment) we could
debate about (NOT!). This is meanwhile starting to become (philo-)sophism,
not programming.

I have more than once in the mailinglist and elsewhere recommended to
"just do it" (Cpt Picard: Engage) if there is some design decision.
I am tired to lead such decisions (have done it for long with XFree86/OS2);
you should be clear about the fact that this is a community effort.

I am not surprised that now Innotek wants to come out with a gcc environment -
they *have* made some design decisions which now, according to some rumour,
don't look too promising for Unix compatibility. It was Innotek's good right
to decide this; they are paying the port after all, and they probably had
some reasons to make their libc the way it is going to be. The good reasons
to make it (likely) different are here with us as well, yet noone so far
stepped forward to decide: this is for now the way to go (well: few prominent
and highly appreciable exceptions: libemu and symlinks, for instance).

You (as the remaining community) have IMHO still several choices left:
- continue to use EMX with all its deficiencies and problems, and never
get a satisfactory Unix environment
- hope for Innotek's libc, maybe it is more usable (rumour says no, but
I have yet to see any line of code before I get a definite opinion)
- finally leave all the philosophical blurb behind and MAKE UNIXOS2
- Wait for Godot

Holger
Post by nickk
_______________________________________________
Emx mailing list
http://xfreeos2.dyndns.org/mailman/listinfo/emx
Ilya Zakharevich
2003-06-20 09:15:34 UTC
Permalink
Post by Holger Veit
Do we really want an improvement of the free OS/2 libc (call it EMX or
different)?
If the answer is yes, then we *must* break something in order to
build something better, faster,more powerful (whatever the design goal
is). This is the unavoidable price to pay.
Now try to prove this. AFAIU, your argument looks very much like bullying.

Hope this helps,
Ilya
Holger Veit
2003-06-20 12:13:34 UTC
Permalink
Post by Ilya Zakharevich
Post by Holger Veit
Do we really want an improvement of the free OS/2 libc (call it EMX or
different)?
If the answer is yes, then we *must* break something in order to
build something better, faster,more powerful (whatever the design goal
is). This is the unavoidable price to pay.
Now try to prove this. AFAIU, your argument looks very much like bullying.
Proof by example:

If we set off_t to longlong, structures like struct stat will change.
We could get around with a specific distinction of struct stat and
struct stat64, forcing each one to explicitly request the 64 bit APIs.
It is unlikely to happen, so "old" code will still not be able to take
advantage of JFS or other modern WSEB filesystems. One could argue that
we would not need support for this then anyway. So why should we change
anything then? Stay with the bugs and problems of EMX.

There are some more incompatibilities to occur; let's assume setuid()
and chown()/chmod() would get implemented; thus an awful lot of software
would suddenly underlie file protection effects, including the effect
that former NO OPs (like setuid()) that simply returned 0 for success,
will now fail. Not only that, some eager porters have used
#define setuid() (0)
or similar to get code work; suddenly such software would subtly fail,
because the process does not get the appropriate privilege and the
original check whether the process gained setuid privilege from the
Unix code which would fail on a real Unix is disabled due to that
(0) return. Consequence: if you want to avoid subtle breaking of the
code, just avoid putting such code in. Stay with the bugs and problems
of EMX.

If this is not sufficient as a proof, I cannot help you. Besides:
I don't think anyone needs backward compatibility at all for a DLL which
is then not even named EMX*.DLL even if it stemmed from EMX and maybe
shared 90% of the code. Old apps can still resolve against old EMX*.DLLs,
and for new apps one has to clearly decide what he wants - again, see
other posting a question of decision making which is fatally flawed here:
Do you want Unix or EM(i)X? Requesting EMUNIX or UNEMIX is not the answer -
this will never happen.

Holger


Holger
Ilya Zakharevich
2003-06-20 13:20:19 UTC
Permalink
Post by Holger Veit
Post by Ilya Zakharevich
Post by Holger Veit
Do we really want an improvement of the free OS/2 libc (call it EMX or
different)?
If the answer is yes, then we *must* break something in order to
build something better, faster,more powerful (whatever the design goal
is). This is the unavoidable price to pay.
Now try to prove this. AFAIU, your argument looks very much like bullying.
It is by example, it is not a proof. [Not mentioning that the example
is flawed.]
Post by Holger Veit
If we set off_t to longlong, structures like struct stat will change.
We could get around with a specific distinction of struct stat and
struct stat64, forcing each one to explicitly request the 64 bit APIs.
Correct, without this backward compatibility breaks.
Post by Holger Veit
It is unlikely to happen
What is "it"? People putting -DEMX_LARGE_FILES into GCCOPT?
Post by Holger Veit
so "old" code will still not be able to take
advantage of JFS or other modern WSEB filesystems.
To take advantage of large files, one needs to use DosOpenL(). Using
L-version of seek()/tell() is secondary; moreover, no EMX_LARGE_FILES
is needed to support applications which do not need to seek far.
Post by Holger Veit
There are some more incompatibilities to occur; let's assume setuid()
and chown()/chmod() would get implemented
I cannot comment on this, since I do not know the semantic of
ownership API on OS/2. But I see all the reason for these arguments
to be as flawed as your arguments above.
Post by Holger Veit
thus an awful lot of software would suddenly underlie file
protection effects, including the effect that former NO OPs (like
setuid()) that simply returned 0 for success, will now fail.
Is setuid() implemented by EMX? Do you say it did not return -1 when
uid != 0?
Post by Holger Veit
Not only that, some eager porters have used
#define setuid() (0)
or similar to get code work; suddenly such software would subtly fail,
because the process does not get the appropriate privilege and the
original check whether the process gained setuid privilege from the
Unix code which would fail on a real Unix is disabled due to that
(0) return. Consequence: if you want to avoid subtle breaking of the
code, just avoid putting such code in. Stay with the bugs and problems
of EMX.
I do not follow your arguments. Did you say:

There are some broken ports lying around. Recompiling such a port
with new, more correct, headers/libraries will require some work.

If yes, I do not consider this as a valid argument against correcting
headers/libraries.
Post by Holger Veit
If this is not sufficient as a proof, I cannot help you.
Well, I did not ask for your help. I already know your
any-backward-compatibilities stand; many people on this list
understand how wrong and destructive such a stand may be. I just
wanted to inform *other* people on this list that there is "one more"
people who does not share this stand.
Post by Holger Veit
I don't think anyone needs backward compatibility at all for a DLL which
is then not even named EMX*.DLL even if it stemmed from EMX and maybe
shared 90% of the code.
Absolutely. But my point is that a backward-compatible DLL which
*does* share the name will be many many times more useful - and will
require only minimal work to set up.
Post by Holger Veit
Do you want Unix or EM(i)X?
Sorry, I do not know what is meant by any one of these terms here.
IMO, one of the most devastating point of views of EM's is the
opposition of "OS/2 applications" and "Unix applications" as two
targets for use of EMX. For me these terms do not mean anything.

We are discussing APIs. The targets are:

a) Have APIs conform to the documentation as much as possible;

b) Have the documentation conform to standards as much as possible;

c) Have support for as many APIs' entry points as possible.

I just do not see where the opposition of Unix/OS2/EMX comes into this
picture.

Hope this helps,
Ilya
Holger Veit
2003-06-21 20:39:58 UTC
Permalink
Post by Ilya Zakharevich
Post by Holger Veit
Post by Ilya Zakharevich
Post by Holger Veit
Do we really want an improvement of the free OS/2 libc (call it EMX or
different)?
If the answer is yes, then we *must* break something in order to
build something better, faster,more powerful (whatever the design goal
is). This is the unavoidable price to pay.
Now try to prove this. AFAIU, your argument looks very much like bullying.
It is by example, it is not a proof. [Not mentioning that the example
is flawed.]
Not flawed. Proof by example is valid to counter an assertion.
Post by Ilya Zakharevich
Post by Holger Veit
If we set off_t to longlong, structures like struct stat will change.
We could get around with a specific distinction of struct stat and
struct stat64, forcing each one to explicitly request the 64 bit APIs.
Correct, without this backward compatibility breaks.
Post by Holger Veit
It is unlikely to happen
What is "it"? People putting -DEMX_LARGE_FILES into GCCOPT?
If you need to recompile stuff anyway, there is no reason to remain
compatible to old conventions.
Post by Ilya Zakharevich
Post by Holger Veit
so "old" code will still not be able to take
advantage of JFS or other modern WSEB filesystems.
To take advantage of large files, one needs to use DosOpenL(). Using
L-version of seek()/tell() is secondary; moreover, no EMX_LARGE_FILES
is needed to support applications which do not need to seek far.
Effectively, DosOpenL *is* DosOpen *is* DosOpen2 *is* Dos16Open. Follow the
traces in the kernel. DosSetFilePtrL is essential, as part of its code is
used in DosRead (which is DosReadL). You can also persuade HPFS to use
a different blocking size to skip over the 2GB limit.

But I guess it is useless to explain you anything, as you are unwilling
to accept anything than your own opinion.
Post by Ilya Zakharevich
Post by Holger Veit
There are some more incompatibilities to occur; let's assume setuid()
and chown()/chmod() would get implemented
I cannot comment on this, since I do not know the semantic of
ownership API on OS/2. But I see all the reason for these arguments
to be as flawed as your arguments above.
You are repeating yourself. Proof by repetition?
Post by Ilya Zakharevich
Post by Holger Veit
thus an awful lot of software would suddenly underlie file
protection effects, including the effect that former NO OPs (like
setuid()) that simply returned 0 for success, will now fail.
Is setuid() implemented by EMX? Do you say it did not return -1 when
uid != 0?
misc/setuid.c return -1 (EINVAL).

This is formally correct, but unacceptable for working ports. The pair
getuid/setuid may return or expect uid=0 all the time, but not all software
delivers this uid. Also code the changes the uid (seteuid/reuid, etc.)
might later expect that the returned is no longer 0. The whole system
works for nice weather conditions only. Result for porting: short-circuit
these functions and adjust stuff as needed.
Post by Ilya Zakharevich
Post by Holger Veit
Not only that, some eager porters have used
#define setuid() (0)
or similar to get code work; suddenly such software would subtly fail,
because the process does not get the appropriate privilege and the
original check whether the process gained setuid privilege from the
Unix code which would fail on a real Unix is disabled due to that
(0) return. Consequence: if you want to avoid subtle breaking of the
code, just avoid putting such code in. Stay with the bugs and problems
of EMX.
There are some broken ports lying around. Recompiling such a port
with new, more correct, headers/libraries will require some work.
If yes, I do not consider this as a valid argument against correcting
headers/libraries.
If you change headers, you modify semantics. I don't consider this as
preserving compatibility anymore. Either the change has no effect on
existing code - then why would you do this (other than adding a new
function), or it has.
Post by Ilya Zakharevich
Post by Holger Veit
If this is not sufficient as a proof, I cannot help you.
Well, I did not ask for your help. I already know your
any-backward-compatibilities stand; many people on this list
understand how wrong and destructive such a stand may be. I just
wanted to inform *other* people on this list that there is "one more"
people who does not share this stand.
Then solve the EMX problem yourself. As you were asking for - clearly
spoken - for a *victim* who should take over EMX so that you can add *your*
ideas and changes, don't count on me on the issue of EMX changes (which
should not change anything) any longer. You will never come to any
conclusion this way.
Post by Ilya Zakharevich
Post by Holger Veit
I don't think anyone needs backward compatibility at all for a DLL which
is then not even named EMX*.DLL even if it stemmed from EMX and maybe
shared 90% of the code.
Absolutely. But my point is that a backward-compatible DLL which
*does* share the name will be many many times more useful - and will
require only minimal work to set up.
Feel free to do so. Be aware that you need some considerable modifications
alone to get EMX to work *correctly* with any gcc > 2.8 (or 2.9). You
might have a small look at some files in the source tree with the
innocent extension .cc (exception and new handling). Keep them in there,
and you need at least for this two compilers (2.8.1 and 3.x), or leave
them out (won't link without removing ordinals), or change them to
the libgcc-3.x.a versions (3.x will short-circuit them in EMX*.DLL
with the own libgcc.a) and break API compatibility.

This can be ironed out, of course, but don't expect anything from me
on this anymore.
Post by Ilya Zakharevich
Post by Holger Veit
Do you want Unix or EM(i)X?
Sorry, I do not know what is meant by any one of these terms here.
IMO, one of the most devastating point of views of EM's is the
opposition of "OS/2 applications" and "Unix applications" as two
targets for use of EMX. For me these terms do not mean anything.
Too bad for you. Meanwhile, even Windows has almost managed, thanks
to large disk partitions, that the only drive letters the ordinary
user knows nowadays are C: for the disk and D: for the CDROM (A: is
obsolete, and USB or other external disk devices are handled by some
obscure software). Since the CD is now merely a container for newly
to be installed software (music is again not handled as files), one can,
without restricting generality, say that Windows no longer has drive
letters. It understands them somewhere, and if you explicitly ask for
them, you can still find them, but it won't need them anymore. Some
day they will disappear like a usable DOS box, and noone will miss them.

OS/2 is this way no more advanced than CP/M almost 20 years ago which is
itself a cheap clone of certain DEC operating systems; cheap in
the way that the underlying conventions have been copied without
the more fundamental concepts.

No one will seriously use EMX for writing "OS/2" software anymore; if at
all, programmers keep their copies of VAC++ or Watcom, or maybe even
CSET/2 and MASM for that (or stay entirely out of the C/C++ ara and use
Java, for instance - ignored here). If there are still efforts, they
are concentrating on porting work. Guess where the most important repository
for portable software is. So, what tools does one need?
Post by Ilya Zakharevich
a) Have APIs conform to the documentation as much as possible;
b) Have the documentation conform to standards as much as possible;
c) Have support for as many APIs' entry points as possible.
I just do not see where the opposition of Unix/OS2/EMX comes into this
picture.
ANSI != POSIX.

a) and b) is secondary, nice to have, but c) helps the programmer. All the
experiments so far aimed at helping the programmer to get some nice
software from Unix to OS/2. Now you are proposing that it shoudn't be too
easy for the programmer, because he mustn't use an optimally tailored tool
for the purpose, but one that is religionusly or politically correct, because
it is "compatible".

Thank you.

Holger
Andreas Buening
2003-06-22 11:06:00 UTC
Permalink
[snip]
Post by Holger Veit
Post by Ilya Zakharevich
Post by Holger Veit
If we set off_t to longlong, structures like struct stat will change.
We could get around with a specific distinction of struct stat and
struct stat64, forcing each one to explicitly request the 64 bit APIs.
Correct, without this backward compatibility breaks.
Post by Holger Veit
It is unlikely to happen
What is "it"? People putting -DEMX_LARGE_FILES into GCCOPT?
If you need to recompile stuff anyway, there is no reason to remain
compatible to old conventions.
What do you want to say? That compatibility is not important? That you
can change the specifications as long as you can "recompile" the code?
It's okay if the Mozilla build in March 2004 requires the libc version
from January 2004 while the file utilities require the incompatible
version from September 2003? Because you can recompile it with gcc
which needs the libc from November 2003? Great idea, especially for
software which need 20 special tools for recompilation you've never
heard from. So you can either change BEGINLIBPATH for every program
or you download the 200 MB build package every month and compile all
the stuff for 2 days non stop?
If it's absolutely necessary to break compatibility we can do it once
and only once. And not again 4 months later when we switch to ELF
and again 3 months later if a new incompatible link handling scheme
is implemented. 99% of all OS/2 user don't have the time and the
motivation to reinstall basically their whole system just for one
new program that requires a new and incompatible libc.

[snip]
Post by Holger Veit
Post by Ilya Zakharevich
Post by Holger Veit
I don't think anyone needs backward compatibility at all for a DLL which
is then not even named EMX*.DLL even if it stemmed from EMX and maybe
shared 90% of the code.
Absolutely. But my point is that a backward-compatible DLL which
*does* share the name will be many many times more useful - and will
require only minimal work to set up.
Feel free to do so. Be aware that you need some considerable modifications
alone to get EMX to work *correctly* with any gcc > 2.8 (or 2.9). You
might have a small look at some files in the source tree with the
innocent extension .cc (exception and new handling). Keep them in there,
and you need at least for this two compilers (2.8.1 and 3.x), or leave
them out (won't link without removing ordinals), or change them to
the libgcc-3.x.a versions (3.x will short-circuit them in EMX*.DLL
with the own libgcc.a) and break API compatibility.
I don't know why EMX needs C++ internally. If this depends on the
gcc version we can't do much about it and C++ is absolutely not
related to our Posix discussion. We could e.g. consider to put the
compiler version dependent stuff into a static library.
Post by Holger Veit
This can be ironed out, of course, but don't expect anything from me
on this anymore.
Post by Ilya Zakharevich
Post by Holger Veit
Do you want Unix or EM(i)X?
Sorry, I do not know what is meant by any one of these terms here.
IMO, one of the most devastating point of views of EM's is the
opposition of "OS/2 applications" and "Unix applications" as two
targets for use of EMX. For me these terms do not mean anything.
Too bad for you. Meanwhile, even Windows has almost managed, thanks
to large disk partitions, that the only drive letters the ordinary
You mean Windows has to be installed on drive C: or 99% of all Winxx
users don't know that they can use another drive letter for this?
(Can they? I'm not familiar with XP)
Post by Holger Veit
for the disk and D: for the CDROM (A: is
obsolete,
You mean the standard Windows user has exactly one hard disk and one
CDROM? No Network drive, DVD drive, MO drive, ZIP drive? What do they
do with their old hard disk if they buy a new one? They throw it into
the basket? Who prevents you from installing OS/2 onto your only
C: partition?
Post by Holger Veit
and USB or other external disk devices are handled by some
obscure software).
In other words USB an other external devices are accessed by some
proprietary software using an proprietary API? And if the vendor
of the devices and the vendor of your OS isn't interested in these
devices any more you can't use them any more? Great deal. I knew
Winxx uses a superior technology but I didn't know it's _that_
superior.
Post by Holger Veit
Since the CD is now merely a container for newly
to be installed software (music is again not handled as files), one can,
without restricting generality, say that Windows no longer has drive
letters.
You mean the "default Winxx user" doesn't know how and why to create
a separate partition for programs, games and data? You mean 99%
of all Winxx users throw away their old hard disk if they buy a new
one? Because using d: for the 2nd hard disk would be evil?
Post by Holger Veit
It understands them somewhere, and if you explicitly ask for
them, you can still find them, but it won't need them anymore. Some
day they will disappear
You mean one of the next Winxx releases will do an incompatible change
and remove the drive letters? And all old applications that expect
a drive letter will fail so that users will have to buy new ones?
Sounds like Microsoft. :-(
Post by Holger Veit
like a usable DOS box,
Usable DOS box. That's an oxymoron, isn't it? ;-)
You mean one of the next Winxx releases will have no shell where
you could enter commands directly? I'm impressed. That's a real
proof of superiority.
Post by Holger Veit
and noone will miss them.
You mean 99% of all Winxx users won't use any applications or games
that are older than 2 years? Might be.
Post by Holger Veit
OS/2 is this way no more advanced than CP/M almost 20 years ago which is
itself a cheap clone of certain DEC operating systems; cheap in
the way that the underlying conventions have been copied without
the more fundamental concepts.
You're right. Winxx is much superior. When did they stop to sell
the last DOS clone WinME containing the same 20 year old bugs?
Post by Holger Veit
No one will seriously use EMX for writing "OS/2" software anymore; if at
If I got 1 cent for every time somebody said "Nobody ever needs xyz" ...
Post by Holger Veit
all, programmers keep their copies of VAC++ or Watcom, or maybe even
CSET/2 and MASM for that (or stay entirely out of the C/C++ ara and use
Java, for instance - ignored here). If there are still efforts, they
are concentrating on porting work. Guess where the most important repository
for portable software is. So, what tools does one need?
Oh, sure, taking into account the great VAC++ support by IBM changes
everything. Oh, wait, was there any support by IBM at all? And Watcom,
great deal. Everybody who used to use EMX may rewrite his code for
Watcom if he wants to be up to date to current C standards. Great.
Post by Holger Veit
Post by Ilya Zakharevich
a) Have APIs conform to the documentation as much as possible;
b) Have the documentation conform to standards as much as possible;
c) Have support for as many APIs' entry points as possible.
I just do not see where the opposition of Unix/OS2/EMX comes into this
picture.
ANSI != POSIX.
Not to forget 0 != 1. And this proves exactly what?
ANSI is the rudimentary C standard all compilers should comply to.
Posix is a Unix-like system independent API. Why should a compiler
not be able to comply to both standards?
Post by Holger Veit
a) and b) is secondary,
Ehm, hello? If the API doesn't conform to the documentation then the
API is mostly useless, and if the docs do not conform to any standard
then it's nearly impossible to write portable programs.
Post by Holger Veit
nice to have, but c) helps the programmer. All the
experiments so far aimed at helping the programmer to get some nice
software from Unix to OS/2. Now you are proposing that it shoudn't be too
easy for the programmer, because he mustn't use an optimally tailored tool
for the purpose, but one that is religionusly or politically correct, because
it is "compatible".
Hello? The presence of an _chdir2 entry point prevents the programmer
in porting Unix applications?


If you think some smileys are missing in this posting you're right.
At some point I didn't know whether I should laugh or cry. Maybe you're
right, maybe not. Maybe you've had this discussion many times but
there are still many people who aren't convinced. And I think I'll
speak for the majority of the people on these mailing lists if I say
that emx (or whatever its future name will be) must be able to compile
classical OS/2 programs _as_ _well_ _as_ ports of Unix software.
We can't know how the Watcom project will go and lots of people
want to write portable programs which is easier with emx than with Watcom
(from what other people said, I myself don't use Watcom).
And second, there must be long term compatibility for programs.


Bye,
Andreas
Stefan Neis
2003-06-22 16:05:18 UTC
Permalink
Post by Andreas Buening
It's okay if the Mozilla build in March 2004 requires the libc version
from January 2004 while the file utilities require the incompatible
version from September 2003?
Even though, I wouldn't like such a situation, where is the essential
problem _if_ all those libc version are in differently named DLL's?
OK, allocating/freeing memory is going to be the same kind of nightmare
that it currently is on Windows, if you mix libraries/DLL's using
different libc versions and don't carefully choose which libc you want to
use, but that's basically a problem known on OS/2 as well (just try to
have some function from a VAC++ compiled library allocate some memory and
free it in an EMX compiled executable. Hm, actually, that might be less
of a problem then I believe ...)
Post by Andreas Buening
I don't know why EMX needs C++ internally.
It doesn't. But if you want to be able to use C++, that needs some support
deep in the internals of EMX. And since C++ was a rapidly evolving
standard, compilers have been evolving rapidly as well, so there are quite
some incompatibilities.
Post by Andreas Buening
You mean the standard Windows user has exactly one hard disk and one
CDROM?
Yes, definitely, maybe some additional network drives, maybe a DVD instead
of CDROM.
Post by Andreas Buening
do with their old hard disk if they buy a new one? They throw it into
the basket?
Together with the whole computer, yes. Don't forget that you do need
completely new hardware every 3-4 years in Windows-country. So why bother
with individual new harddisks?
Post by Andreas Buening
In other words USB an other external devices are accessed by some
proprietary software using an proprietary API?
Typically there's a hard drive emulation layer, AFAIK. But if the vendor
doesn't care to support it anymore (or if Windows changes its driver
modell once again), you essentially _are_ lost, yes.
Post by Andreas Buening
You mean the "default Winxx user" doesn't know how and why to create
a separate partition for programs, games and data?
The default Winxx user meanwhile gets its box with WinXX pre-installed
to the only partition existing on the hard drive and without installation
media, yes. Unless you buy stuff like PartitionMagic (newest version), you
are unable to repartition.
Post by Andreas Buening
You mean 99%
of all Winxx users throw away their old hard disk if they buy a new
one?
99% don't know what a hard disk is. They have "computer", monitor,
keyboard and mouse. If you're lucky, they now about CDROMs and CDs. ;-)
Post by Andreas Buening
Because using d: for the 2nd hard disk would be evil?
Because, if you buy a new computer anyway, why not buy one with a larger
hard disk. :-(
Post by Andreas Buening
You mean one of the next Winxx releases will do an incompatible change
and remove the drive letters? And all old applications that expect
a drive letter will fail so that users will have to buy new ones?
Why not? Sounds like switching from Win3.1 to Win95.
Post by Andreas Buening
Post by Holger Veit
like a usable DOS box,
Usable DOS box. That's an oxymoron, isn't it? ;-)
You mean one of the next Winxx releases will have no shell where
you could enter commands directly?
Probably just that it won't be compatible any more. Just try running any
DOS games in NT/2000/XP's DOS box and you will notice the difference
between "usable DOS box" (like on OS/2) and "Windows' DOS box".
Post by Andreas Buening
You're right. Winxx is much superior. When did they stop to sell
the last DOS clone WinME containing the same 20 year old bugs?
Don't know, but AFAIK it's quite difficult to buy anything but WinXP
nowadays.
Post by Andreas Buening
Post by Holger Veit
No one will seriously use EMX for writing "OS/2" software anymore; if at
If I got 1 cent for every time somebody said "Nobody ever needs xyz" ...
Actually, EMX was always used almost exclusively for porting software,
wasn't it? If you've been developping OS/2 only software, VAC++ or
similar was the "natural" choice. And for such projects, there really is
no reason to switch to gcc, is there? "Porting" Mozilla, OpenOffice or
whatever is a different thing.
Post by Andreas Buening
Oh, sure, taking into account the great VAC++ support by IBM changes
everything. Oh, wait, was there any support by IBM at all?
Irrelevant for existing OS/2 only software. The successfully worked around
VAC++ bugs by now ...
Post by Andreas Buening
Post by Holger Veit
(POSIX and ANSI)
Why should a compiler not be able to comply to both standards?
There are contradictions between both standards. Only on minor details,
but they do exist. :-(
Post by Andreas Buening
Ehm, hello? If the API doesn't conform to the documentation then the
API is mostly useless, and if the docs do not conform to any standard
then it's nearly impossible to write portable programs.
Well, if the API behaves the same way as on Linux or BSD, it doesn't
really matter, that it doesn't conform to an official standard, does it?
And if the documentation for some function is not quite up to date, it's
not that important either, as long as you know that e.g. Linux man pages
give the correct description.
Post by Andreas Buening
Hello? The presence of an _chdir2 entry point prevents the programmer
in porting Unix applications?
No, but strict ANSI compliance would prevent porting Unix applications as
"Unix API" is _not_ strictly standard compliant.
Post by Andreas Buening
speak for the majority of the people on these mailing lists if I say
that emx (or whatever its future name will be) must be able to compile
classical OS/2 programs _as_ _well_ _as_ ports of Unix software.
Breaking that behaviour wasn't at all the point (at least not _my_ point).
I rather think of stuff like fread/fopen/fwrite/fseek silently supporting
large files. This _is_ incompatible with existing applications, but unless
they do something stupid/buggy (like assigning off_t's to int variables),
recompiling is enough to make them use the new API - and if they don't
need large files, even ignoring said stupid bugs and just recompiling
makes the application work again.
Meanwhile I do believe that this doesn't necessarily mean breaking binary
compatibility. Whoever thinks that it is important enough should be able
to take existing implementations, rename them to
fread32/fopen32/fwrite32/fseek32 and assign the "right" ordinals to those
and everything should work fine, if I understand the concept correctly.

I.e. just get the thing started, get as much Unix compatibility while
marking possible incompatibilities with EMX and resolve those later by
suitable wrappers or duplication of existing code. Once we do have e.g.
"setuid" we can still discuss on whether or not the existing ordinal
should be used for the dummy implementation or the real one.
Post by Andreas Buening
We can't know how the Watcom project will go and lots of people
want to write portable programs which is easier with emx than with Watcom
(from what other people said, I myself don't use Watcom).
Well, if you're using gcc on the other platforms, emx certainly simplifies
things. OTOH, if portable means Linux/Windows/OS2, you can just as well
use Watcom, _if_ you start from scratch.

Regards,
Stefan
Sebastian Wittmeier (ShadoW)
2003-06-22 16:45:41 UTC
Permalink
Post by Stefan Neis
Meanwhile I do believe that this doesn't necessarily mean breaking binary
compatibility. Whoever thinks that it is important enough should be able
to take existing implementations, rename them to
fread32/fopen32/fwrite32/fseek32 and assign the "right" ordinals to those
and everything should work fine, if I understand the concept correctly.
Ok. But then compatibility only exists in one direction: You can use
old programs with the new DLLs. Not the other way round.
Why would you do it? You only have to maintain one set of EMX DLLs and
can access all internal structures.
But on the other side you are more limited in the implementation.

So please someone explain: What good is in accessing the internal
EMX???.DLL structures of other applications? Why are you not easily
able to use EMU???.DLL?

Sebastian
Stefan Neis
2003-06-22 17:24:37 UTC
Permalink
Post by Sebastian Wittmeier (ShadoW)
Post by Stefan Neis
fread32/fopen32/fwrite32/fseek32 and assign the "right" ordinals to those
and everything should work fine, if I understand the concept correctly.
Ok. But then compatibility only exists in one direction: You can use
old programs with the new DLLs. Not the other way round.
Sure. However, assuming you want large file support in your new programs,
the old DLLs won't help anyway. Also, that has always been the case for
EMX. Just try using EMX-0.9c DLL's (or even older ones) with a current
program.

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Ilya Zakharevich
2003-06-23 10:04:18 UTC
Permalink
Post by Stefan Neis
Post by Andreas Buening
I don't know why EMX needs C++ internally.
It doesn't. But if you want to be able to use C++, that needs some support
deep in the internals of EMX.
Could you elaborate on this?
Post by Stefan Neis
Post by Andreas Buening
speak for the majority of the people on these mailing lists if I say
that emx (or whatever its future name will be) must be able to compile
classical OS/2 programs _as_ _well_ _as_ ports of Unix software.
Breaking that behaviour wasn't at all the point (at least not _my_ point).
I rather think of stuff like fread/fopen/fwrite/fseek silently supporting
large files. This _is_ incompatible with existing applications
No, it is not. At least not with most of them, who do not do any seek.
Post by Stefan Neis
Meanwhile I do believe that this doesn't necessarily mean breaking binary
compatibility. Whoever thinks that it is important enough should be able
to take existing implementations, rename them to
fread32/fopen32/fwrite32/fseek32 and assign the "right" ordinals to those
and everything should work fine, if I understand the concept correctly.
The situation is not that simple. Your forgot about linking with old
libraries. Again, one needs to consider both linking with DLLs, and
linking with static libraries.

When you take all the ramifications into account, you will see that
one can't just re-ordinal the entry points; one needs both modes.
Thus -DEMX_LARGE_FILES and -DEMX_NO_LARGE_FILES (whichever is the
default is not very relevant, with GCCOPT supported).

Hope this helps,
Ilya
Stefan Neis
2003-06-23 14:08:20 UTC
Permalink
Post by Ilya Zakharevich
Post by Stefan Neis
Post by Andreas Buening
I don't know why EMX needs C++ internally.
It doesn't. But if you want to be able to use C++, that needs some support
deep in the internals of EMX.
Could you elaborate on this?
Not really, as I don't really know the details, so I'm just guessing.
Anyway, exception handling with C++-code called from C-code (e.g. main())
sounds like said C-code must be aware of some of the internals of C++.
Post by Ilya Zakharevich
No, it is not. At least not with most of them, who do not do any seek.
So why do I need the non-large-file enabled stuff at all?
Post by Ilya Zakharevich
The situation is not that simple. Your forgot about linking with old
libraries.
For C++ that's routinely broken with every gcc update, so I don't feel
a big urge to go through hoops to avoid doing it _once_ for C as well.

Regqards,
Stefan
Ilya Zakharevich
2003-06-23 14:41:42 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
Post by Stefan Neis
It doesn't. But if you want to be able to use C++, that needs some support
deep in the internals of EMX.
Could you elaborate on this?
Not really, as I don't really know the details, so I'm just guessing.
Well, I doubt it very much that *any* support is needed.
Post by Stefan Neis
Anyway, exception handling with C++-code called from C-code (e.g. main())
Why main() should be the same for C and C++? It is magical anyway...
Do gcc -S, and watch the output...

And the extern-"C"-function of C++ should not (by definition) pass
through any exceptions...
Post by Stefan Neis
Post by Ilya Zakharevich
No, it is not. At least not with most of them, who do not do any seek.
So why do I need the non-large-file enabled stuff at all?
??? If you want to read/read more than 2G, you *must* inform the OS
about this. This is what DosOpenL() is about.
Post by Stefan Neis
Post by Ilya Zakharevich
The situation is not that simple. Your forgot about linking with old
libraries.
For C++ that's routinely broken with every gcc update, so I don't feel
a big urge to go through hoops to avoid doing it _once_ for C as well.
For all the third-party libraries on your system? You just can't.
Some part of them won't recompile - even if you have the source -
would would not hold for some of them.

Hope this helps,
Ilya
Stefan Neis
2003-06-23 16:10:32 UTC
Permalink
Post by Ilya Zakharevich
Post by Stefan Neis
Post by Ilya Zakharevich
Post by Stefan Neis
It doesn't. But if you want to be able to use C++, that needs some support
deep in the internals of EMX.
Could you elaborate on this?
Not really, as I don't really know the details, so I'm just guessing.
Well, I doubt it very much that *any* support is needed.
I had the impression that both EM and Holger implied that such a
thing would be necessary. Might have been a misunderstanding on my part.
Post by Ilya Zakharevich
Post by Stefan Neis
Post by Ilya Zakharevich
No, it is not. At least not with most of them, who do not do any seek.
So why do I need the non-large-file enabled stuff at all?
??? If you want to read/read more than 2G, you *must* inform the OS
about this. This is what DosOpenL() is about.
What's wrong about _always_ using DosOpenL() (if available) and use 64 bit
values for seek related stuff if most programs do not do any seek anyway?
Post by Ilya Zakharevich
Post by Stefan Neis
Post by Ilya Zakharevich
The situation is not that simple. Your forgot about linking with old
libraries.
For C++ that's routinely broken with every gcc update, so I don't feel
a big urge to go through hoops to avoid doing it _once_ for C as well.
For all the third-party libraries on your system?
Exactly. If I have to do it for the bunch of all the third-party C++
libraries, what's the problem with doing it for a few C libraries that
might be on my system?
Post by Ilya Zakharevich
You just can't.
Some part of them won't recompile - even if you have the source -
would would not hold for some of them.
Any prominent example of such a library would help me see the problem.
I can't think of any.

Regards,
Stefan
Ilya Zakharevich
2003-06-23 16:25:47 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
Post by Stefan Neis
Post by Ilya Zakharevich
No, it is not. At least not with most of them, who do not do any seek.
So why do I need the non-large-file enabled stuff at all?
??? If you want to read/read more than 2G, you *must* inform the OS
about this. This is what DosOpenL() is about.
What's wrong about _always_ using DosOpenL() (if available)
EMX being unmaintained. No other reason.
Post by Stefan Neis
and use 64 bit values for seek related stuff if most programs do not
do any seek anyway?
The argument that "most programs" do something does not help with the
decision of "_always_ using" something else.
Post by Stefan Neis
Post by Ilya Zakharevich
For all the third-party libraries on your system?
Exactly. If I have to do it for the bunch of all the third-party C++
libraries, what's the problem with doing it for a few C libraries that
might be on my system?
For example, if you have 0 "third-party C++", then your argument does
not make sense. Given that a "C++ library" is an oxymoron anyway, I
do not expect you to have much more than 0. ;-)
Post by Stefan Neis
Post by Ilya Zakharevich
You just can't.
Some part of them won't recompile - even if you have the source -
would would not hold for some of them.
Any prominent example of such a library would help me see the
problem. I can't think of any.
No concrete example comes to my mind - though tar.exe is "very close"
to be an example. You have the source, but you can't recompile (due
to an orthogonal reason to what we discuss, though).

Yours,
Ilya
Stefan Neis
2003-06-24 12:28:06 UTC
Permalink
Post by Ilya Zakharevich
For example, if you have 0 "third-party C++", then your argument does
not make sense. Given that a "C++ library" is an oxymoron anyway, I
do not expect you to have much more than 0. ;-)
If you put a 1 in front of that 0, you're approximately right.
For C libraries, it's about the same (mostly XFree related), but
recompiling shouldn't be a problem if it's necessary.
Post by Ilya Zakharevich
Post by Stefan Neis
Any prominent example of such a library would help me see the
problem. I can't think of any.
No concrete example comes to my mind - though tar.exe is "very close"
to be an example. You have the source, but you can't recompile (due
to an orthogonal reason to what we discuss, though).
IIRC, there are newer versions which are compilable, though missing
some esoteric feature...

Regards,
Stefan
Ilya Zakharevich
2003-06-24 13:04:42 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
For example, if you have 0 "third-party C++", then your argument does
not make sense. Given that a "C++ library" is an oxymoron anyway, I
do not expect you to have much more than 0. ;-)
If you put a 1 in front of that 0, you're approximately right.
For C libraries, it's about the same (mostly XFree related), but
recompiling shouldn't be a problem if it's necessary.
I have an order (or two) more here. And some of them take hours to
recompile. (And I do not have a ready-to-recompile environment but
for a handful of them.)

Hope this helps,
Ilya
Thomas E. Dickey
2003-06-22 12:55:55 UTC
Permalink
Post by Andreas Buening
Post by Holger Veit
Too bad for you. Meanwhile, even Windows has almost managed, thanks
to large disk partitions, that the only drive letters the ordinary
You mean Windows has to be installed on drive C: or 99% of all Winxx
users don't know that they can use another drive letter for this?
(Can they? I'm not familiar with XP)
You can use another drive letter (XP included), but the boot.ini file
resides on C:
--
T.E.Dickey <dickey-***@public.gmane.org>
http://invisible-island.net
ftp://invisible-island.net
Ilya Zakharevich
2003-06-22 09:20:51 UTC
Permalink
Post by Holger Veit
Post by Ilya Zakharevich
Post by Holger Veit
Post by Ilya Zakharevich
Post by Holger Veit
is). This is the unavoidable price to pay.
Now try to prove this. AFAIU, your argument looks very much like bullying.
It is by example, it is not a proof. [Not mentioning that the example
is flawed.]
Not flawed. Proof by example is valid to counter an assertion.
I disagree with both parts.
Post by Holger Veit
Post by Ilya Zakharevich
Post by Holger Veit
It is unlikely to happen
What is "it"? People putting -DEMX_LARGE_FILES into GCCOPT?
If you need to recompile stuff anyway, there is no reason to remain
compatible to old conventions.
Since one does not need to recompile in most of the situations (i.e.,
when one does not need off64_t), there is every reason.
Post by Holger Veit
Post by Ilya Zakharevich
To take advantage of large files, one needs to use DosOpenL(). Using
L-version of seek()/tell() is secondary; moreover, no EMX_LARGE_FILES
is needed to support applications which do not need to seek far.
Effectively, DosOpenL *is* DosOpen *is* DosOpen2 *is* Dos16Open. Follow the
traces in the kernel.
I do not care about "effectively". If one does not use DosOpenL(),
one can't read/write files above 2G. I do not see why any other
argument is relevant.
Post by Holger Veit
You can also persuade HPFS to use a different blocking size to skip
over the 2GB limit.
On DASDs - yes. Of course, not on files.
Post by Holger Veit
Post by Ilya Zakharevich
Post by Holger Veit
thus an awful lot of software would suddenly underlie file
protection effects, including the effect that former NO OPs (like
setuid()) that simply returned 0 for success, will now fail.
Is setuid() implemented by EMX? Do you say it did not return -1 when
uid != 0?
misc/setuid.c return -1 (EINVAL).
Thanks, I can't check this now (this laptop has no EMX source).
Post by Holger Veit
This is formally correct, but unacceptable for working ports.
This depends. Nobody complained about Perl (but Perl is a special
point, since one can edit a Perl script to avoid these calls).
Post by Holger Veit
The pair getuid/setuid may return or expect uid=0 all the time, but
not all software delivers this uid. Also code the changes the uid
(seteuid/reuid, etc.) might later expect that the returned is no
longer 0. The whole system works for nice weather conditions
only. Result for porting: short-circuit these functions and adjust
stuff as needed.
Then I see no reason why changing EINVAL return to anything else may
break anything.
Post by Holger Veit
Post by Ilya Zakharevich
There are some broken ports lying around. Recompiling such a port
with new, more correct, headers/libraries will require some work.
If yes, I do not consider this as a valid argument against correcting
headers/libraries.
If you change headers, you modify semantics. I don't consider this as
preserving compatibility anymore.
There is a compatibility and compatibility. Making a former error
condition into something which is documented it might have happened is
of the latter ;-) kind.
Post by Holger Veit
Then solve the EMX problem yourself. As you were asking for - clearly
spoken - for a *victim* who should take over EMX so that you can add *your*
ideas and changes, don't count on me on the issue of EMX changes (which
should not change anything) any longer.
??? I never counted on you. You made your stand quite clear many
years ago already: you are interested in "vertical" applications only.
When you deliver a "turnkey" solution, your approach of "breaking
everything as far as it is not seen if one uses your software only" is
quite viable. No argument about this.

But other people have other agenda. They need different approaches.

Of course, if the "victim" would benefit from my "ideas and changes"
only, then this would be a kind of a skewed relationship ;-) - I would
provide all the changes, and all that the "victim" would do is
bothering me to make things better.

But there is another fundamental difference between you and me: it
looks like you believe that you (and me? ;-) are the *only* persons
capable of a contribution to EMX. My wish for coordinator clearly
indicates the opposite: IMO, there are *many* people who can improve
EMX with a reasonable amount of effort.

I even believe that there are many people who "sit" on
(almost-)ready-to-distribute solutions to EMX problems (like I do).
Post by Holger Veit
Feel free to do so. Be aware that you need some considerable modifications
alone to get EMX to work *correctly* with any gcc > 2.8 (or 2.9).
I see no reason to upgrade gcc from 2.8.1 (for the purpose of EMX buld).
Post by Holger Veit
No one will seriously use EMX for writing "OS/2" software anymore.
I would say more: I never saw any reason to write OS/2 software at all
(with a possible exception of WPS enhancers). Why confine yourself?
Post by Holger Veit
Post by Ilya Zakharevich
a) Have APIs conform to the documentation as much as possible;
b) Have the documentation conform to standards as much as possible;
c) Have support for as many APIs' entry points as possible.
I just do not see where the opposition of Unix/OS2/EMX comes into this
picture.
ANSI != POSIX.
??? So what?
Post by Holger Veit
a) and b) is secondary, nice to have
For me a) and b) are tantamount.
Post by Holger Veit
but c) helps the programmer.
Without a and b "c" is useless - what good is a feature if you do not
know what it will do. Without "c" one needs to reinvent the wheel
again - pitiful work, but doable.

"c" is nice to have, no more. And if one can add "c" via an external
library, with no support from EMX, there is no need to mix this into
the current discussion.

Of course, it does make sense to discuss which hooks one needs to add
to EMX to simplify addition of externally supported enhancements...

Hope this helps,
Ilya
Holger Veit
2003-06-22 12:58:32 UTC
Permalink
Admin notice: please don't put emx-***@public.gmane.org AND ***@xfreeos2.dyndns.org
into your To: or Cc:, they are the same host, and result in duplicate
postings. One of them is sufficient (os2-unix is of course another list).
[...]
Post by Ilya Zakharevich
Post by Holger Veit
Post by Ilya Zakharevich
What is "it"? People putting -DEMX_LARGE_FILES into GCCOPT?
If you need to recompile stuff anyway, there is no reason to remain
compatible to old conventions.
Since one does not need to recompile in most of the situations (i.e.,
when one does not need off64_t), there is every reason.
So you just open files and read them sequentially? You are sure that
stream I/O doesn't rely on seek/tell anywhere?
Post by Ilya Zakharevich
Post by Holger Veit
Post by Ilya Zakharevich
To take advantage of large files, one needs to use DosOpenL(). Using
L-version of seek()/tell() is secondary; moreover, no EMX_LARGE_FILES
is needed to support applications which do not need to seek far.
Effectively, DosOpenL *is* DosOpen *is* DosOpen2 *is* Dos16Open. Follow the
traces in the kernel.
I do not care about "effectively". If one does not use DosOpenL(),
one can't read/write files above 2G. I do not see why any other
argument is relevant.
You can but this is another story. You don't want to see that the whole
"programmers should use different APIs (fseek vs fseek64 or even open/open64)
when they want to read large files" argument is as stupid as Bill Gates'
comment that "640k is enough for everybody". This is a similar attitude as
this technical debate on optimizating the loader by use of ordinals vs.
named entries. The whole OS/2 kernel is full of suboptimal code - if one looked
at the thunking interpreter in DOSCALL1, one would avoid "32bit" Dos* calls
like a plague - topic shift.

Having more than one API for a similar purpose basically results in two
sets of tools; those that deal with small (<2G) files, and others that can
handle big ones. You surely want to add such garbage to autoconf/automake
rulesets, don't you?

[...]
Post by Ilya Zakharevich
Post by Holger Veit
This is formally correct, but unacceptable for working ports.
This depends. Nobody complained about Perl (but Perl is a special
point, since one can edit a Perl script to avoid these calls).
Post by Holger Veit
The pair getuid/setuid may return or expect uid=0 all the time, but
not all software delivers this uid. Also code the changes the uid
(seteuid/reuid, etc.) might later expect that the returned is no
longer 0. The whole system works for nice weather conditions
only. Result for porting: short-circuit these functions and adjust
stuff as needed.
Then I see no reason why changing EINVAL return to anything else may
break anything.
Look at common daemon code that expects *not* to run as root. It issues a
seteuid to become another user (like 'mail' or 'oracle' or whatever) and
the call
a) fails with -1: correct error check will terminate the program.
b) succeeds, but is a NOOP. Later the deamon code will be very surprised
to find out it is still 'root': correct error check will terminate the
program.
If a person porting such a thing want to get it work, he will ignore
the known EMX behaviour and invent an own mechanism that returns the
"correct" behaviour for this process. Maybe by rewriting the logic of the
setuid family at least to the degree needed, or by commenting out error
checks. I don't know how many people have invented workarounds for
prominent problems like strcasecmp, or setuid/setpgrp/setsid, or setitimer.
strcasecmp is among that the most harmless candidate.
Post by Ilya Zakharevich
Post by Holger Veit
Post by Ilya Zakharevich
There are some broken ports lying around. Recompiling such a port
with new, more correct, headers/libraries will require some work.
If yes, I do not consider this as a valid argument against correcting
headers/libraries.
If you change headers, you modify semantics. I don't consider this as
preserving compatibility anymore.
There is a compatibility and compatibility. Making a former error
condition into something which is documented it might have happened is
of the latter ;-) kind.
I and common users don't distinguish between compatibility and compatibility.
The effect is just: this damn thing used to work befor, and now it doesn't
anymore. It doesn't help whether it is then documented.
Post by Ilya Zakharevich
But other people have other agenda. They need different approaches.
Maybe. Maybe not. But you should not forget that this here is not a
customer/vendor relation. The customers don't pay any efforts or spent
lifetime to the people who (could) contribute. This in turn means, according
to another proverb: If you pay the orchestra, you decide what they'll play.
Not these other people with their other agendas. As the EMX code is public,
anyone of these can, in full agreement with the GPL, modify anything from
the name to the content. Without any maintainer.
Post by Ilya Zakharevich
Of course, if the "victim" would benefit from my "ideas and changes"
only, then this would be a kind of a skewed relationship ;-) - I would
provide all the changes, and all that the "victim" would do is
bothering me to make things better.
If you find such a person, well go ahead.
Post by Ilya Zakharevich
But there is another fundamental difference between you and me: it
looks like you believe that you (and me? ;-) are the *only* persons
capable of a contribution to EMX. My wish for coordinator clearly
I don't think so. I rather think that most people are afraid that their
contribution will affect the holy grail of compatibility that you and some
more people keep holding high. This is closely related to the question
of acceptance by "the community", or precisely, by a number of speakers
with their own agendas. libemu, or Posix/2, is an example of code that
was not accepted by the community, in that it was regularly used in ports
and continously improved to weed out the bugs. Why? My opinion is:
it was not fully integrated, and maybe at the moment it would be used, a
better and greater EMX would come up which would obsolete all this. Now,
I doubt that there will be a new EMX "threatening" (by EM, although I can't
read his mind), but the panic of suddenly becoming incompatible with whatever
mainstream one might imagine is a good explanation of why there is no
forthcoming. It was observable while people waited for the great Windows 95
(which supposedly did no longer rely on DOS), then on Win98 (which supposedly
was more stable because it was not based on DOS), then on WinME (no DOS, for
sure). At the same time, there was a rather stable OS for long already;
you know it, but it was not "compatible". Eventually, there was Win2K
(still not "correct") and WinXP - but meanwhile there is a whole new generation
of software and people forgot about their old software which no longer
works.

Watcom C/C++ is now OpenWatcom, so all complaints that this might have been
a good tool, but yet, it was too expensive are now void. But still it is
not dominant, as it is not "compatible".

Not that anyone now gets the impression that have, within this mail turned from"con-compatibility" to "pro-compatibility" - I am just giving an explantion
that people are trapped by the compatibility curse. M$ did the right thing,
from a technical point of view. They eroded the compatibility issue by
more and more adding new attractive features which forced programmers to
move on and forced users to upgrade.

The compatibility trap is deadly for OS/2 or eCS future, as it will
severely restrict the options. Now: "deadly" - there is the bad mouth
again: OS/2 is dead. Should say rather: stale mate.
Post by Ilya Zakharevich
indicates the opposite: IMO, there are *many* people who can improve
EMX with a reasonable amount of effort.
Technically yes, no doubt. But why didn't it happen then? Don't repeat the
strawman "no maintainer" argument: it doesn't work: while many people might
not be a genius like EM to invent such a thing from scratch, they can
recompile it at least, with own (maybe just small) improvements. Add
strcasecmp aliases, and call this EMX 0.9d.1, or call it EMU 1.0.0, and
there is public code one epsilon better. Put it up at hobbes, and you are
done? You are not done, because the grail keepers will howl - how can you,
and ordinal 1234 is not okay for strcasecmp, you should better use 2000,
and some code might now no longer compile because there is a
#define strcasecmp stricmp in it, and anyway, this has discussed for long
and has been reported to EM, and will be done differently in the next EMX
patch, etc. etc. blurb. A technically skilled programmer will dare this
experiment once, and is then cured for ever from walking into an established
minefield.
Post by Ilya Zakharevich
I even believe that there are many people who "sit" on
(almost-)ready-to-distribute solutions to EMX problems (like I do).
But you need a strawman to distribute this, or literally, burn instead of you?
Post by Ilya Zakharevich
Post by Holger Veit
Post by Ilya Zakharevich
a) Have APIs conform to the documentation as much as possible;
b) Have the documentation conform to standards as much as possible;
c) Have support for as many APIs' entry points as possible.
I just do not see where the opposition of Unix/OS2/EMX comes into this
picture.
ANSI != POSIX.
??? So what?
You have a good chance to shoot yourself in the foot when you attempt to
serve two masters.
Post by Ilya Zakharevich
Post by Holger Veit
a) and b) is secondary, nice to have
For me a) and b) are tantamount.
Post by Holger Veit
but c) helps the programmer.
Without a and b "c" is useless - what good is a feature if you do not
know what it will do. Without "c" one needs to reinvent the wheel
again - pitiful work, but doable.
c) is an as clean as possible Unix or POSIX API, so it is documented (this
is a) ). The emphasis is on "clean" not "many". I don't care about
some DOSish APIs like "_getdrive()", as long as they don't disturb
anything, they may remain there - but if they somehow do (say a
"readlink() which would return backslash paths) I'd drop them.
Post by Ilya Zakharevich
"c" is nice to have, no more. And if one can add "c" via an external
library, with no support from EMX, there is no need to mix this into
the current discussion.
This effort turned out to be doomed - there is supposedly only one "libc",
not multiple ones. More helper libraries will mess up the build system
at some place.
Post by Ilya Zakharevich
Of course, it does make sense to discuss which hooks one needs to add
to EMX to simplify addition of externally supported enhancements...
There used to be a distinction between Unix man(2) and man(3), something EMX
to some degree also separates. Provided this would be the borderline, one
would no longer need EMX at all; you could port libc-BSD or glibc on top
of that, and maybe add some OS/2 APIs like EA support afterwards. A design
that scatters hooks into internal structures and APIs all over the code
reminds me too much on the kernel and PMMERGE - IBM's compatibility curses.

Holger
Ilya Zakharevich
2003-06-22 15:30:01 UTC
Permalink
Post by Holger Veit
Post by Ilya Zakharevich
Since one does not need to recompile in most of the situations (i.e.,
when one does not need off64_t), there is every reason.
So you just open files and read them sequentially?
Yes. Probably you missed this discussion on the newsgroups: doing tar
-f file fails; doing tar > file failes in EMX-compiled shells; doing
tar >file succeeds from CMD.EXE.
Post by Holger Veit
You are sure that stream I/O doesn't rely on seek/tell anywhere?
??? It works on pipes, right?
Post by Holger Veit
Post by Ilya Zakharevich
I do not care about "effectively". If one does not use DosOpenL(),
one can't read/write files above 2G. I do not see why any other
argument is relevant.
You can but this is another story.
Well, I do not know which "other" story do you mean. ;-)
Post by Holger Veit
You don't want to see that the whole "programmers should use
different APIs (fseek vs fseek64 or even open/open64) when they want
to read large files" argument is as stupid as ....
I do not know what you want to say here. Depending on
-DEMX_LARGE_FILES and -DEMX_NO_LARGE_FILES, fseek() should mean one of
fseek32() and fseek64(). For backward compatibility, it is convenient
to have -DEMX_LARGE_FILES as default for 2..3 years. Anyway, these
variables may be set in GCCOPT. AFAIK, this is what Solaris does; and
I did not see anybody complaining that it is as stupid as...
Post by Holger Veit
Having more than one API for a similar purpose basically results in two
sets of tools; those that deal with small (<2G) files, and others that can
handle big ones. You surely want to add such garbage to autoconf/automake
rulesets, don't you?
??? It is already there, since all the major players do it this way.
Post by Holger Veit
Post by Ilya Zakharevich
Then I see no reason why changing EINVAL return to anything else may
break anything.
Look at common daemon code that expects *not* to run as root. It issues a
seteuid to become another user (like 'mail' or 'oracle' or whatever) and
the call
a) fails with -1: correct error check will terminate the program.
b) succeeds, but is a NOOP. Later the deamon code will be very surprised
to find out it is still 'root': correct error check will terminate the
program.
If a person porting such a thing want to get it work, he will ignore
the known EMX behaviour and invent an own mechanism that returns the
"correct" behaviour for this process.
The correct behaviour for setuid() is to return -1 unless uid=0.
Since the program in question has no way to get a uid which is !=0,
there is no need to do anything else to port it.
Post by Holger Veit
Maybe by rewriting the logic of the
setuid family at least to the degree needed, or by commenting out error
checks. I don't know how many people have invented workarounds for
prominent problems like strcasecmp, or setuid/setpgrp/setsid, or setitimer.
strcasecmp is among that the most harmless candidate.
I still see no example of how changing the setuid() in the CRT library
will hurt your hypothetical program.

[BTW, *I* would make a wrapper which calls EMX's setuid(), and would
ignore the error if the newuid==olduid==0. This way I would benefit
from a possible change in EMX's setuid(), *and* get a reasonable
behaviour too.]
Post by Holger Veit
Post by Ilya Zakharevich
There is a compatibility and compatibility. Making a former error
condition into something which is documented it might have happened is
of the latter ;-) kind.
I and common users don't distinguish between compatibility and compatibility.
Sorry, but I have reason to think that I understand "common users"
better than you. [Of course, this would not stop you from thinking
the same. ;-]
Post by Holger Veit
The effect is just: this damn thing used to work befor, and now it doesn't
anymore. It doesn't help whether it is then documented.
Yes, bug-for-bug compatibility is a bad target; and 0.1% of programs
will break no matter how careful you are. But numbers matter; on many
systems LIBPATHSTRICT will help, and given a very few failing
programs, a wrapper is a viable solution.
Post by Holger Veit
Post by Ilya Zakharevich
But other people have other agenda. They need different approaches.
Maybe. Maybe not. But you should not forget that this here is not a
customer/vendor relation.
We all know how much damn you claim you are giving the users of your
software (though I reserve my judgement about how your deeds
correspond to your words ;-). Personally, I prefer the supplier/user
relationship which is orthogonal to the question of money exchange
(with me on either ends of the pipe).
Post by Holger Veit
Not these other people with their other agendas. As the EMX code is public,
anyone of these can, in full agreement with the GPL, modify anything from
the name to the content. Without any maintainer.
EM's code quality and QA is hard to match without a cooperative effort
- with patches flying through mailing lists - or newsgroups - and
people actually vgrepping them.
Post by Holger Veit
I don't think so. I rather think that most people are afraid that their
contribution will affect the holy grail of compatibility that you and some
more people keep holding high.
If such people are indeed "most", too bad for us. However, I watch
the discussions through serveral portholes, and my (educated?) guess
that the number of such people is well below what one can count on one
hand. ;-)

Such people are going to exclude themselves from the effort. But what
matters is how many people *remain*, not how many people are
excluded...
Post by Holger Veit
This is closely related to the question
of acceptance by "the community", or precisely, by a number of speakers
with their own agendas. libemu, or Posix/2, is an example of code that
was not accepted by the community, in that it was regularly used in ports
[skipped]

I never believed that anything would come out of this (but I hoped
nevertheless that it would). Actually, I do not remember what it
wanted to be; but IIRC, it tried to solve problems which are mostly
irrelevant (i.e., take very little proportion of porting time) by
making some "magic actions" (like opening a "wrong file"), and by some
pipe dreaming ("let us design a new program loader").

The success of EMX is that it makes things run by a more-or-less
straightforward translation to OS/2 API; there is very few "magic
actions" (the exception is, e.g., the fork() logic). This way, if EMX
support is not enough for a particular task, you can resolve to using
OS/2 API (since, e.g., file handles are compatible).
Post by Holger Veit
The compatibility trap is deadly for OS/2 or eCS future, as it will
severely restrict the options.
Quite the opposite: the fact that your old programs "just work" in the
new environment *opens* options, since of has more resources to make
some new stuff. I have seen many "let's start from scratch" frenzies;
very few of them succeed; of course, *when* they succeed, it is a
large step forward.
Post by Holger Veit
Technically yes, no doubt. But why didn't it happen then? Don't repeat the
strawman "no maintainer" argument: it doesn't work
As I said, my believe is that it does.
Post by Holger Veit
Post by Ilya Zakharevich
I even believe that there are many people who "sit" on
(almost-)ready-to-distribute solutions to EMX problems (like I do).
But you need a strawman to distribute this, or literally, burn
instead of you?
I need a person to coordinate an effort which is much better done by
several people than by one.
Post by Holger Veit
Post by Ilya Zakharevich
Post by Holger Veit
Post by Ilya Zakharevich
I just do not see where the opposition of Unix/OS2/EMX comes into this
picture.
ANSI != POSIX.
??? So what?
You have a good chance to shoot yourself in the foot when you attempt to
serve two masters.
Sorry for being so stupid, but I still have no idea what you are
talking about...

As I said, the best side of EMX is that due to shallow translation, it
is easy to combine the use of EMX's API with the use of OS/2 API. Any
such program is neither "Unix", nor "OS/2" (whatever these terms may
mean). And as I said, any sufficiently advance program/library more
or less *must* be designed this way (to avoid shortcomings of EMX).
Post by Holger Veit
c) is an as clean as possible Unix or POSIX API, so it is documented (this
is a) ). The emphasis is on "clean" not "many". I don't care about
some DOSish APIs like "_getdrive()", as long as they don't disturb
anything, they may remain there - but if they somehow do (say a
"readlink() which would return backslash paths) I'd drop them.
This is "b" - a limitation of API (e.g., a possible incompatibility of
readlink() and _getdrive()). [Although I have no idea what you mean
by "backslash path".]
Post by Holger Veit
Post by Ilya Zakharevich
"c" is nice to have, no more. And if one can add "c" via an external
library, with no support from EMX, there is no need to mix this into
the current discussion.
This effort turned out to be doomed - there is supposedly only one "libc",
not multiple ones. More helper libraries will mess up the build system
at some place.
Well, I see that on Solaris I use -lz without breaking up the build
system. Why should z.lib be in libc on OS/2? Moreover, I do not see
why c.lib should not link with 20 different DLLs (as opposed to 4 or 5
now) - we are not on Unix with its stupid limitation of link_library
== DLL.
Post by Holger Veit
Post by Ilya Zakharevich
Of course, it does make sense to discuss which hooks one needs to add
to EMX to simplify addition of externally supported enhancements...
There used to be a distinction between Unix man(2) and man(3), something EMX
to some degree also separates.
Do you know Unixish architectures where this distinction is no more?
I do not. And with long-file-support in many Unices this distinction
is more and more pronounced...

Ilya
Stefan Neis
2003-06-23 14:24:46 UTC
Permalink
Post by Ilya Zakharevich
Post by Holger Veit
handle big ones. You surely want to add such garbage to autoconf/automake
rulesets, don't you?
??? It is already there, since all the major players do it this way.
Sure,
CXXLAGS="-DSOME_LARGE_FILE_MACRO" CFLAGS="-DSOME_LARGE_FILE_MACRO" ./configure
Great! And if some sub-library has (not) been using the flag while you did
(not), the stuff that you will get after hours of configure/make process
is just garbage. It works as long as _everybody_ is either ignoring the
flag or _everybody_ is using it.
Why would I want to duplicate all those problems on OS/2?
Post by Ilya Zakharevich
Since the program in question has no way to get a uid which is !=0,
there is no need to do anything else to port it.
There are some multi-user additions being developped, so setuid might well
be able to get a uid != 0 in the not so distant future.
Post by Ilya Zakharevich
Yes, bug-for-bug compatibility is a bad target; and 0.1% of programs
will break no matter how careful you are. But numbers matter; on many
systems LIBPATHSTRICT will help, and given a very few failing
programs, a wrapper is a viable solution.
What good is LIBPATHSTRICT if DLL's with different names are used anyway?
Post by Ilya Zakharevich
Post by Holger Veit
of acceptance by "the community", or precisely, by a number of speakers
with their own agendas. libemu, or Posix/2, is an example of code that
was not accepted by the community, in that it was regularly used in ports
I never believed that anything would come out of this (but I hoped
nevertheless that it would). Actually, I do not remember what it
wanted to be;
A simple layer of BSD compatibility function in addition to what EMX
provides.
Post by Ilya Zakharevich
but IIRC, it tried to solve problems which are mostly
irrelevant (i.e., take very little proportion of porting time) by
making some "magic actions" (like opening a "wrong file"), and by some
pipe dreaming ("let us design a new program loader").
None of those is related to Posix/2 (at least not at its current state).

Actually, trying to use it on a larger scale for UnixOS/2, the biggest
problem was software already containing own workarounds for stuff missing
in EMX, so one ends up with duplicate symbols (e.g. setuid and various
others). Patching e.g. perl accordingly is a minor problem, but people
tend to not want patching working stuff...

Regards,
Stefan
Ilya Zakharevich
2003-06-23 14:46:21 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
Post by Holger Veit
handle big ones. You surely want to add such garbage to autoconf/automake
rulesets, don't you?
??? It is already there, since all the major players do it this way.
Sure,
CXXLAGS="-DSOME_LARGE_FILE_MACRO" CFLAGS="-DSOME_LARGE_FILE_MACRO" ./configure
Great! And if some sub-library has (not) been using the flag while you did
(not), the stuff that you will get after hours of configure/make process
is just garbage. It works as long as _everybody_ is either ignoring the
flag or _everybody_ is using it.
Why would I want to duplicate all those problems on OS/2?
Because doing it any other way leads to yet more trouble.
Post by Stefan Neis
Post by Ilya Zakharevich
Since the program in question has no way to get a uid which is !=0,
there is no need to do anything else to port it.
There are some multi-user additions being developped, so setuid might well
be able to get a uid != 0 in the not so distant future.
??? We were discussing *existing* programs, did not we?
Post by Stefan Neis
Post by Ilya Zakharevich
Yes, bug-for-bug compatibility is a bad target; and 0.1% of programs
will break no matter how careful you are. But numbers matter; on many
systems LIBPATHSTRICT will help, and given a very few failing
programs, a wrapper is a viable solution.
What good is LIBPATHSTRICT if DLL's with different names are used anyway?
??? This whole discussion is about making old programs work better
without recompile (by maintaining EMX again). Using DLLs with
different names is an orthogonal topic.
Post by Stefan Neis
Actually, trying to use it on a larger scale for UnixOS/2, the biggest
problem was software already containing own workarounds for stuff missing
in EMX, so one ends up with duplicate symbols (e.g. setuid and various
others). Patching e.g. perl accordingly is a minor problem, but people
tend to not want patching working stuff...
How many packages (in percents) worked out-of-the-box?

Thanks,
Ilya
Stefan Neis
2003-06-23 16:03:45 UTC
Permalink
Post by Ilya Zakharevich
Post by Stefan Neis
There are some multi-user additions being developped, so setuid might well
be able to get a uid != 0 in the not so distant future.
??? We were discussing *existing* programs, did not we?
So what? Since "porting" often consists of "./configure && make" there are
programs which rely on set/getuid to work on OS/2 just the same as on
Unix. Mailing lists are full of question like "what can I do to get
application xyz working, it's falling on set/getuid, strncasecmp,
whatever".
Post by Ilya Zakharevich
??? This whole discussion is about making old programs work better
without recompile (by maintaining EMX again). Using DLLs with
different names is an orthogonal topic.
No sorry. The whole discussion is about making old programs work better
by recompiling with a new libc and leaving old programs as they are.
At least that was my impression... :-/
Post by Ilya Zakharevich
Post by Stefan Neis
Actually, trying to use it on a larger scale for UnixOS/2, the biggest
problem was software already containing own workarounds for stuff missing
in EMX, so one ends up with duplicate symbols (e.g. setuid and various
others). Patching e.g. perl accordingly is a minor problem, but people
tend to not want patching working stuff...
How many packages (in percents) worked out-of-the-box?
Say 20% of the previously "fixed" ones ("broken" in terms of Posix/2),
which was all that was tested. In most instances, removing some
#ifdef __EMX__
various rubbish
#endif
was sufficient to solve the problems, two packages turned up real bugs in
Posix/2 (fixed by now).

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Ilya Zakharevich
2003-06-23 16:18:47 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
??? We were discussing *existing* programs, did not we?
So what? Since "porting" often consists of "./configure && make" there are
programs which rely on set/getuid to work on OS/2 just the same as on
Unix. Mailing lists are full of question like "what can I do to get
application xyz working, it's falling on set/getuid, strncasecmp,
whatever".
I still do not understand how this supports (IMO broken) Holger's
argument that existence of such programs may hinder upgrading of
EMX-supplied functions.
Post by Stefan Neis
Post by Ilya Zakharevich
??? This whole discussion is about making old programs work better
without recompile (by maintaining EMX again). Using DLLs with
different names is an orthogonal topic.
No sorry. The whole discussion is about making old programs work better
by recompiling with a new libc and leaving old programs as they are.
At least that was my impression... :-/
Well, it is not from the moment I was CC'ed (I do not subscribe to
unixos2, so it can be that the discussion was like you say before this).

Ilya
Sebastian Wittmeier (ShadoW)
2003-06-23 16:22:02 UTC
Permalink
Post by Ilya Zakharevich
Post by Stefan Neis
There are some multi-user additions being developped, so setuid might well
be able to get a uid != 0 in the not so distant future.
??? We were discussing *existing* programs, did not we?
If we want to make "old programs work better without recompile" and
OS/2 gets multi-user capable, setuid won't be a dummy function any
longer, and will return the real uid. Some existing ports can't handle
that added functionality. Either we leave the old setuid dummy for old
programs (no improvement for existing programs), or we change the
setuid (some existing programs cease to work correctly).

Sebastian
Ilya Zakharevich
2003-06-23 16:43:14 UTC
Permalink
Post by Sebastian Wittmeier (ShadoW)
Post by Ilya Zakharevich
Post by Stefan Neis
There are some multi-user additions being developped, so setuid might well
be able to get a uid != 0 in the not so distant future.
??? We were discussing *existing* programs, did not we?
If we want to make "old programs work better without recompile" and
OS/2 gets multi-user capable, setuid won't be a dummy function any
longer, and will return the real uid. Some existing ports can't handle
that added functionality.
[You mean getuid(), right?] So one will need to use EMXOPT=-!=0x40000
(or whatever it would be chosen) with these programs. That's all.
Even with *broken* ports.

Hope this helps,
Ilya
Andreas Buening
2003-06-23 20:13:22 UTC
Permalink
Post by Sebastian Wittmeier (ShadoW)
Post by Ilya Zakharevich
Post by Stefan Neis
There are some multi-user additions being developped, so setuid might well
be able to get a uid != 0 in the not so distant future.
??? We were discussing *existing* programs, did not we?
If we want to make "old programs work better without recompile" and
OS/2 gets multi-user capable, setuid won't be a dummy function any
longer, and will return the real uid. Some existing ports can't handle
that added functionality. Either we leave the old setuid dummy for old
programs (no improvement for existing programs), or we change the
setuid (some existing programs cease to work correctly).
Apart from broken ports that define their own set/getuid(), is there
any reason why the program behaviour should rely on the current emx
behaviour? At the moment I can't think of any example but I've never
dealt with this API extensively.
I mean no code will contain something like
if (getuid() != -1) system("rm -fR /");
;-)


And for the case OS/2 might ever get multiuser capabilities,
If a program can't deal with user IDs or permissions then it can't deal
with user IDs or permissions (like a typical DOS program). In this
case it's irrelevant whether you use this program with the old emx
library (which doesn't support users) or the new libc (which does
support
users but the programs doesn't know). If you enable your multiuser
capabilities and the program fails then it fails unless you want to
run it as root.


As far as I can see we have three kinds of function problems:

a) Functions that have to be added (like the all times favorite
strcasecmp): No problem, doesn't hurt old programs.

b) Functions with new behaviour (like set/getuid):
- Current sources may contain dummy defines; nasty but must be
removed anyway,
- Old programs (binaries): Not effected if we keep the old
ordinal number and the broken old function together (the
new implementation of the function gets a new ordinal number).
- Alternatively we don't keep the old implementation and the
program has to deal with the new (correct) behaviour.
We can decide this differently for every function depending on
how likely is a software failure for this case.

c) Functions that change its declaration (like fseek if we change
to 64 bit by default; how many of them are there?):
- Old programs (binaries): If we want to keep binary compatibility
the old function must keep its ordinal (but may get a new name
-> fseek32)
- We can make the new behaviour (fseek64) the default and still
provide a macro to switch to the old behaviour.


Bye,
Andreas
Stefan Neis
2003-06-24 12:56:40 UTC
Permalink
Post by Andreas Buening
- Old programs (binaries): Not effected if we keep the old
ordinal number and the broken old function together (the
new implementation of the function gets a new ordinal number).
c) Functions that change its declaration (like fseek if we change
- Old programs (binaries): If we want to keep binary compatibility
the old function must keep its ordinal (but may get a new name
-> fseek32)
However, as Ilya pointed out, this doesn't solve the problems for old
libraries (binaries), which don't reference the ordinal, just the function
name...

Regards,
Stefan
Ilya Zakharevich
2003-06-24 13:09:13 UTC
Permalink
Post by Stefan Neis
Post by Andreas Buening
- Old programs (binaries): Not effected if we keep the old
ordinal number and the broken old function together (the
new implementation of the function gets a new ordinal number).
c) Functions that change its declaration (like fseek if we change
- Old programs (binaries): If we want to keep binary compatibility
the old function must keep its ordinal (but may get a new name
-> fseek32)
However, as Ilya pointed out, this doesn't solve the problems for old
libraries (binaries), which don't reference the ordinal, just the function
name...
I do not see how this could be a problem. First of all, EMX CRT DLL
has no exports by name, right? [Can't check it now.] Second, when
you add a renamed function for one which was exported by name, you
export the new function by ordinal only. And keep the name pointing
to the old guy.

The problem with libraries which I meant is that a function could take
off_t as an argument (or a return value). If the meaning of off_t
changes in the calling program, but not in the library (or visa
versa), this leads to problems which are extremely hard to debug.

Hope this helps,
Ilya
Stefan Neis
2003-06-24 16:13:51 UTC
Permalink
Post by Ilya Zakharevich
I do not see how this could be a problem. First of all, EMX CRT DLL
has no exports by name, right? [Can't check it now.]
I think so yes. But (static) libraries typically don't reference DLL
entry points, do they? I think they just reference function names which
get resolved to DLL entry points or whatever at link time?
Post by Ilya Zakharevich
The problem with libraries which I meant is that a function could take
off_t as an argument (or a return value). If the meaning of off_t
changes in the calling program, but not in the library (or visa
versa), this leads to problems which are extremely hard to debug.
Oh, yes. That's something I learned under Solaris and Linux and that's
why I keep saying that following their scheme is not a very good idea.
OTOH, if you want to continue existing libraries without recompilation,
that's impossible to avoid.
Personally, I tend to recommend to recompile everything once and be done
with the problem, instead of fighting with those flags for years (and get
"extremely hard to debug" problems every now and then because some library
you are using was compiled the wrong way...), but I do see your point.

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Ilya Zakharevich
2003-06-25 12:17:56 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
I do not see how this could be a problem. First of all, EMX CRT DLL
has no exports by name, right? [Can't check it now.]
I think so yes. But (static) libraries typically don't reference DLL
entry points, do they? I think they just reference function names which
get resolved to DLL entry points or whatever at link time?
Correct.
Post by Stefan Neis
Post by Ilya Zakharevich
The problem with libraries which I meant is that a function could take
off_t as an argument (or a return value). If the meaning of off_t
changes in the calling program, but not in the library (or visa
versa), this leads to problems which are extremely hard to debug.
Oh, yes. That's something I learned under Solaris and Linux and that's
why I keep saying that following their scheme is not a very good idea.
OTOH, if you want to continue existing libraries without recompilation,
that's impossible to avoid.
I do not see any problem whatsoever. C_LIBRARY_PATH should be
different, that's all. It is easy to write a tiny script which will
sort existing libraries into 2 categories:

a) compatible with old API only;
b) do not call the changed API;

The newly compiled libraries may go into "b" or a third category

c) compatible with the new API only.

As far as we do not have *many* such forks, the situation is handable.

[A part that recompiling all libraries will solve all the problems
skipped - as I said, I do not consider this feasible.]

Hope this helps,
Ilya
Andreas Buening
2003-06-27 18:48:35 UTC
Permalink
Ilya Zakharevich wrote:

[snip]
Post by Ilya Zakharevich
I do not see any problem whatsoever. C_LIBRARY_PATH should be
different, that's all. It is easy to write a tiny script which will
a) compatible with old API only;
b) do not call the changed API;
The newly compiled libraries may go into "b" or a third category
c) compatible with the new API only.
As far as we do not have *many* such forks, the situation is handable.
[A part that recompiling all libraries will solve all the problems
skipped - as I said, I do not consider this feasible.]
Okay, just to summarize: Am I correct that

1) Existing executables and dlls can still use the old API which
will be provided by the new libc using the old ordinal numbers,

2) Existing static libraries can still be used if a) either the
compilation and linkage is done only with old libraries and
the old emx or b) the compilation and linkage is done with
old libraries or libraries generated by (-D_OLD_EMX_STYLE)
and the new libc,

3) On the long term all old libraries will (hopefully) vanish?


Bye,
Andreas
Ilya Zakharevich
2003-07-01 12:32:52 UTC
Permalink
Post by Andreas Buening
Okay, just to summarize: Am I correct that
1) Existing executables and dlls can still use the old API which
will be provided by the new libc using the old ordinal numbers,
Yes, this is extremely easy to implement: just preserve functions
sitting on old ordinals.
Post by Andreas Buening
2) Existing static libraries can still be used if a) either the
compilation and linkage is done only with old libraries and
the old emx or b) the compilation and linkage is done with
old libraries or libraries generated by (-D_OLD_EMX_STYLE)
and the new libc,
This needs also a support from the header files (e.g., mapping tell()
to one of of tell32() and tell64()). Some tools may break (those
which run nm on the object files, and expect to see _tell, not
_tell64); this may also be fixable, but I do not think such tools will
be widespread.

But I would object to the naming of your macro. Which style is OLD,
and which is NEW changes when time goes. It is much better to name
capability macros by the described capability, not by a timestamp.
Post by Andreas Buening
3) On the long term all old libraries will (hopefully) vanish?
If a library has no linkage with changed functions, and has no
function with a changed typedef in the signature, there is no need to
obsolete this. If a library is very specialized and would never be
critical in a big-file situation, there is also no need to fix it.

So I'd say it like this: libraries which better be recompiled would
eventually be recompiled ;-).

Hope this helps,
Ilya

P.S. But unless EM says otherwise, I think that without a volunteer
maintainer all this is going to be a pipe dream.

P.P.S. Carthage must be destroyed.
Andreas Buening
2003-07-06 20:24:06 UTC
Permalink
[snip]
Post by Ilya Zakharevich
Post by Andreas Buening
2) Existing static libraries can still be used if a) either the
compilation and linkage is done only with old libraries and
the old emx or b) the compilation and linkage is done with
old libraries or libraries generated by (-D_OLD_EMX_STYLE)
and the new libc,
This needs also a support from the header files (e.g., mapping tell()
Of course, static libraries and headers have to be consistent.

[snip]
Post by Ilya Zakharevich
But I would object to the naming of your macro. Which style is OLD,
and which is NEW changes when time goes. It is much better to name
capability macros by the described capability, not by a timestamp.
The macro name wasn't intended to be a draft. :-)
Post by Ilya Zakharevich
Post by Andreas Buening
3) On the long term all old libraries will (hopefully) vanish?
If a library has no linkage with changed functions, and has no
function with a changed typedef in the signature, there is no need to
obsolete this. If a library is very specialized and would never be
critical in a big-file situation, there is also no need to fix it.
Nevertheless, the safe way would be to deprecate old libraries.

[snip]
Post by Ilya Zakharevich
P.S. But unless EM says otherwise, I think that without a volunteer
maintainer all this is going to be a pipe dream.
We'll see. Btw, where have gone all the volunteers who wanted to host
the new emx CVS?

For the practical schedule I think it would be a good idea to
divide the development into three steps:
1. Restructuring of the current emx build system (no idea how much
work would be necessary for this) to make sure the new build
system works before we change anything.
2. Appliance of all known bug fixes to emx and the last 100%
compatible emx release (0.9g) as a proof of concept.
3. Update of the headers and addition of new features/functions.


[snip]


Bye,
Andreas
Ilya Zakharevich
2003-07-07 11:24:25 UTC
Permalink
Post by Andreas Buening
Post by Ilya Zakharevich
Post by Andreas Buening
2) Existing static libraries can still be used if a) either the
compilation and linkage is done only with old libraries and
the old emx or b) the compilation and linkage is done with
old libraries or libraries generated by (-D_OLD_EMX_STYLE)
and the new libc,
This needs also a support from the header files (e.g., mapping tell()
Of course, static libraries and headers have to be consistent.
This is not what I meant. For full compatibility, when a ABI/API-fork
(if you know what I mean ;-) is performed, the new functions should
have different link-time name, and the header files should #define old
names into the new ones. Actually, the same should be done for the
old names; i.e., fstat() should be replaced by fstat32() and fstat64()
- with "purely named" fstat() also sitting somewhere for the sake of
./configure scripts.
Post by Andreas Buening
Post by Ilya Zakharevich
Post by Andreas Buening
3) On the long term all old libraries will (hopefully) vanish?
If a library has no linkage with changed functions, and has no
function with a changed typedef in the signature, there is no need to
obsolete this. If a library is very specialized and would never be
critical in a big-file situation, there is also no need to fix it.
Nevertheless, the safe way would be to deprecate old libraries.
If we do enough, then deprecation is not needed; depreciation is a
better solution.

(E.g.: "we honestly expect that many old libraries will work; but if
you encounter problems and if want more support, recompiling/replacing
your libraries can encourage more people to help you" ;-).
Post by Andreas Buening
Post by Ilya Zakharevich
P.S. But unless EM says otherwise, I think that without a volunteer
maintainer all this is going to be a pipe dream.
We'll see. Btw, where have gone all the volunteers who wanted to host
the new emx CVS?
It may be ignorance, but in many years of my experience I never saw a
situation when CVS helps.

It is *crucial* that the patches fly through a mailing list (or,
better, an mailing-list/archived-newsgroup combo) - so that as many
people as possible can comment on this.

It is inevitable that patches will conflict - so a conflict
resolution scenario is required.

At some moment a decision must be made that these and those patches
are accepted into the development branch - so somebody who has a
power to do so is required.
Post by Andreas Buening
1. Restructuring of the current emx build system (no idea how much
work would be necessary for this) to make sure the new build
system works before we change anything.
IMO: DO NOT do this. Use 2.8.1 + dmake.

There are *pressing needs* to fix things. Restructuring is a luxury
with no (?) known benefits.
Post by Andreas Buening
2. Appliance of all known bug fixes to emx and the last 100%
compatible emx release (0.9g) as a proof of concept.
AFAIK, there is no need to have the *last* compatible release. The
amount of work to make new releases compatible is muniscule comparing
to QA work. And in the absense of QA framework it is crucial to have
a large pool of applications which have a chance to break due to
yet-uncovered bug - so we can fix this bug early ;-).

Hope this clarifies my position,
Ilya
Andreas Buening
2003-07-08 07:40:15 UTC
Permalink
[snip]
Post by Ilya Zakharevich
Post by Andreas Buening
Of course, static libraries and headers have to be consistent.
This is not what I meant. For full compatibility, when a ABI/API-fork
(if you know what I mean ;-) is performed, the new functions should
Not really. ;-)
Post by Ilya Zakharevich
have different link-time name, and the header files should #define old
names into the new ones. Actually, the same should be done for the
old names; i.e., fstat() should be replaced by fstat32() and fstat64()
- with "purely named" fstat() also sitting somewhere for the sake of
./configure scripts.
For configure scripts only the linker needs to find them. So fstat
will be most likely only exist in the import library and will be an
alias for fstat64.

[snip]
Post by Ilya Zakharevich
Post by Andreas Buening
Post by Ilya Zakharevich
P.S. But unless EM says otherwise, I think that without a volunteer
maintainer all this is going to be a pipe dream.
We'll see. Btw, where have gone all the volunteers who wanted to host
the new emx CVS?
It may be ignorance, but in many years of my experience I never saw a
situation when CVS helps.
We need a central source archive, anyway.
Post by Ilya Zakharevich
It is *crucial* that the patches fly through a mailing list (or,
better, an mailing-list/archived-newsgroup combo) - so that as many
people as possible can comment on this.
ACK. If we have no maintainer we need at least some quality control. :-)

[snip]
Post by Ilya Zakharevich
Post by Andreas Buening
1. Restructuring of the current emx build system (no idea how much
work would be necessary for this) to make sure the new build
system works before we change anything.
IMO: DO NOT do this. Use 2.8.1 + dmake.
Somebody mentioned that the current emx build system should be improved
if I remember correctly. I haven't looked closely at this, yet.
If we want to do this we should do it right at the beginning.
Post by Ilya Zakharevich
There are *pressing needs* to fix things. Restructuring is a luxury
with no (?) known benefits.
Post by Andreas Buening
2. Appliance of all known bug fixes to emx and the last 100%
compatible emx release (0.9g) as a proof of concept.
AFAIK, there is no need to have the *last* compatible release. The
amount of work to make new releases compatible is muniscule comparing
to QA work. And in the absense of QA framework it is crucial to have
a large pool of applications which have a chance to break due to
yet-uncovered bug - so we can fix this bug early ;-).
Of course, it isn't necessary to have a _compatible_ release but
it is (IMHO) necessary to have a _working_ release directly at the
beginning so we can revert to this working version if anything goes
wrong. And in this case it makes sense to make the first working
release also a compatible release. When applying the (few) bug fixes
we'll see where the possible bottlenecks are of our mailinglist/CVS
system and whether it works at all.


Btw, could reply to the UnixOS/2 list directly by 'to' not by 'cc'?
cc'ed posts seem to get lost.


Bye,
Andreas
Ilya Zakharevich
2003-07-08 11:52:41 UTC
Permalink
Post by Andreas Buening
Post by Ilya Zakharevich
Post by Andreas Buening
Of course, static libraries and headers have to be consistent.
This is not what I meant. For full compatibility, when a ABI/API-fork
(if you know what I mean ;-) is performed, the new functions should
Not really. ;-)
I mean when functions with "identical functionality" (so the change
can be made transparent via header files), but incompatible ABI or API
are introduced.
Post by Andreas Buening
Post by Ilya Zakharevich
have different link-time name, and the header files should #define old
names into the new ones. Actually, the same should be done for the
old names; i.e., fstat() should be replaced by fstat32() and fstat64()
- with "purely named" fstat() also sitting somewhere for the sake of
./configure scripts.
For configure scripts only the linker needs to find them. So fstat
will be most likely only exist in the import library and will be an
alias for fstat64.
[Some configure scripts check the header files too - but this should
not be a problem.]
Post by Andreas Buening
Post by Ilya Zakharevich
Post by Andreas Buening
We'll see. Btw, where have gone all the volunteers who wanted to host
the new emx CVS?
It may be ignorance, but in many years of my experience I never saw a
situation when CVS helps.
We need a central source archive, anyway.
In my experience, a mailing list archive serves this purpose well
enough. [Why I object to CVS: it creates a warm cozy filling that
things are settled up. This filling is counterproductive - e.g.,
since it is wrong. ;-]
Post by Andreas Buening
Post by Ilya Zakharevich
Post by Andreas Buening
1. Restructuring of the current emx build system (no idea how much
work would be necessary for this) to make sure the new build
system works before we change anything.
IMO: DO NOT do this. Use 2.8.1 + dmake.
Somebody mentioned that the current emx build system should be improved
if I remember correctly. I haven't looked closely at this, yet.
If we want to do this we should do it right at the beginning.
I repeat:

a) I know of no issue with 2.8.1 + dmake (but of course, this may be
just ignorance).

b) I see no reason to do this "in the beginning" (actually, I do not
see any reason to do this at all, if it requires more than man-hour of
work).
Post by Andreas Buening
Of course, it isn't necessary to have a _compatible_ release but
??? You probably missed another negative? I see no need for
uncompatible release...
Post by Andreas Buening
it is (IMHO) necessary to have a _working_ release directly at the
beginning so we can revert to this working version if anything goes
wrong.
Again, I do not follow. All the releases should be treated this way...

Yours,
Ilya
Stefan Neis
2003-07-08 15:10:15 UTC
Permalink
Post by Ilya Zakharevich
a) I know of no issue with 2.8.1 + dmake (but of course, this may be
just ignorance).
While I agree, the problem is that whatever new source one adds (e.g.
Posix/2), it won't be happy with dmake. While mixing two different makes
is certainly possible, it's also possibly confusing.
Post by Ilya Zakharevich
Post by Andreas Buening
it is (IMHO) necessary to have a _working_ release directly at the
beginning so we can revert to this working version if anything goes
wrong.
Again, I do not follow. All the releases should be treated this way...
I suppose, that's the CVS point of view. Start CVS with something that's
working, so their is a solid grounding...

Regards,
Stefan
Ilya Zakharevich
2003-07-08 15:33:40 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
a) I know of no issue with 2.8.1 + dmake (but of course, this may be
just ignorance).
While I agree, the problem is that whatever new source one adds (e.g.
Posix/2), it won't be happy with dmake. While mixing two different makes
is certainly possible, it's also possibly confusing.
??? Why adding a couple of new C files would require a change of $MAKE?
Post by Stefan Neis
Post by Ilya Zakharevich
Post by Andreas Buening
it is (IMHO) necessary to have a _working_ release directly at the
beginning so we can revert to this working version if anything goes
wrong.
Again, I do not follow. All the releases should be treated this way...
I suppose, that's the CVS point of view. Start CVS with something that's
working, so their is a solid grounding...
??? Start with the EM's released version.

Thanks for your contributions,
Ilya
Stefan Neis
2003-07-10 07:52:15 UTC
Permalink
Post by Ilya Zakharevich
Post by Stefan Neis
Post by Ilya Zakharevich
a) I know of no issue with 2.8.1 + dmake (but of course, this may be
just ignorance).
While I agree, the problem is that whatever new source one adds (e.g.
Posix/2), it won't be happy with dmake. While mixing two different makes
is certainly possible, it's also possibly confusing.
??? Why adding a couple of new C files would require a change of $MAKE?
Personally, I'd like at least seeing Posix/2 being added into libc. That's
close to 300 source files ...

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Ilya Zakharevich
2003-07-10 11:53:25 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
??? Why adding a couple of new C files would require a change of $MAKE?
Personally, I'd like at least seeing Posix/2 being added into libc. That's
close to 300 source files ...
If they all are covered by .c.obj rule, there is little difference in
numbers. ;-)

Is there a URL for the docs? 300 source files = 300 API entries
functions, I assume? Or is is 100 API entries, 200 common
for-internal-use functions?

Thanks,
Ilya
Stefan Neis
2003-07-10 16:23:48 UTC
Permalink
Post by Ilya Zakharevich
Is there a URL for the docs? 300 source files = 300 API entries
functions, I assume? Or is is 100 API entries, 200 common
for-internal-use functions?
Actually, any set of BSD man pages should do ...

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Ilya Zakharevich
2003-07-10 16:41:45 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
Is there a URL for the docs? 300 source files = 300 API entries
functions, I assume? Or is is 100 API entries, 200 common
for-internal-use functions?
Actually, any set of BSD man pages should do ...
Eh??? Do you claim that anything mentioned in any BSD manpage would work?

I doubt it very much - at least not with just 300 C files.... ;-)

Ilya
Curtis W. Rendon
2003-07-10 16:58:33 UTC
Permalink
Post by Ilya Zakharevich
Post by Stefan Neis
Post by Ilya Zakharevich
Is there a URL for the docs? 300 source files = 300 API entries
functions, I assume? Or is is 100 API entries, 200 common
for-internal-use functions?
Actually, any set of BSD man pages should do ...
Eh??? Do you claim that anything mentioned in any BSD manpage would work?
I doubt it very much - at least not with just 300 C files.... ;-)
Absolutely. I depend on them on my FreeBSD system, and when the LINUX
systems I work with fail to work as expected, I access BSD man pages.

Curtis
--
Curtis W. Rendon -- mithra-ogEGBHC/***@public.gmane.org
www.realtime.net/~mithra uce-***@public.gmane.org

A long memory is the most subversive idea in America.

***@H: 2666 @ 66974
Stefan Neis
2003-07-10 23:46:04 UTC
Permalink
Post by Ilya Zakharevich
Eh??? Do you claim that anything mentioned in any BSD manpage would work?
manpages concerning BSD's libc of course, after all, we are talking about
libc, aren't we... ;-)

Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Ilya Zakharevich
2003-07-11 12:32:31 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
Eh??? Do you claim that anything mentioned in any BSD manpage would work?
manpages concerning BSD's libc of course, after all, we are talking about
libc, aren't we... ;-)
I still have my doubt that your claims have any base under them. ;-)
E.g., do you claim that *any version* of any manpage in the section 3
(or is it 3c?) is 100% applicable without any restriction and
reservation?

Do all the ioctl()s work? [Hmm, they may be documented in different
sections ;-] What about link(), symlink() etc.? ;-)

Ilya
Andreas Buening
2003-07-10 21:41:33 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
Post by Stefan Neis
Post by Ilya Zakharevich
a) I know of no issue with 2.8.1 + dmake (but of course, this may be
just ignorance).
While I agree, the problem is that whatever new source one adds (e.g.
Posix/2), it won't be happy with dmake. While mixing two different makes
is certainly possible, it's also possibly confusing.
??? Why adding a couple of new C files would require a change of $MAKE?
Personally, I'd like at least seeing Posix/2 being added into libc. That's
close to 300 source files ...
I think the absolute number of source files won't be the problem.
It depends more on what we want the build system should do. Compile
the sources, link the dlls, create the import and static libraries,
a reasonable test suite, separation of parts of the libc into several
import libraries, create all docs (info, man, inf, html) from one
documentation "source" files automatically, support for optional libc
feature (e.g. internationalization).

The question is can the current build system do this or can it be
implemented? If not, then it might be the better decision to create
a new build system.


Bye,
Andreas
Stefan Neis
2003-07-10 23:52:10 UTC
Permalink
Post by Andreas Buening
The question is can the current build system do this or can it be
implemented? If not, then it might be the better decision to create
a new build system.
I rather liked the idea to intially have to different branches, one for
those fiddling with the make system and one for people fiddling with the
code itself and then merge those together when they work. That way, lack
of progress in one direction doesn't stop progress in the other direction.

Looks like we are entering a cycle ... ;-)

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Curtis W. Rendon
2003-07-08 16:32:00 UTC
Permalink
Post by Ilya Zakharevich
...
Post by Andreas Buening
Post by Ilya Zakharevich
Post by Andreas Buening
We'll see. Btw, where have gone all the volunteers who wanted to host
the new emx CVS?
It may be ignorance, but in many years of my experience I never saw a
situation when CVS helps.
We need a central source archive, anyway.
In my experience, a mailing list archive serves this purpose well
enough. [Why I object to CVS: it creates a warm cozy filling that
things are settled up. This filling is counterproductive - e.g.,
since it is wrong. ;-]
...
A mail archive won't substitute for a code library, nor will it keep up
with code modifications.
--
Curtis W. Rendon -- mithra-ogEGBHC/***@public.gmane.org
www.realtime.net/~mithra uce-***@public.gmane.org

A long memory is the most subversive idea in America.

***@H: 2666 @ 66974
Ilya Zakharevich
2003-07-09 07:27:49 UTC
Permalink
Post by Curtis W. Rendon
Post by Ilya Zakharevich
Post by Andreas Buening
We need a central source archive, anyway.
In my experience, a mailing list archive serves this purpose well
enough. [Why I object to CVS: it creates a warm cozy filling that
things are settled up. This filling is counterproductive - e.g.,
since it is wrong. ;-]
s/filling/feeling/g, sorry...
Post by Curtis W. Rendon
A mail archive won't substitute for a code library, nor will it keep up
with code modifications.
Do not know what you mean. If it can't "keep up", then the whole idea
is ruined, since *all* the patches are supposed to be viewed by the
mailing list. [And I do not know what is "a code library" in the
context of discussion of EMX updates.]

Having CVS makes a couple of operations quickier, but not much. As
far as it is kept as a *support tool*, it is fine. However, IMO, it
should not be considered as the primary modus operandi - this nourishes
wrong expectations and misguided priorities.

Hope this helps,
Ilya
Curtis W. Rendon
2003-07-09 16:52:04 UTC
Permalink
Post by Ilya Zakharevich
Post by Curtis W. Rendon
Post by Ilya Zakharevich
Post by Andreas Buening
We need a central source archive, anyway.
In my experience, a mailing list archive serves this purpose well
enough. [Why I object to CVS: it creates a warm cozy filling that
things are settled up. This filling is counterproductive - e.g.,
since it is wrong. ;-]
s/filling/feeling/g, sorry...
Post by Curtis W. Rendon
A mail archive won't substitute for a code library, nor will it keep up
with code modifications.
Do not know what you mean. If it can't "keep up", then the whole idea
is ruined, since *all* the patches are supposed to be viewed by the
mailing list. [And I do not know what is "a code library" in the
context of discussion of EMX updates.]
A code library is the central repository so that whomever is to generate
object libraries can get from the code library (CVS,RCS,whatever) a
designated group of source for the build that has been declared to all
be of the same version. A revision control system makes it easier to
manage, and revert if needed, than multiple directories of versions.
Post by Ilya Zakharevich
Having CVS makes a couple of operations quickier, but not much. As
far as it is kept as a *support tool*, it is fine. However, IMO, it
should not be considered as the primary modus operandi - this nourishes
wrong expectations and misguided priorities.
I think we are actually in agreement. I do not propose *online* CVS for
distribution of code patches either. I agree that I prefer the patches
to be on the mailing list. I was thinking in terms of tools for the "Oh
High Maintainer of the Libraries".

Curtis
--
Curtis W. Rendon -- mithra-ogEGBHC/***@public.gmane.org
www.realtime.net/~mithra uce-***@public.gmane.org

A long memory is the most subversive idea in America.

***@H: 2666 @ 66974
Ilya Zakharevich
2003-07-09 17:54:22 UTC
Permalink
Post by Curtis W. Rendon
A code library is the central repository so that whomever is to generate
object libraries can get from the code library (CVS,RCS,whatever) a
designated group of source for the build that has been declared to all
be of the same version. A revision control system makes it easier to
manage, and revert if needed, than multiple directories of versions.
This is probably just a fact in my biography, but I never use RCS for
more than a separate file. Anything more, and having multiple
versioned directories looks much more productive (especially with
things which take hours to recompile).
Post by Curtis W. Rendon
I think we are actually in agreement. I do not propose *online* CVS for
distribution of code patches either. I agree that I prefer the patches
to be on the mailing list. I was thinking in terms of tools for the "Oh
High Maintainer of the Libraries".
Of course, how the "maintainer" actually maintains things is not of my
concern; and my personal preferences should not be taken with anything
but a grain of salt...

Hope this helps,
Ilya
Andreas Buening
2003-07-10 21:41:10 UTC
Permalink
[snip]
Post by Ilya Zakharevich
Post by Andreas Buening
Somebody mentioned that the current emx build system should be improved
if I remember correctly. I haven't looked closely at this, yet.
If we want to do this we should do it right at the beginning.
a) I know of no issue with 2.8.1 + dmake (but of course, this may be
just ignorance).
The gcc version is more or less irrelevant as long as we're talking
about C, so 2.8.1 is okay. Whether we want dmake should be discussed.
Post by Ilya Zakharevich
b) I see no reason to do this "in the beginning" (actually, I do not
see any reason to do this at all, if it requires more than man-hour of
work).
Post by Andreas Buening
Of course, it isn't necessary to have a _compatible_ release but
??? You probably missed another negative? I see no need for
uncompatible release...
I'm sorry. This statement was really misleading. I meant a 100%
forward _and_ backward compatible release so that we can be sure
that "our" emx release is identical (except bug fixes) to EM's release.


Bye,
Andreas
Ilya Zakharevich
2003-07-11 12:27:30 UTC
Permalink
Post by Andreas Buening
Post by Ilya Zakharevich
a) I know of no issue with 2.8.1 + dmake (but of course, this may be
just ignorance).
The gcc version is more or less irrelevant as long as we're talking
about C, so 2.8.1 is okay. Whether we want dmake should be discussed.
I see no need for a discussion. dmake vs make is *orthogonal* to bug
fixes and enhancements. Using portable constructs is the best; I do
not see any reason to *discuss* an update to portable makefiles.

But only *when* such an update is made available. It is not a
prerequisite. The situation is tolerable without such an update. The
situation is fine with such an update.

Is there anything else to discuss? ;-)

Ilya
Andreas Buening
2003-07-12 13:47:08 UTC
Permalink
Post by Ilya Zakharevich
Post by Andreas Buening
Post by Ilya Zakharevich
a) I know of no issue with 2.8.1 + dmake (but of course, this may be
just ignorance).
The gcc version is more or less irrelevant as long as we're talking
about C, so 2.8.1 is okay. Whether we want dmake should be discussed.
I see no need for a discussion. dmake vs make is *orthogonal* to bug
fixes and enhancements. Using portable constructs is the best; I do
not see any reason to *discuss* an update to portable makefiles.
But only *when* such an update is made available. It is not a
prerequisite. The situation is tolerable without such an update. The
situation is fine with such an update.
I see. Although some people think there should be an update nobody
is willing to do so. ;-)
So we agree that want to start with submitting bugfixes before we do
the big step - adding new features/functions?
Post by Ilya Zakharevich
Is there anything else to discuss? ;-)
Sure. :-)

- Who will run the CVS server?
- Which mailing list do we want to use to discuss the patches?
The emx list seems to be quite empty now. Maybe we can revive
it for that purpose.
- Which macros do we want support to control the header files?
I mean things like _POSIX_SOURCE, _LARGEFILE_SOURCE, _ISO_XXX
and _EMX_OLD_API versus _EMX_NEW_API (or whatever).
- Which new features/functions do we want to add directly into
the emx import libraries and which ones do we want to put
into seperate import libraries (e.g. regex to avoid name clashes)?
- When do we want to start?


Bye,
Andreas
Adrian Gschwend
2003-07-12 14:10:55 UTC
Permalink
Post by Andreas Buening
- Who will run the CVS server?
I volunteer to do that if you don't want to do it at sourceforge. But
sourceforge is fine as well for me.
Post by Andreas Buening
- Which mailing list do we want to use to discuss the patches?
The emx list seems to be quite empty now. Maybe we can revive
it for that purpose.
sounds good. We should avoid creating new mailinglists :-)

cu

Adrian
--
Adrian Gschwend
@ netlabs.org

ktk [a t] netlabs.org
-------
Free Software for OS/2 and eCS
http://www.netlabs.org
Andreas Buening
2003-07-15 22:08:32 UTC
Permalink
Post by Adrian Gschwend
Post by Andreas Buening
- Who will run the CVS server?
I volunteer to do that if you don't want to do it at sourceforge. But
sourceforge is fine as well for me.
For me, too. :-)
As long as nobody objects or tells us of any "must have" features
at sourceforge, I'd say set up a CVS repository at netlabs.
Btw, what's the "current" emx source? The sources from the gcc 2.8.1
package plus emx 0.9f? And I think we should also add the emx docs
in an extra directory (but what do we want to use as "source" language
for the docs?).

[snip]


Bye,
Andreas
Adrian Gschwend
2003-07-16 09:08:35 UTC
Permalink
Post by Andreas Buening
As long as nobody objects or tells us of any "must have" features
at sourceforge, I'd say set up a CVS repository at netlabs.
fine for me
Post by Andreas Buening
Btw, what's the "current" emx source? The sources from the gcc 2.8.1
package plus emx 0.9f? And I think we should also add the emx docs
in an extra directory (but what do we want to use as "source" language
for the docs?).
Should I create an empty repository or should I take some codebase as
initial repository? The second option means less work for the person
who would checkin the initial source otherwise.

If someone can point me to this "current" OS/2 source I will set up the
repository tomorrow. I have to hurry a bit because I will be in
holidays for three weeks soon (leaving monday). So all who would like
to have access to this repository should send me a proposal for a
username and (if you want) a password (otherwise I generate one for
you). Please send that ASAP.

btw how do we want to call the repository? simply libc?

cu

Adrian
--
Adrian Gschwend
@ netlabs.org

ktk [a t] netlabs.org
-------
Free Software for OS/2 and eCS
http://www.netlabs.org
Andreas Buening
2003-07-17 04:26:37 UTC
Permalink
[snip]
Post by Adrian Gschwend
Should I create an empty repository or should I take some codebase as
initial repository? The second option means less work for the person
who would checkin the initial source otherwise.
If someone can point me to this "current" OS/2 source I will set up the
repository tomorrow.
If I remember correctly I've installed the sources that are part
of gcc 2.8.1 from hobbes (gccsrc*.zip but I'm not sure) plus some
update from the emx fixes (the last one, i.e. 0.9f?).

The question is how we want to organize the code. Do we want just the
libc code (emx/src/lib and emx/src/os2 (?)) or also the tools and
the docs?

After thinking about this it seems reasonable for me to separate
the libc and its docs from the tools (we could name the package
emxlibc or emxlib or emxlibs or emxdll). Something like

emx/src/lib -> lib
emx/src/os2 -> os2
emx/src/include/defs.h -> lib/defs.h
emx/include -> include
(Have I missed anything important?)

The docs can be handled later. I've just looked at some of the Makefile.
At least the hardcoded paths have to be changed. Opinions?


[snip]


Bye,
Andreas
Ilya Zakharevich
2003-07-17 09:33:54 UTC
Permalink
Post by Andreas Buening
If I remember correctly I've installed the sources that are part
of gcc 2.8.1 from hobbes (gccsrc*.zip but I'm not sure) plus some
update from the emx fixes (the last one, i.e. 0.9f?).
I never needed gcc sources. Extract

http://hobbes.nmsu.edu/pub/os2/dev/emx/v0.9d/emxsrcr.zip

then extract

http://hobbes.nmsu.edu/pub/os2/dev/emx/v0.9d/emxfix04.zip

on top of it.
Post by Andreas Buening
The question is how we want to organize the code. Do we want just the
libc code (emx/src/lib and emx/src/os2 (?)) or also the tools and
the docs?
Docs are a must. emx/include from the development package should be
included too.

Tools may be nice to have too - but please, make them a separate
project. They do not need so much QA, so it does not make sense to
treat them on the same list.
Post by Andreas Buening
The docs can be handled later. I've just looked at some of the Makefile.
At least the hardcoded paths have to be changed. Opinions?
The principal lesson of software development is that if nobody
*forces* docs and source to be syncronized, they won't be. :-( It is
crucial to have docs as close to the source as possible. [With
`literate programming' as the extreme point.]

So please: no "docs later" approach.

Ilya
Ilya Zakharevich
2003-07-14 13:00:00 UTC
Permalink
Post by Andreas Buening
- Which macros do we want support to control the header files?
I mean things like _POSIX_SOURCE, _LARGEFILE_SOURCE, _ISO_XXX
and _EMX_OLD_API versus _EMX_NEW_API (or whatever).
I propose to use descriptive macros, as in _EMX_OFF_T_IS_64BIT etc.
If one discovers a way to use different macro names so that existing
./configure autodetection works, one can "mirror" these
configure-recognized names to the descriptive ones.
Post by Andreas Buening
- Which new features/functions do we want to add directly into
the emx import libraries and which ones do we want to put
into seperate import libraries (e.g. regex to avoid name clashes)?
I would think that by default everything goes to c.lib. If somebody
on the list recognizes a conflict, then the question makes sense.

Until this we should not spend time on such discussions. (Until a
giant merge, such as one with posix2, happens; but given that the
proponents of this merge are not ready to discuss *what exactly* they
want to merge, I do not think this is going to be on the agenda soon.
;-)
Post by Andreas Buening
- When do we want to start?
I will not be able to merge my "async processes after pid wrap" patch
until mid-september (left the build tree in different hemisphere
without taking diff ;-[). But there is plenty of stuff to do anyway...

Any reason for delay?

Ilya
Stefan Neis
2003-07-14 14:08:02 UTC
Permalink
Post by Ilya Zakharevich
but given that the
proponents of this merge are not ready to discuss *what exactly* they
want to merge, I do not think this is going to be on the agenda soon.
;-)
It's all available via anonymous CVS from Sourceforge. If you insist,
I _can_ send you a set of bsd man-pages, but I won't send such a
"mail bomb" to the list ...

BTW, you are right, the "link()" function is currently failing "most of
the time", but that's just a bug in the implementation and there's even a
patch to make it work, it's just not yet incorporated into Posix/2 CVS as
I want to first see if this discussion here has any result - I'd rather
incorporate patches just once. ;-)

Regards,
Stefan
Ilya Zakharevich
2003-07-15 09:23:07 UTC
Permalink
Post by Stefan Neis
It's all available via anonymous CVS from Sourceforge. If you insist,
I _can_ send you a set of bsd man-pages, but I won't send such a
"mail bomb" to the list ...
I do not care about BSD. I do not care about man-pages.

My point is very simple: obviously, no merge should happen until the
documentation *of what the things do exactly* is available. So the
proponents of the merge should at least:

a) convert the docs to EMX format;

b) have the list of implemented functions in one bulk;

c) have the list of limitations (comparing to BSD) in one bulk;

d) merge the description of limitations with entries for specific functions.

I do not see how one can discuss the merge without having lists of `b'
and `c' before our eyes. The steps `a' and `d' may be postponed
indeed - at least if one can access the BSD manpages through a
convenient URL (which I expect is available).

Hope this helps,
Ilya
Ilya Zakharevich
2003-07-15 18:52:30 UTC
Permalink
Post by Ilya Zakharevich
b) have the list of implemented functions in one bulk;
Since you insist on getting it by mail, here it is (output of
nm cExt.a and slightly edited - I considered to sort it alphabetically,
but that would break the logical grouping...)
strunvis strvis strvisx unvis vis basename dirname _tolower _toupper
isalnum isalpha isascii isblank iscntrl isdigit isgraph islower isprint
ispunct isspace isupper isxdigit toascii tolower toupper perror strerror
rename setbuffer setlinebuf arc4random arc4random_addrandom
arc4random_stir
daemon cgetcap cgetclose cgetent cgetfirst cgetmatch cgetnext cgetnum
cgetset cgetstr cgetustr getenv putenv setenv unsetenv getopt_long
getopt_long_only getsubopt heapsort mergesort qabs qdiv strtoq strtouq
radixsort sradixsort drand48 erand48 jrand48 lcong48 lrand48 mrand48
nrand48 seed48 srand48 initstate random setstate srandom srandomdev valloc
alarm ualarm alphasort scandir dlclose dlerror dlopen dlsym fts_children
fts_close fts_open fts_read fts_set getwd glob globfree lockf ftok msgctl
msgget msgrcv msgsnd nice poll group_from_gid user_from_uid realpath
insque
lfind lsearch remque tdelete tfind tsearch twalk getmode shmat shmctl
shmdt
shmget closelog openlog setlogmask syslog vsyslog dbopen hcreate hdestroy
hsearch dbm_clearerr dbm_close dbm_delete dbm_dirfno dbm_error dbm_fetch
dbm_firstkey dbm_nextkey dbm_open dbm_rdonly dbm_store dbmclose dbminit
delete fetch firstkey nextkey store mpool_close mpool_delete mpool_filter
mpool_get mpool_new mpool_open mpool_put mpool_sync regcomp catclose
catgets
catopen bm_comp bm_exec bm_free strlcpy strmode strsep memcasecmp
strcasecmp
strncasecmp strtoll strtoull getitimer setitimer getdomainname
setdomainname inet_addr inet_aton xdr_authunix_parms authnone_create
authunix_create authunix_create_default bindresvport clnt_create
clnt_pcreateerror clnt_perrno clnt_perror clnt_spcreateerror clnt_sperrno
clnt_sperror clntraw_create callrpc clnttcp_create clntudp_bufcreate
clntudp_create endrpcent getrpcbyname getrpcbynumber getrpcent setrpcent
getrpcport get_myaddress pmap_set pmap_unset pmap_getmaps pmap_getport
xdr_pmap xdr_pmaplist clnt_broadcast pmap_rmtcall xdr_rmtcall_args
xdr_rmtcallres xdr_callmsg xdr_callhdr xdr_des_block xdr_opaque_auth
xdr_replymsg svc_getreq svc_getreqset svc_getreqset2 svc_register
svc_sendreply svc_unregister svcerr_auth svcerr_decode svcerr_noproc
svcerr_noprog svcerr_progvers svcerr_systemerr svcerr_weakauth
xprt_register xprt_unregister svcraw_create svc_run registerrpc
svcfd_create svctcp_create svcudp_bufcreate svcudp_create xdr_bool
xdr_bytes xdr_char xdr_enum xdr_free xdr_int xdr_int16_t xdr_int32_t
xdr_long xdr_netobj xdr_opaque xdr_short xdr_string xdr_u_char xdr_u_int
xdr_u_int16_t xdr_u_int32_t xdr_u_long xdr_u_short xdr_union xdr_void
xdr_wrapstring xdr_array xdr_vector xdr_double xdr_float xdrmem_create
xdrrec_create xdrrec_endofrecord xdrrec_eof xdrrec_skiprecord xdr_pointer
xdr_reference xdrstdio_create xdr_domainname xdr_keydat xdr_mapname
xdr_peername xdr_valdat xdr_ypbind_binding xdr_ypbind_resp
xdr_ypbind_resptype xdr_ypbind_setdom xdr_ypmaplist xdr_ypreq_key
xdr_ypreq_nokey xdr_ypresp_all xdr_ypresp_key_val xdr_ypresp_maplist
xdr_ypresp_master xdr_ypresp_order xdr_ypresp_val xdr_ypstat yperr_string
yp_match yp_next ypprot_err xdr_ypresp_all_seq yp_all yp_bind yp_unbind
yp_first yp_get_default_domain yp_maplist yp_master yp_order sys_nerr
exit_server list_named_pipes mkfifo remove_mkfifo madvise mmap mprotect
msync munmap getpagesize getpriority getrlimit getrusage sethostid
setpriority setrlimit vlimit killpg psignal sigaddset sigblock sigdelset
sigemptyset sigfillset siginterrupt sigismember sigpause sigsetmask sigvec
strsignal sys_signame fchmod fstat lchmod lstat stat statfs chflags chown
endgrent fchflags fchown geteuid getgrent initgroups issetugid lchown
mknod
readlink setegid seteuid setgrent setregid setreuid vfork sync confstr
fpathconf getdtablesize pathconf sysconf sysctl uname wait3 wait4 abs atan
atanf ceil ceilf ceill copysign copysignf cos cosf div fabs fabsl finite
finitef floor floorf floorl fpgetmask fpgetround fpgetsticky fpsetmask
fpsetround fpsetsticky frexpl ilogb ilogbf labs ldiv logb logbf modf rint
rintf scalbn scalbnf significand significandf sin sinf sqrtl tan tanf
acosh1p acoshl asinhl atanhl coshm1 drand exp10 exp10l log1p log1pf pow2
pvec sincos asinh cbrt cbrtf erf erfc erfcf erff expm1 expm1f fabsf frexp
ldexp frexpf isinf isnan isinff isnanf ldexpf modff nextafter nextafterf
tanh tanhf acos acosh asin atan2 atanh cosh sinh acosf asinf atan2f atanhf
coshf exp expf sinhf fmod fmodf cabs hypot cabsf hypotf j0 j1 jn y0 y1 yn
gamma gamma_r lgamma lgamma_r log log10 acoshf asinhf gammaf gammaf_r j0f
j1f jnf lgammaf lgammaf_r log10f logf y0f y1f ynf pow powf drem remainder
dremf remainderf scalb scalbf sqrt sqrtf
Post by Ilya Zakharevich
c) have the list of limitations (comparing to BSD) in one bulk;
- Not all functions are well tested
- Emulating links via extended attributes is not yet incorporated.
- None else known.
I see no reason to include things about which "nothing is known".
Note: mathematical functions are essentially the same as on EMX, however
there are occasional fixes to the BSD sources, so I believe it's easier
to simply compile current BSD sources of "libm" into EMX than to monitor
changes and copy and paste them one by one into EMX.
Of course it is easier. But I do not think that this is a viable
approach. Even for the functions with no architecture dependence,
somebody need to analyse the diffs. Functions without
architecture-dependence which were NOT available on OS/2 do not need a
lot of discussion.

IMO, functions with OS/2 specific code should be discussed one-by-one
(or at least group-by-group).
Does this huge list help in any way?
Of course. Without this list it was absolutely unclear what you were
talking about...

Ilya

P.S. To speed up the discussion, this list should be broken into two
parts: functions which were available as functions or macros in
EMX, and the rest.
Stefan Neis
2003-07-16 05:58:35 UTC
Permalink
Post by Ilya Zakharevich
Post by Ilya Zakharevich
c) have the list of limitations (comparing to BSD) in one bulk;
- Not all functions are well tested
- Emulating links via extended attributes is not yet incorporated.
- None else known.
I see no reason to include things about which "nothing is known".
Nobody used the phrase "nothing is known". I said "no limitation
(comparing to BSD) is known", i.e. works as designed.
Post by Ilya Zakharevich
Of course it is easier. But I do not think that this is a viable
approach. Even for the functions with no architecture dependence,
somebody need to analyse the diffs. Functions without
architecture-dependence which were NOT available on OS/2 do not need a
lot of discussion.
That (i.e. analyzing the differences between e.g. BSD implementation of
log and EMX implementation of log) is what a group of five or more people
has been doing over the past 2 years. There are a couple of special cases
(mostly involving NaN somehow) where both implementations are slightly
different where BSD implementation is surprisingly more correct according
to Posix standard (some code like "if (arg is special) return special
value" having been added over the past three or so years, while nothing
happened to EMX), but otherwise the code is not very different (it quite
looks like EMX math library is based on some antique BSD version as
well).
Feel free to restart evaluating those files, if you feel like it,
though...
Post by Ilya Zakharevich
Does this huge list help in any way?
Of course. Without this list it was absolutely unclear what you were
talking about...
Is it that much clearer now?
Post by Ilya Zakharevich
P.S. To speed up the discussion, this list should be broken into two
parts: functions which were available as functions or macros in
EMX, and the rest.
Feel free to do so...

Regards,
Stefan
Ilya Zakharevich
2003-07-16 12:35:45 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
I see no reason to include things about which "nothing is known".
Nobody used the phrase "nothing is known". I said "no limitation
(comparing to BSD) is known", i.e. works as designed.
I see that some kind of mmap()ing is included. Given no support from
the swapper logic, I doubt it "works as designed". Likewise for
shm*(). Likewise for dlopen().

This undermines your statement a lot. I'm completely puzzled at your
intentions. We *know* that some APIs can't be implemented without
reservations. We *know* that including such skewed implementations
into the default C library can *break* a lot of programs (due to
./configure taking different decisions).

I expect that you must know all this as well. What is the point then
in stating things otherwise?

[about the math library:]
Post by Stefan Neis
Feel free to restart evaluating those files, if you feel like it,
though...
The whole point is that it's not me, but the list who should be able
to evaluate things. But maybe we can circumvent the whole discussion:
if it turns out that EM did not check things line-by-line (just
borrowed the source from somewhere without a lot of QA), why should we? ;-)

So, Eberhard, how the math-part of the C library was created?
Post by Stefan Neis
Post by Ilya Zakharevich
P.S. To speed up the discussion, this list should be broken into two
parts: functions which were available as functions or macros in
EMX, and the rest.
Feel free to do so...
My point is that whoever wants the merge should do some (minimal)
amount of work *before* the discussion of the merge may *start*. Sad,
of course, but we are working with a very fragile universe...

Thanks again for what you did already,
Ilya
Eberhard Mattes
2003-07-17 09:20:26 UTC
Permalink
Post by Ilya Zakharevich
So, Eberhard, how the math-part of the C library was created?
From scratch.
Eberhard

(P.S.: epocemx uses fdlibm.)
Stefan Neis
2003-07-17 13:38:02 UTC
Permalink
Post by Ilya Zakharevich
I see that some kind of mmap()ing is included. Given no support from
the swapper logic, I doubt it "works as designed". Likewise for
shm*().
Well, no problems have been reported with that code. So either nobody is
using it, or it works better than you do expect - at least in the
particular cases where it is being used...
Post by Ilya Zakharevich
Likewise for dlopen().
I know that I'm using dlopen/sym/close a lot myself and I don't see, why
it shouldn't work. ;-)
If you're alluding to RTLD_LAZY not being supported, then let me state
that it's unsupported under *BSD as well, so I maintain the "works as
designed" claim at least for that function. (well, FreeBSD actually might
support it, but OpenBSD currently does not - nor do a number of BSD
clones, AFAIK).
Post by Ilya Zakharevich
I expect that you must know all this as well. What is the point then
in stating things otherwise?
Underline a need for testing. Unless functions like shm* are actually
available you'll never get feedback on the quality of the implementation
and missing features. I prefer to start with the fiction that everything
just should work and if an application doesn't compile/run as intended,
one can still start fixing bugs in libc or adding work-arounds for things
that just _are_ different.
That _can_/will admittedly be fruestrating if you're recompiling something
that already was successfully ported to OS/2 (or updating to a newer
version), as I've seen on the example of Perl myself (Posix/2's poll is
apparently slightly buggy when it comes to handling stdin/out), but I do
believe that it's easier in the long run ...

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Ilya Zakharevich
2003-07-17 14:20:58 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
I see that some kind of mmap()ing is included. Given no support from
the swapper logic, I doubt it "works as designed". Likewise for
shm*().
Well, no problems have been reported with that code.
Why should we wait for reports, if we know *beforehands* about the problems?
Post by Stefan Neis
Post by Ilya Zakharevich
Likewise for dlopen().
I know that I'm using dlopen/sym/close a lot myself and I don't see, why
it shouldn't work. ;-)
How can we appreciate your comment if you refuse to explain *what it
is going to do*?! Depending on the semantic you decided to implement,
different problems will arise.
Post by Stefan Neis
If you're alluding to RTLD_LAZY not being supported
The obvious problem is how to interpret NULL given as the name of the
library.
Post by Stefan Neis
I maintain the "works as designed" claim at least for that function
Let me reiterate: this says nothing until you explain what is your design.
Post by Stefan Neis
Underline a need for testing. Unless functions like shm* are actually
available you'll never get feedback on the quality of the implementation
and missing features.
??? I'm pretty sure that most of the people on the list can guess the
*principal* problems beforehands. *Given* widely-accepted solutions
(or workarounds) to these problems, one can discuss questions of the
quality of implementation. However, IMO it does not make sense to
discuss bugfixes for something with major design flaws.
Post by Stefan Neis
I prefer to start with the fiction that everything
just should work and if an application doesn't compile/run as intended,
Sorry, I can't see how these two phrases can survive in one sentence.

If you decide to implement a couple of simple (or juicy ;-) particular
cases of how the function may be used - it is just great. Put your
solution into a library, document the limitations.

Now, when one ports an application, *and one cannot avoid* using these
particular calls, the porters can check whether the calls are used as
you expected. If so, then they can use your library.

However, the situation is completely different if you include your
function into the C library. Now ./configure scripts start to detect
the presence of the (non-fully implemented) feature automatically. Now
they start to use your implementation without any porter's
intervention; they use your implementation even if the application
itself could use some different choice of #defines which would work 100%.

In a ./configure world, only functions with 100% semantic equivalence
should be into the C library. (At least, this is a noble goal ;-]; in
practice the question of inclusion is negotiable - but functions need
to be *negotiated* on-by-one - or group-by-group.)

Even if an implementation of a function follows the letter of the
documentation, but not the spirit, including it in the C library may
be still questionable. For example, such is the situation with
mmap() [which cannot implement page discarding efficiently].

Yes, the application which *assume* that mmap() is present become
slightly easier to port (one does not need to add -D_INCLUDE_MMAP
-lmmap to the compile/link line). But the application for which
mmap() is a *choice* could become unusable: it may have contained an
#ifdef path which works perfectly well with malloc(). However, when
application thinks that mmap() is present, it starts to work very
wastefully in the mmap()-on-OS/2 context.

Hope this helps,
Ilya
Stefan Neis
2003-07-17 15:21:17 UTC
Permalink
Post by Ilya Zakharevich
Why should we wait for reports, if we know *beforehands* about the problems?
I don't, that's the problem. :-(
Post by Ilya Zakharevich
Post by Ilya Zakharevich
Likewise for dlopen().
How can we appreciate your comment if you refuse to explain *what it
is going to do*?! Depending on the semantic you decided to implement,
different problems will arise.
Let me clarify some things:
- Posix/2 is not _my_ project. In fact, I implemented close to nothing in
the whole library. Recently, I fixed a couple of bugs that were annyoing
me and made the fixed version available.
- The only code I actively contributed is for getopt_long (taken from
BSD) and current libz.
- Mostly, I'm a user of Posix/2 and very happy about it. If it would get
some more testing (instead of e.g. complaining all the time about
strcasecmp being missing in EMX), that would help improve it, which I -
as a user - would of course appreciate. ;-)
Post by Ilya Zakharevich
The obvious problem is how to interpret NULL given as the name of the
library.
Now, that you mention it, I vaguely remember that there is such a problem.
But since that's never used anyway ( ;-) ) what's the point in claiming
that dlopen doesn't work?
Post by Ilya Zakharevich
Let me reiterate: this says nothing until you explain what is your design.
Being as BSD-compatible as possible. At least I believe that was the
design by those providing the code (_not_ me).
Post by Ilya Zakharevich
Now, when one ports an application, *and one cannot avoid* using these
particular calls, the porters can check whether the calls are used as
you expected. If so, then they can use your library.
The problem is, they can't - at least not easily: configure script won't
detect the function thus it will not be used (and in the worst case,
there will be a problem at link time with multiple defined symbols).

I do realize that incomplete functions automatically being used also has
a potential of causing trouble. Maybe I'm under-estimating that potential
as practical experiments have been relatively few, but I do have the
impression that it's not as bad as you do believe (and by large).
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Ilya Zakharevich
2003-07-17 15:39:03 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
The obvious problem is how to interpret NULL given as the name of the
library.
Now, that you mention it, I vaguely remember that there is such a problem.
But since that's never used anyway ( ;-) ) what's the point in claiming
that dlopen doesn't work?
a) I never claimed that dlopen() does not work;

b) I can't claim it until you (this is plural ;-) describe *how* the
implementation is *supposed* to work;

c) I maintain two packages relying on dlopen(); one of them would
never use NULL (perl); another one would *typically* use NULL (PARI).
This is 50%. Too much for "never" (even with smilies).

d) In both cases I have my own implementation of dlopen() which cater
to particular modes of usage.

"b" is the crucial point. *All* this discussion is about your (?)
claim "just look in BSD manpages". *All* I'm trying to establish is
that this is a fallacy. It just makes no sense (with exception of
several trivial cases, such as strcasecmp()).
Post by Stefan Neis
Post by Ilya Zakharevich
Let me reiterate: this says nothing until you explain what is your design.
Being as BSD-compatible as possible. At least I believe that was the
design by those providing the code (_not_ me).
I repeat: this makes no sense. Most cases where one *can* be fully
compatible are already in EMX. The rest requires changes of semantic.
"As close as possible" says nothing if you can go in many different
directions to change the semantic.
Post by Stefan Neis
Post by Ilya Zakharevich
Now, when one ports an application, *and one cannot avoid* using these
particular calls, the porters can check whether the calls are used as
you expected. If so, then they can use your library.
The problem is, they can't - at least not easily: configure script won't
detect the function thus it will not be used (and in the worst case,
there will be a problem at link time with multiple defined symbols).
If ./configure is able to detect that something is missing, then it
should be able to substitute something else (otherwise why check for
it)?

Moreover, it is trivial to ask configure to check additional libraries.

Hope this helps (and sorry that this discussion looks as directed
personally at you),
Ilya
Stefan Neis
2003-07-17 16:11:30 UTC
Permalink
Post by Ilya Zakharevich
I repeat: this makes no sense. Most cases where one *can* be fully
compatible are already in EMX.
str(n)casecmp, usleep, nanosleep, special cases of almost any mathematical
function are just the exceptions I can think of within the first 15
seconds and without having a list of functions before my eyes.
Post by Ilya Zakharevich
If ./configure is able to detect that something is missing, then it
should be able to substitute something else (otherwise why check for
it)?
Sure. But what is it supposed to do if e.g. both usleep and nanosleep
are missing? Typically, you get a link error after compiling for a couple
of hours. Great.
Post by Ilya Zakharevich
Hope this helps (and sorry that this discussion looks as directed
personally at you),
I'm taking it that way since I seem to be the only one interested in
integrating the results of Posix/2 projects into libcNG. But it's way to
much work for a single person with rather limited time (and -even worse
- limited knowledge about all the internals) to do all those things that
you consider to be necessary. So I'll just have to stay with libc + add-on
library (and yes, it was really easy to recompile gcc.exe to add cExt to
the list of libraries to be used to make configure happy, but I really
would have appreciated to not be forced to waste my time with this). But
then, I don't care much about debugging EMX DLL's as I can override any
buggy function with a different implementation, anyway.

Regards,
Stefan
--
Micro$oft is not an answer. It is a question. The answer is 'no'.
Andreas Buening
2003-07-15 22:08:39 UTC
Permalink
Ilya Zakharevich wrote:

[snip]
Post by Ilya Zakharevich
My point is very simple: obviously, no merge should happen until the
documentation *of what the things do exactly* is available. So the
a) convert the docs to EMX format;
b) have the list of implemented functions in one bulk;
c) have the list of limitations (comparing to BSD) in one bulk;
d) merge the description of limitations with entries for specific functions.
I do not see how one can discuss the merge without having lists of `b'
and `c' before our eyes. The steps `a' and `d' may be postponed
indeed - at least if one can access the BSD manpages through a
convenient URL (which I expect is available).
This makes sense. A patch should contain of

- a ChangeLog entry (who did what and when)
- a short description (why and what)
- the patch itself
- the updated docs (if necessary)
- the updated Makefile (if necessary; can also be done by someone else
if the contributor of the patch doesn't know how to do it).


Bye,
Andreas
Stefan Neis
2003-07-16 06:03:22 UTC
Permalink
Post by Andreas Buening
This makes sense. A patch should contain of
- a ChangeLog entry (who did what and when)
- a short description (why and what)
- the patch itself
- the updated docs (if necessary)
- the updated Makefile (if necessary; can also be done by someone else
if the contributor of the patch doesn't know how to do it).
You do realize that it would take decades to incorporate all the missing
functions from Posix/2 that way, don't you?
I guess, that's the point where I definitely loose interest in doing the
project this way (or even at all, actually, though that later part might
change once I calm down again).

Regards,
Stefan
Ilya Zakharevich
2003-07-16 12:46:33 UTC
Permalink
Post by Stefan Neis
Post by Andreas Buening
- a ChangeLog entry (who did what and when)
- a short description (why and what)
- the patch itself
- the updated docs (if necessary)
- the updated Makefile (if necessary; can also be done by someone else
if the contributor of the patch doesn't know how to do it).
You do realize that it would take decades to incorporate all the missing
functions from Posix/2 that way, don't you?
You do realize that some of my 2-3 lines fixes to EMX sit in the queue
for *years* now? And while I think that EM had chosen a wrong model
for the EMX development, this is only a part of the reason. There
*are* reasons why not all things should be harried.
Post by Stefan Neis
I guess, that's the point where I definitely loose interest in doing the
project this way.
This would be a big loss. But I hope that in a long term you might be
able to reconsider.

Currently, I do not expect that all of the "your" library can be
merged in one step. But maybe you will be able to layer the list of
APIs into parts which require different level of discussion/QA, and
some parts of this may be merged quick.

E.g., as I said, I think that the functions with no EMX specific
implementation, and which were not in EMX before, can be merged after
a minimal discussion (and a documentation conversion) only.

Hope this helps,
Ilya
Andreas Buening
2003-07-17 04:27:00 UTC
Permalink
Post by Stefan Neis
Post by Andreas Buening
This makes sense. A patch should contain of
- a ChangeLog entry (who did what and when)
- a short description (why and what)
- the patch itself
- the updated docs (if necessary)
- the updated Makefile (if necessary; can also be done by someone else
if the contributor of the patch doesn't know how to do it).
You do realize that it would take decades to incorporate all the missing
functions from Posix/2 that way, don't you?
I realize that this would require a lot of work. Neither do I expect
that we just have to do copy&paste from posix/2 nor do I expect that
you do all the work. The current problem is that even a one-line-change
is a long-term and basically undoable project. If we get the stage
that everbody can apply patches instead of writing his own emx clone
then we've already made huge progress. Im my opinion the idea is that
necessary functions are added by people who need them. If nobody adds
a specific function then it's obviously not urgently needed. Then we
can discuss those functions later. I also had a lot of stuff I'd like
to see in emx and I'll have no moral problems to grab some code from
posix/2. ;-)
Post by Stefan Neis
I guess, that's the point where I definitely loose interest in doing the
project this way (or even at all, actually, though that later part might
change once I calm down again).
So calm down, please. :-)


Bye,
Andreas
Ilya Zakharevich
2003-07-17 09:25:47 UTC
Permalink
Post by Andreas Buening
The current problem is that even a one-line-change
is a long-term and basically undoable project. If we get the stage
that everbody can apply patches instead of writing his own emx clone
then we've already made huge progress.
IMO, this stage is nothing. The first usable stage: we need a
maintainer, who

a) will provide a "linear" sequence of snapshots (instead of hundreds
of "orthogonal" patches);

b) will have power to say "no, I do not think I want to incorporate
this in th current stage";

c) will use the power of `b' to nag people to improve things;

d) In (very rare?) situations when several patches - which he deems
worthwhile - conflict, and he can't nag people to make them
compatible, he will fix them.

"b" is the hardest part, since it assumes very deep attention to what
happens on the list.

Read my lips: without such a position filled, things would not work.

Just reiterating things again,
Ilya
Andreas Buening
2003-06-23 20:13:39 UTC
Permalink
Post by Stefan Neis
Post by Ilya Zakharevich
Post by Holger Veit
handle big ones. You surely want to add such garbage to autoconf/automake
rulesets, don't you?
??? It is already there, since all the major players do it this way.
Sure,
CXXLAGS="-DSOME_LARGE_FILE_MACRO" CFLAGS="-DSOME_LARGE_FILE_MACRO" ./configure
Great! And if some sub-library has (not) been using the flag while you did
(not), the stuff that you will get after hours of configure/make process
is just garbage. It works as long as _everybody_ is either ignoring the
flag or _everybody_ is using it.
To avoid this I'd say we should do it the other way around and make
the "new" (reasonable) behaviour the default. People (== programmers)
should be aware that the next emx (or whatever) release will be a major
one. If still anybody needs 32bit support he is free to do so and to
define 20 macros for this purpose. We could provide a big
_EMX_OLD_BEHAVIOUR macro that includes all other "old style" macros but
that's it.

And, of course, there has to be the definition that all UnixOS/2 (tm)
libraries have to use e.g. large file support.


[snip]

Bye,
Andreas
Sebastian Wittmeier (ShadoW)
2003-06-22 14:14:40 UTC
Permalink
Hi,
IMHO that discussion seems a bit chaotic.
I wonder, which advantages both proposals have (Holger's vs. Illya's).

Holger's - new DLLs
----------------------------
advantage: maximum Unix compatibility for newly compiled programs
advantage: every old application will continue to work (with old DLLs)
disadvantage: old applications won't support new concepts/features

Illya's - improve existing DLLs
------------------------------------------
advantage: more Unix compatibility for newly compiled programs (GCCOPT)
advantage: even old applications profit from certain changes
disadvantage: not fully compatible, some programs may not work any
longer

Why is the interoperability between old and new applications needed
(mentioned by EM)?
Where do existing libraries fit into that picture?

Can't these concepts be combined?
- create new DLLs (like Holger's approach)
- test old programs, if they work with new semantics
- rename EMX DLL names in executables (but no recompiling needed)

Sebastian
Eberhard Mattes
2003-06-21 20:44:37 UTC
Permalink
Post by Holger Veit
I don't think anyone needs backward compatibility at all for a DLL which
is then not even named EMX*.DLL even if it stemmed from EMX and maybe
shared 90% of the code. Old apps can still resolve against old EMX*.DLLs,
and for new apps one has to clearly decide what he wants
I guess you need some hooks in emx.dll so that unixos2.dll can access
the process table (and maybe other stuff) of emx.dll to enable
interoperation between old and new apps.

Eberhard
Eberhard Mattes
2003-06-19 09:11:23 UTC
Permalink
Post by Holger Veit
Since most Unixes started broken by concept with the old 'long' offset
type
At least one of them started with a 32-bit offset broken into two
16-bit words) -- guess the meaning of 'l' in 'lseek()'... There was a
time before 'long'. Ditto for 64-bit offsets. Hard disks grew faster
than anyone had ever dreamed.
Post by Holger Veit
The fseek code will itself fall down to a 64 bit compliant lseek(), and
this will, depending on the presence of Dos*L routines or not (loadable
DLL module) decide whether to accept the call or not if an offset >2GB is
specified.
Probably not relevant: you don't know beforehand whether the resulting
offset will be >= 2GB.

Eberhard
Sebastian Wittmeier (ShadoW)
2003-06-22 16:40:23 UTC
Permalink
Post by Andreas Buening
Post by Holger Veit
No one will seriously use EMX for writing "OS/2" software anymore; if at
all, programmers keep their copies of VAC++ or Watcom, or maybe even
CSET/2 and MASM for that (or stay entirely out of the C/C++ ara and use
Java, for instance - ignored here). If there are still efforts, they
are concentrating on porting work. Guess where the most important repository
for portable software is. So, what tools does one need?
There is not much choice anymore with compilers on OS/2.
So writing OS/2 programs with gcc is more important than ever.
Post by Andreas Buening
Hello? The presence of an _chdir2 entry point prevents the programmer
in porting Unix applications?
I like drive letters. You can quickly switch between them, and each has
its own current directory. The concept of current directory is
important - all commonly used operating systems have it; with drive
letters you get more then one. They don't have to reflect real hard
drives; there is TVFS, Netdrive, LVM, ...

Sebastian
Ilya Zakharevich
2003-06-23 09:56:50 UTC
Permalink
Post by Sebastian Wittmeier (ShadoW)
Post by Andreas Buening
Hello? The presence of an _chdir2 entry point prevents the programmer
in porting Unix applications?
I like drive letters. You can quickly switch between them, and each has
its own current directory. The concept of current directory is
important - all commonly used operating systems have it; with drive
letters you get more then one. They don't have to reflect real hard
drives; there is TVFS, Netdrive, LVM, ...
IIRC, from (circa) WinNT on Win32 the current directory is not
localized per drive. CMD.EXE has its own logic to emulate the old
behaviour.

[All this from memory of discussions long ago on perl5-porters.]

Hope this helps,
Ilya
Continue reading on narkive:
Loading...