runtime choice of the dynamic loader

[u***@aetey.se]

Hello,

The dynamic loader path is currently being hardwired into the pcc executable.

I find this too restrictive, runtime configurability of the path is
sometimes important. I suggest a patch like the following (a more general
solution for configurability would be probably better but this patch is
sufficent for my purposes) :

--- cc/cc/cc.c.ori 2014-08-24 21:03:38.420292533 +0200
+++ cc/cc/cc.c 2014-08-24 21:17:43.274929318 +0200
@@ -995,6 +995,9 @@
/*
* Linker
*/
+ if (dynlinker[0] && dynlinker[1]) /* there is some default */
+ if ((t=getenv("PCC_DYNAMIC_LINKER"))) /* we want to change it */
+ dynlinker[1] = t;
setup_ld_flags();
if (run_linker())
exandrm(0);

This may result in some dead code if dynlinker is not set to be present
but does not lead to a crash (the dead code is possibly optimized away?
this can otherwise be done differently, via macros).

Of course, a corresponding description should be added in the documentation.

------- some background:

In general, I see hardwiring the dynamic loader path into the produced
executables as a bad design. "Properly used" executables (started by a
runtime loader explicitly) do not need this so the configurability of the
unused string is not an issue per se. I could let pcc always hardwire
"/:-b" :)

Unfortunately there are too many scripts which assume that a freshly
compiled executable is runnable "as-is" even if it is dynamically
linked. Hardwiring in the executable a path to the loader (which
happens to be available at that path at the moment) helps to satisfy
these oversimplified assumptions.

(Such scripts otherwise usually have to go out of their way to add
LD_LIBRARY_PATH or other means to find the needed shared libraries, they
could exactly as well add an explicit runtime loader to the command -
but unfortunately they often do not).

Regards,
Rune

[Joerg Sonnenberger]

Post by u***@aetey.se
I find this too restrictive, runtime configurability of the path is
sometimes important. I suggest a patch like the following (a more general
solution for configurability would be probably better but this patch is

This feels wrong, can't you just add -Wl,--dynamic-linker,foo?

Joerg

[u***@aetey.se]

Post by Joerg Sonnenberger

Post by u***@aetey.se
I find this too restrictive, runtime configurability of the path is
sometimes important. I suggest a patch like the following (a more general
solution for configurability would be probably better but this patch is

This feels wrong, can't you just add -Wl,--dynamic-linker,foo?

Then I assume you get two conflicting directives to the linker and you
have to know which one will be honored, I feel that supplying the right
and only one is cleaner.

Or does pcc "driver" recognize special cases like that and substitute
the arguments at the "right" place? I may have missed this (?) but I doubt
that it does.

In any case, defining environment variables is much simpler and more
robust than fiddling with the arguments (which have much less regular
syntax and interrelations). I tried (and have in different situations
to use) both approaches and know from the experience which one is easier
and more reliable.

Rune

[u***@aetey.se]

Post by u***@aetey.se
In any case, defining environment variables is much simpler and more
robust than fiddling with the arguments (which have much less regular
syntax and interrelations). I tried (and have in different situations
to use) both approaches and know from the experience which one is easier
and more reliable.

May be I should explain more clearly: this is not for one time manual
compiles, this is for wrapping the compiler for use in regular production.
It matters how reliable the wrapping is against all the possible arguments
the users and/or scripts can supply.

The environment variable does not influence any semantics, just moves the
decision about the default dynamic linker from the compiler compile time
to the compiler wrap time, a crucial difference in practice.

Rune

[Joerg Sonnenberger]

Post by u***@aetey.se

Post by Joerg Sonnenberger

Post by u***@aetey.se
I find this too restrictive, runtime configurability of the path is
sometimes important. I suggest a patch like the following (a more general
solution for configurability would be probably better but this patch is

This feels wrong, can't you just add -Wl,--dynamic-linker,foo?

Then I assume you get two conflicting directives to the linker and you
have to know which one will be honored, I feel that supplying the right
and only one is cleaner.

The latter one wins. If that is not the user supplied one, it's a bug in
the driver.

Post by u***@aetey.se
In any case, defining environment variables is much simpler and more
robust than fiddling with the arguments (which have much less regular
syntax and interrelations). I tried (and have in different situations
to use) both approaches and know from the experience which one is easier
and more reliable.

I completely disagree on this. Environment variables are a head ache and
introduce more problems than they ever solve. LD_LIBRARY_PATH is a
wonderful example for that.

Joerg

[u***@aetey.se]

Post by Joerg Sonnenberger

Post by u***@aetey.se
robust than fiddling with the arguments (which have much less regular
syntax and interrelations). I tried (and have in different situations
to use) both approaches and know from the experience which one is easier
and more reliable.

I completely disagree on this. Environment variables are a head ache and
introduce more problems than they ever solve. LD_LIBRARY_PATH is a
wonderful example for that.

:)

You refer to a wonderful example of a fundamentally broken design,
but actually out of context.

I know pretty well why LD_LIBRARY_PATH (and LD_PRELOAD for that matter)
are not to be used - and consequently I actually use the explicit
library-path _argument_ to the runtime loader, but this has very little
in common with the topic.

(if you disagree, show a practical scenario where the dynamic-linker
variable would unavoidably lead to a broken behaviour, like
LD_LIBRARY_PATH does; what would be the _worst_ outcome?? :)

In other words, yes, command line arguments are the best or the only
acceptable approach in certain cases. The same goes for environment
variables.

I am open to discussing this in-depth, but this may be offtopic for the
list. Here I just share my experience, point of view and a suggestion
as a deployer of pcc.

Rune

[u***@aetey.se]

Post by u***@aetey.se
In other words, yes, command line arguments are the best or the only
acceptable approach in certain cases. The same goes for environment
variables

in other cases, like this one.

Rune

[Rich Felker]

Post by u***@aetey.se
Hello,
The dynamic loader path is currently being hardwired into the pcc executable.
I find this too restrictive, runtime configurability of the path is
sometimes important. I suggest a patch like the following (a more general
solution for configurability would be probably better but this patch is
--- cc/cc/cc.c.ori 2014-08-24 21:03:38.420292533 +0200
+++ cc/cc/cc.c 2014-08-24 21:17:43.274929318 +0200
@@ -995,6 +995,9 @@
/*
* Linker
*/
+ if (dynlinker[0] && dynlinker[1]) /* there is some default */
+ if ((t=getenv("PCC_DYNAMIC_LINKER"))) /* we want to change it */
+ dynlinker[1] = t;

Forgive me if I missed this before, but is there a reason
-Wl,-dynamic-linker,... doesn't work? Or do you just want a clean way
to set a default configuration rather than having to add LDFLAGS all
over the place?

Rich

[Thorsten Glaser]

Post by Rich Felker
-Wl,-dynamic-linker,... doesn't work? Or do you just want a clean way
to set a default configuration rather than having to add LDFLAGS all
over the place?

This is all but clean. Environment variables for things like this
aren’t the Unix way.

The -Wl,* approach has the problem that -Wl,* does not parse the
options passed down to the linker, so ld(1) sees -dynamic-linker
twice. Should not be a problem in reality, though.

bye,
//mirabilos

[u***@aetey.se]

Post by Thorsten Glaser
This is all but clean. Environment variables for things like this
aren’t the Unix way.

The original Unix way is actually static linking.
I could argue that dynamic linking is not the Unix way :)

It is the way of assuming a single dynamic loader for all purposes
and the highly flawed decision to hardcode its path into every
executable which you probably mean by the "Unix way". (?)

It creates arbitrary constraints which I do not want to (and do not)
accept. Yes it is the drawbacks of such a way which I want to avoid.

Post by Thorsten Glaser
The -Wl,* approach has the problem that -Wl,* does not parse the
options passed down to the linker, so ld(1) sees -dynamic-linker
twice. Should not be a problem in reality, though.

You still say that this method is cleaner than using a dedicated variable
with a quite unambiguous semantics and scope?

E.g. what about a different ld(1) ? Is it the Unix way to assume that
all linkers are GNU ld and treat the arguments exactly in the same way? :)

Actually the pcc driver makes this assumption but at least my proposal
does not make it worse. Adding extra ld flags makes it worse (i.e. more
fragile).

Regards,
Rune

[Thorsten Glaser]

Post by u***@aetey.se
I could argue that dynamic linking is not the Unix way :)

I could not agree more ;)

Post by u***@aetey.se
It is the way of assuming a single dynamic loader for all purposes
and the highly flawed decision to hardcode its path into every
executable which you probably mean by the "Unix way". (?)

No, what I mean is: use flags not environment variables.

Post by u***@aetey.se
You still say that this method is cleaner than using a dedicated variable
with a quite unambiguous semantics and scope?

Yes.

Post by u***@aetey.se
E.g. what about a different ld(1) ? Is it the Unix way to assume that
all linkers are GNU ld and treat the arguments exactly in the same way? :)

To the compiler driver, -Wl,* just passes variables to the linker.
This is the interface. Which option the linker takes is target-
specific. On systems with GNU ld, they take -dynamic-linker foo,
and it’s those systems *you* are going to override that on – if
you’re on other systems, use -Wl,-something-else instead.

Post by u***@aetey.se
Actually the pcc driver makes this assumption but at least my proposal

It doesn’t. It’s the target definition which defines the
default flags; if you port pcc to a platform with an ld(1)
not supporting this option, you’d obviously leave it out.
It looks like you do not have much experience in porting
compilers to platforms…

bye,
//mirabilos

[u***@aetey.se]

Post by Thorsten Glaser

Post by u***@aetey.se
E.g. what about a different ld(1) ? Is it the Unix way to assume that
all linkers are GNU ld and treat the arguments exactly in the same way? :)

specific. On systems with GNU ld, they take -dynamic-linker foo,
and it’s those systems *you* are going to override that on – if
you’re on other systems, use -Wl,-something-else instead.

Then how do you know that this overrides the first -dynamic-loader?
This is just a quite arbitrary assumption which pcc driver code makes.

Post by Thorsten Glaser

Post by u***@aetey.se
Actually the pcc driver makes this assumption but at least my proposal

It doesn’t. It’s the target definition which defines the
default flags; if you port pcc to a platform with an ld(1)

The order and the expected semantics of the flags is in fact hardcoded,
isn't it? I do not say that this is "wrong" per se, it is just the way
it is, making and hardcoding certain assumptions for convenience.

Post by Thorsten Glaser
not supporting this option, you’d obviously leave it out.

Wait, you assume among others that there is a single and only ld(1)
and that its features are known at pcc building time. Neither of these
facts is there for granted. Or otherwise you mean by "platform" something
quite restricted, without specifying the constraints (and/or possibly
not realizing some of them?).

Post by Thorsten Glaser
It looks like you do not have much experience in porting
compilers to platforms…

:-D

No better arguments than stating that it is you who is knowledgeable?

(Oh well you probably did not mean this but I couldn't help)

Have a nice day,
Rune

[Thorsten Glaser]

Post by u***@aetey.se
Wait, you assume among others that there is a single and only ld(1)
and that its features are known at pcc building time.

Of course there is. Differences will just have to be a different
platform. Unless pcc gains something like gcc’s spec files.

You may notice that GNU/Linux-glibc and GNU/Linux-µClibc are,
indeed, distinct GCC platforms.

bye,
//mirabilos

[u***@aetey.se]

Post by Thorsten Glaser

Post by u***@aetey.se
Wait, you assume among others that there is a single and only ld(1)
and that its features are known at pcc building time.

Of course there is. Differences will just have to be a different
platform. Unless pcc gains something like gcc’s spec files.

Feel free to call every toolchain a "platform", but actually it is
desirable and very much possible (talking about C) to mix object and
executable files prepared by different tools. I usually associate
with the word "platform" a certain runtime milieu, not a toolchain.

It is also useful to be able to combine the tools freely as well as
the executables and object files.

The hardcoded borders between the different implementations of compatible
toolchains are unnecessary and harmful, they reflect the historical
practice of hardwiring a lot instead of runtime configurability.

You happen to perceive these artificial borders as something natural
while I am suggesting to improve the situation.

Post by Thorsten Glaser
You may notice that GNU/Linux-glibc and GNU/Linux-µClibc are,
indeed, distinct GCC platforms.

This depends on the corresponding definitions.

Again, by your expression you seem to assume that there is a single
and true way of thinking about things (otherwise it is impossible to
"notice" :)

I tried to explain that a different approach brings advantages.
Now I am done with this due effort and leave the topic.

Take care,
Rune

[Thorsten Glaser]

Post by u***@aetey.se
Feel free to call every toolchain a "platform", but actually it is
desirable and very much possible (talking about C) to mix object and
executable files prepared by different tools. I usually associate

You cannot mix object files from different C libraries. The dynamic
linker is normally part of the libc package so…

Anyway: too much softcoding is *also* bad, not just hardcoding.
Optimise for the common case to keep bugs low.

bye,
//mirabilos

[u***@aetey.se]

Post by Thorsten Glaser

Post by u***@aetey.se
Feel free to call every toolchain a "platform", but actually it is
desirable and very much possible (talking about C) to mix object and
executable files prepared by different tools. I usually associate

You cannot mix object files from different C libraries. The dynamic
linker is normally part of the libc package so…

To avoid misunderstanding (I didn't expect it but apparently I was
not clear enough)

The expression "different tools" above refers to various implementations
of macroprocessors, compilers, optimizers, assemblers, linkers and
similar.

It does not denote libraries and interfaces (these also can happen to
be compatible with each other, like musl and a subset of glibc, but this
is not what I had in mind).

Rune

[u***@aetey.se]

Post by Rich Felker
-Wl,-dynamic-linker,... doesn't work? Or do you just want a clean way
to set a default configuration rather than having to add LDFLAGS all
over the place?

I feel you expressed it very well.

Yes I need a clean way to specify a default configuration or
several ones.

The actual path to the "most suitable" dynamic linker is yet undefined
at the compiler compilation time and subject to change too.

Rune

[Anders Magnusson]

Is there any previous art in this area? I.e. how is the same done with gcc?

-- Ragge

Post by u***@aetey.se
Hello,
The dynamic loader path is currently being hardwired into the pcc executable.
I find this too restrictive, runtime configurability of the path is
sometimes important. I suggest a patch like the following (a more general
solution for configurability would be probably better but this patch is
--- cc/cc/cc.c.ori 2014-08-24 21:03:38.420292533 +0200
+++ cc/cc/cc.c 2014-08-24 21:17:43.274929318 +0200
@@ -995,6 +995,9 @@
/*
* Linker
*/
+ if (dynlinker[0] && dynlinker[1]) /* there is some default */
+ if ((t=getenv("PCC_DYNAMIC_LINKER"))) /* we want to change it */
+ dynlinker[1] = t;
setup_ld_flags();
if (run_linker())
exandrm(0);
This may result in some dead code if dynlinker is not set to be present
but does not lead to a crash (the dead code is possibly optimized away?
this can otherwise be done differently, via macros).
Of course, a corresponding description should be added in the documentation.
In general, I see hardwiring the dynamic loader path into the produced
executables as a bad design. "Properly used" executables (started by a
runtime loader explicitly) do not need this so the configurability of the
unused string is not an issue per se. I could let pcc always hardwire
"/:-b" :)
Unfortunately there are too many scripts which assume that a freshly
compiled executable is runnable "as-is" even if it is dynamically
linked. Hardwiring in the executable a path to the loader (which
happens to be available at that path at the moment) helps to satisfy
these oversimplified assumptions.
(Such scripts otherwise usually have to go out of their way to add
LD_LIBRARY_PATH or other means to find the needed shared libraries, they
could exactly as well add an explicit runtime loader to the command -
but unfortunately they often do not).
Regards,
Rune
_______________________________________________
Pcc mailing list
http://lists.ludd.ltu.se/cgi-bin/mailman/listinfo/pcc

[Rich Felker]

For changing the dynamic linker on a per-invocation basis with GCC,
you'd use -Wl. There may also be an environment variable to do so, but
I think that's just s feature of one particular toolchain variant I've
seen (perhaps Aboriginal Linux's toolchain? I don't remember). For
configuring it on a global basis, it's part of the built-in target
profile GCC is configured for, or you can use gcc -dumpspecs to dump a
specs file and put it in the gcc lib directory with the name "specs"
and edit it to specify your preferred default (for dynamic linker and
many other things).

Rich

Post by Anders Magnusson
Is there any previous art in this area? I.e. how is the same done with gcc?
-- Ragge

Post by u***@aetey.se
Hello,
The dynamic loader path is currently being hardwired into the pcc executable.
I find this too restrictive, runtime configurability of the path is
sometimes important. I suggest a patch like the following (a more general
solution for configurability would be probably better but this patch is
--- cc/cc/cc.c.ori 2014-08-24 21:03:38.420292533 +0200
+++ cc/cc/cc.c 2014-08-24 21:17:43.274929318 +0200
@@ -995,6 +995,9 @@
/*
* Linker
*/
+ if (dynlinker[0] && dynlinker[1]) /* there is some default */
+ if ((t=getenv("PCC_DYNAMIC_LINKER"))) /* we want to change it */
+ dynlinker[1] = t;
setup_ld_flags();
if (run_linker())
exandrm(0);
This may result in some dead code if dynlinker is not set to be present
but does not lead to a crash (the dead code is possibly optimized away?
this can otherwise be done differently, via macros).
Of course, a corresponding description should be added in the documentation.
In general, I see hardwiring the dynamic loader path into the produced
executables as a bad design. "Properly used" executables (started by a
runtime loader explicitly) do not need this so the configurability of the
unused string is not an issue per se. I could let pcc always hardwire
"/:-b" :)
Unfortunately there are too many scripts which assume that a freshly
compiled executable is runnable "as-is" even if it is dynamically
linked. Hardwiring in the executable a path to the loader (which
happens to be available at that path at the moment) helps to satisfy
these oversimplified assumptions.
(Such scripts otherwise usually have to go out of their way to add
LD_LIBRARY_PATH or other means to find the needed shared libraries, they
could exactly as well add an explicit runtime loader to the command -
but unfortunately they often do not).
Regards,
Rune
_______________________________________________
Pcc mailing list
http://lists.ludd.ltu.se/cgi-bin/mailman/listinfo/pcc

_______________________________________________
Pcc mailing list
http://lists.ludd.ltu.se/cgi-bin/mailman/listinfo/pcc

[u***@aetey.se]

Post by Rich Felker
For changing the dynamic linker on a per-invocation basis with GCC,
you'd use -Wl. There may also be an environment variable to do so, but
I think that's just s feature of one particular toolchain variant I've
seen (perhaps Aboriginal Linux's toolchain? I don't remember). For
configuring it on a global basis, it's part of the built-in target
profile GCC is configured for, or you can use gcc -dumpspecs to dump a
specs file and put it in the gcc lib directory with the name "specs"
and edit it to specify your preferred default (for dynamic linker and
many other things).

This is what I am doing with gcc - generating the appropriate specs
file and artificially prepending the corresponding -specs=... argument,
which does not feel pretty/clean (putting it in the gcc lib directory
might be a possibility but would interfere with reusing the directory).

Post by Rich Felker

Post by Anders Magnusson
Is there any previous art in this area? I.e. how is the same done with gcc?

Rune

[Antoine Leca]

Post by u***@aetey.se
--- cc/cc/cc.c.ori 2014-08-24 21:03:38.420292533 +0200
+++ cc/cc/cc.c 2014-08-24 21:17:43.274929318 +0200

Why should this go into the MI driver? Could it not be put into the
ccconfig.h platform-dependent driver "configuration" stuff?

Post by u***@aetey.se
@@ -995,6 +995,9 @@
/*
* Linker
*/
+ if (dynlinker[0] && dynlinker[1]) /* there is some default */
+ if ((t=getenv("PCC_DYNAMIC_LINKER"))) /* we want to change it */
+ dynlinker[1] = t;

This is blindly assuming that the selection of the dynamic linker is
achieved by just changing the 2nd term of the dynlinker[] array, and
only this term.
It is probably true for Linux and would probably apply to any the usual
ELF suspects, and is completely irrelevant and innocuous to platforms
such as Windows or V7 where the dynamic linker is out of reach of the
compiler wills; but I wonder if it is reasonable to require it to all
the platforms (thus my recommendation above to move that elsewhere.)

Post by u***@aetey.se
Of course, a corresponding description should be added in the documentation.

Sure. Particularly since I guess the security zealots might see the
dependence to an external environment variable for the *produced*
executable as a security hazard.

Post by u***@aetey.se
Unfortunately there are too many scripts which assume that a freshly
compiled executable is runnable "as-is" even if it is dynamically
linked. Hardwiring in the executable a path to the loader (which
happens to be available at that path at the moment) helps to satisfy
these oversimplified assumptions.

This is just a very specific use case, an example, isn't it?
I certainly can see more useful cases for the feature (and I guess there
can be designed some evil ones as well), but I thought the main use case
was to have an easy way to externally reconfigure pcc (perhaps a binary
version of pcc brought from some "ports" or "depot") to fit the target
environment at hand.


Antoine