SitesLinksOttawaLifePhotosTravelToolsJournalBlog
See More Stuff
Wednesday, September 18, 2013

Shared Library Symbol Conflicts (on Solaris)

This is the solaris version of Shared Library Symbol Conflicts on Linux. Refer to that post for full details. This diagram shows the problem.

Symbol Conflicts

Summary of Switches

docs.oracle.com

-c            Directs the CC driver to suppress linking with ld and, instead, produce a .o file for each source file.
-o filename   Sets the name of the output file
-xar          Creates archive libraries.
-s            Strip the symbol table from the executable file. This option removes all symbol information from output executable files. This option is passed to ld.
-Lpath        Adds path to the library search paths. This option is passed	to ld.
-llib         Add library liblib.a or liblib.so to linker's	list of search libraries.
-G            Build a dynamic shared library instead of an executable file; see the ld(1) man page and the C++ User's Guide.
-h[ ]lname    Assigns the name lname to the generated shared dynamic library. In general, the name after -h should be exactly the same as the one after	-o.
-xldscope={v} If you do not	specify	-xldscope, the compiler	assumes -xldscope=global
   hidden     Hidden linker scoping is more restric-tive than symbolic and global linker scoping. All references within a dynamic load module will bind to a definition within that module. The symbol will not be visible outside of the module.

Build up to the Full Demo

work.cxx

#include <iostream>
void DoThing()
{
  printf("work \n");
}

main.cxx

#include <iostream>
void DoThing();
int main()
{
  printf("start \n");
  DoThing();
  printf("finished \n");
  return 0;
}

I'm using an old compiler:

# /isv/sw6.0_u2p/SUNWspro/bin/CC -V
CC: Sun WorkShop 6 update 2 C++ 5.3 Patch 111685-06 2002/03/09

Build this simple app:

/isv/sw6.0_u2p/SUNWspro/bin/CC -c work.cxx -o work.o
/isv/sw6.0_u2p/SUNWspro/bin/CC -xar -o libwork.a work.o

/isv/sw6.0_u2p/SUNWspro/bin/CC -c main.cxx -o main.o
/isv/sw6.0_u2p/SUNWspro/bin/CC -xar -o libmain.a main.o

The next command will fail because of the order on the link line:

# /isv/sw6.0_u2p/SUNWspro/bin/CC -s -L. -o main.exe -lwork -lmain
Undefined                       first referenced
 symbol                             in file
void DoThing()                   ./libmain.a(main.o)
ld: fatal: Symbol referencing errors. No output written to main.exe

This is the correct way:

/isv/sw6.0_u2p/SUNWspro/bin/CC -s -L. -o main.exe -lmain -lwork
./main.exe
start
work
finished

conflict.cxx

#include <iostream>
void DoThing()
{
  printf("conflict \n");
}

Show the conflict in the static lib scenario:

/isv/sw6.0_u2p/SUNWspro/bin/CC -c conflict.cxx -o conflict.o
/isv/sw6.0_u2p/SUNWspro/bin/CC -xar -o libconflict.a conflict.o

First link with priority given to work:

/isv/sw6.0_u2p/SUNWspro/bin/CC -s -L. -o main.exe -lmain -lwork -lconflict
./main.exe
start
work
finished

Next link with priority given to conflict:

/isv/sw6.0_u2p/SUNWspro/bin/CC -s -L. -o main.exe -lmain -lconflict -lwork
./main.exe
start
conflict
finished

Now show the conflict in the shared object scenario:

rm libconflict.a

/isv/sw6.0_u2p/SUNWspro/bin/CC -G -h libconflict.so conflict.o -o libconflict.so
/isv/sw6.0_u2p/SUNWspro/bin/CC -s -L. -o main.exe -lmain -lconflict

export LD_LIBRARY_PATH=.
./main.exe
start
conflict
finished

Add a layer to the shared library, so we can later control the symbols, and have main call through that layer.

layer.cxx

#include <iostream>
void DoThing();
void DoLayer()
{
  printf("layer \n");
  DoThing();
}
/isv/sw6.0_u2p/SUNWspro/bin/CC -c layer.cxx -o layer.o
/isv/sw6.0_u2p/SUNWspro/bin/CC -G -h libconflict.so layer.o conflict.o -o libconflict.so

main.cxx

#include <iostream>
void DoLayer();
int main()
{
  printf("start \n");
  DoLayer();
  printf("finished \n");
  return 0;
}
/isv/sw6.0_u2p/SUNWspro/bin/CC -c main.cxx -o main.o
/isv/sw6.0_u2p/SUNWspro/bin/CC -xar -o libmain.a main.o
/isv/sw6.0_u2p/SUNWspro/bin/CC -s -L. -o main.exe -lmain -lconflict

export LD_LIBRARY_PATH=.
./main.exe
start
layer
conflict
finished

Now modify main so that it is calling DoThing from both work and conflict.

main.cxx

#include <iostream>
void DoThing();
void DoLayer();
int main()
{
  printf("start \n");
  DoThing();
  DoLayer();
  printf("finished \n");
  return 0;
}

We now have the problem shown in the diagram above.

/isv/sw6.0_u2p/SUNWspro/bin/CC -c main.cxx -o main.o
/isv/sw6.0_u2p/SUNWspro/bin/CC -xar -o libmain.a main.o work.o
/isv/sw6.0_u2p/SUNWspro/bin/CC -s -L. -o main.exe -lmain -lconflict

export LD_LIBRARY_PATH=.
./main.exe
start
work
layer
work
finished

Changing the order on the link line reverses the problem.

/isv/sw6.0_u2p/SUNWspro/bin/CC -s -L. -o main.exe -lconflict -lmain

export LD_LIBRARY_PATH=.
./main.exe
start
conflict
layer
conflict
finished

We can see that currently all symbols are visible.

/usr/ccs/bin/nm -C layer.o
layer.o:

[Index]   Value      Size    Type  Bind  Other Shndx   Name

[2]     |         0|       0|SECT |LOCL |0    |3      |
[1]     |         0|       0|FILE |LOCL |0    |ABS    |layer.cxx
[4]     |         0|       0|NOTY |GLOB |0    |UNDEF  |printf
[5]     |        16|      44|FUNC |GLOB |0    |2      |void DoLayer()
                                                       [__1cHDoLayer6F_v_]
[3]     |         0|       0|NOTY |GLOB |0    |UNDEF  |void DoThing()
                                                       [__1cHDoThing6F_v_]
/usr/ccs/bin/nm -C conflict.o
conflict.o:

[Index]   Value      Size    Type  Bind  Other Shndx   Name

[2]     |         0|       0|SECT |LOCL |0    |3      |
[1]     |         0|       0|FILE |LOCL |0    |ABS    |conflict.cxx
[3]     |         0|       0|NOTY |GLOB |0    |UNDEF  |printf
[4]     |        16|      36|FUNC |GLOB |0    |2      |void DoThing()
                                                       [__1cHDoThing6F_v_]
/usr/ccs/bin/nm -C libconflict.so
libconflict.so:

[Index]   Value      Size    Type  Bind  Other Shndx   Name

[22]    |         0|       0|SECT |LOCL |0    |21     |
[21]    |         0|       0|SECT |LOCL |0    |20     |
[20]    |         0|       0|SECT |LOCL |0    |19     |
...
[55]    |      1224|      44|FUNC |GLOB |0    |7      |void DoLayer()
                                                       [__1cHDoLayer6F_v_]
[42]    |      1288|      36|FUNC |GLOB |0    |7      |void DoThing()
                                                       [__1cHDoThing6F_v_]
[46]    |         0|       0|NOTY |WEAK |0    |UNDEF  |void __Cimpl::cplus_fini()
                                                       [__1cH__CimplKcplus_fini6F_v_]
[45]    |         0|       0|NOTY |WEAK |0    |UNDEF  |void __Cimpl::cplus_init()
                                                       [__1cH__CimplKcplus_init6F_v_]
[52]    |         0|       0|NOTY |WEAK |0    |UNDEF  |void __Crun::do_exit_code_in_range(void*,void*)
                                                       [__1cG__CrunVdo_exit_code_in_range6Fpv1_v_]

The Linux Fix doesn't work on Solaris

You might naively try the same fix that worked on Linux:

# /isv/sw6.0_u2p/SUNWspro/bin/CC -fvisibility=hidden -c layer.cxx -o layer.o
CC: Warning: Option -fvisibility=hidden passed to ld, if ld is invoked, ignored otherwise

But this error really means: "I don't know about this switch, so I'm ignoring it"

# /isv/sw6.0_u2p/SUNWspro/bin/CC -fake=hidden -c layer.cxx -o layer.o
CC: Warning: Option -fake=hidden passed to ld, if ld is invoked, ignored otherwise

The correct switch is -xldscope. forums.oracle.com discusses -fvisibility=hidden vs. -xldscope=hidden and sites gcc.gnu.org which claims that the option -fvisibility=hidden has the same effect as the Sun CC option -xldscope=hidden.

Unfortunately, -xldscope isn't supported by my ancient compiler:

# /isv/sw6.0_u2p/SUNWspro/bin/CC -xldscope=hidden -c layer.cxx -o layer.o
CC: Warning: Option -xldscope=hidden passed to ld, if ld is invoked, ignored otherwise

# /isv/sw6.0_u2p/SUNWspro/bin/CC -V
CC: Sun WorkShop 6 update 2 C++ 5.3 Patch 111685-06 2002/03/09

# /isv/sw6.0_u2p/SUNWspro/bin/CC -help
... some omitted ...
-Bdynamic             Allow dynamic linking
-Bstatic              Require static linking
-Bsymbolic            Bind references to global symbols to their definitions
-c                    Compile only; produce .o files but suppress linking
-dryrun               Show compiler commands built by driver; no compilation
-d{y|n}               Allow or disallow dynamic libraries for the entire executable
-filt[=<a>[,<a>...]]  Control the filtering of ld output
-flags                Same as -xhelp=flags
-G                    Build a dynamic shared library
-g                    Compile for debugging
-H                    Print path name of each file included during compilation
-h<name>              Assign <name> to generated dynamic shared library
-help                 Same as -xhelp=flags
-I<path>              Add path to #include file search paths
-i                    Tell linker to ignore any LD_LIBRARY_PATH setting
-L<path>              Pass to linker to add <path> to the library search paths
-l<name>              Add lib<name>.a or lib<name>.so to linker search list
-mc                   Remove duplicate strings from .comment section of output files
-mr                   Remove all strings from .comment section of output files
-o <outputfile>       Set name of output file or executable to <outputfile>
-readme               Same as -xhelp=readme
-s                    Strip symbol table from the executable file
-V                    Same as -verbose=version
-verbose=<a>[,<a>...] Control verbosity during compilation
-w                    Suppress most compiler warning messages
-xar                  Create archive library with instantiated templates
-xhelp=<f>            Display on-line help information; <f>={flags|readme}
-xpg                  Compile for profiling with gprof
-z <x>                Link editor option
All other options are passed down to ld.

bugs.freedesktop.org says:
-xldscope=<a> Indicates the appropriate linker scoping within the source program; <a>={global|symbolic|hidden}
-xldscope=hidden will do what -fvisibility=hidden does in gcc.
-xldscope was first introduced in Studio 8 compiler.

Most of my solaris boxes are old:

# uname -a
SunOS mybox 5.8 Generic_117350-60 sun4u sparc SUNW,Sun-Fire-V210
# /isv/sw6.0_u2p/SUNWspro/bin/CC -V
CC: Sun WorkShop 6 update 2 C++ 5.3 Patch 111685-06 2002/03/09

Here's a newer one:

# uname -a
SunOS mybox 5.10 Generic_142909-17 sun4v sparc SUNW,SPARC-Enterprise-T5120

# /usr/bin/CC -V
CC: Sun C++ 5.10 SunOS_sparc 128228-09 2010/06/24
Usage: CC [ options ] files.  Use 'CC -flags' for details

# /usr/bin/CC -flags | grep xldscope
-xldscope=<a>         Indicates the appropriate linker scoping within the source program; <a>={global|symbolic|hidden}

Build our demo on the new box:

/usr/bin/CC -c work.cxx -o work.o
/usr/bin/CC -c main.cxx -o main.o
/usr/bin/CC -c conflict.cxx -o conflict.o
/usr/bin/CC -c layer.cxx -o layer.o
/usr/bin/CC -xar -o libmain.a main.o work.o
/usr/bin/CC -G -h libconflict.so layer.o conflict.o -o libconflict.so
/usr/bin/CC -s -L. -o main.exe -lmain -lconflict

LD_LIBRARY_PATH=.
export LD_LIBRARY_PATH
./main.exe
start
work
layer
work
finished

Now build with all symbols hidden:

/usr/bin/CC -xldscope=hidden -c conflict.cxx -o conflict.o
/usr/bin/CC -xldscope=hidden -c layer.cxx -o layer.o
/usr/bin/CC -G -h libconflict.so layer.o conflict.o -o libconflict.so
/usr/bin/CC -s -L. -o main.exe -lmain -lconflict

The final compile fails because no symbols are exported.

Undefined                       first referenced
 symbol                             in file
void DoLayer()                   ./libmain.a(main.o)
ld: fatal: Symbol referencing errors. No output written to main.exe

I'd like to use the same pattern matching symbol file solutions that I used on Linux, but it's not supported.

vi export.sym

{
 global: *DoLayer*;
 local: *;
};

/usr/bin/CC -c conflict.cxx -o conflict.o
/usr/bin/CC -c layer.cxx -o layer.o

/usr/bin/CC -G -Wl,--version-script=export.sym -h libconflict.so layer.o conflict.o -o libconflict.so

gnu.org describes the use of --version-script, but code.google.com shows it being removed from solaris builds because it's not supported on solaris.

This contradicts the authoratative akkadia.org which states that solaris does support --version-script. From airs.com it appears that linker symbol versioning was invented at sunc but enhanced by Ulrich Drepper in the GNU linker. It sounds like solaris supports symbol versioning for the purpose of having two versions of the same function in a single .so (for backwards compatibility), not for the purpose of preventing private symbols from being exported.

stackoverflow.com finally explains that on solaris you can either use the GCC compile or the Solaris-Studio compiler. Thus GNU style version-script is supported on Solaris when you use the GNU style compile/link, not when you use the native compile/link.

That doesn't mean that native solaris doesn't have support for symbol barriers, just that they don't support the GNU style --version-script option.

oracle.com describes solaris capabilities:
- Until the release of the Sun Studio 8 compilers, linker mapfiles were the only way to change the default symbol processing by the linker.
- The mapfile mechanism is useful with languages such as C, but difficult to exploit with languages such as C++.
- Sun added linker scoping as a language extension with the release of the Sun Studio 8 C/C++ compilers.
- Although the symbol visibility is specified in the source file, it actually defines how a symbol can be accessed once it has become part of an executable or shared object.
- There is a new compiler flag: -xldscope={ global| symbolic| hidden}
- There is a new C/C++ source language interface: __global, __symbolic, and __hidden declaration specifiers were introduced to specify symbol visibility at declarations and definitions of external symbols and class types.
- The mixed use of -xldscope=hidden and __symbolic will yield the same effect as __declspec(dllexport) in DLLs on Windows (explained in the later part of the article).
- Consider the following example: __hidden struct __symbolic BinaryTree node;
- Many of the linker scoping bugs were already fixed and distributed as patches since the release of Sun Studio 8.
- Sun Studio 9 compilers introduced a new keyword called __declspec and supports dllexport and dllimport storage-class attributes (or specifiers)
- Syntax: storage... __declspec( dllimport ) type declarator...
- On Sun Solaris, __declspec(dllimport) maps to __global and the __declspec(dllexport) maps to the __symbolic specifier.

Solaris Solution

In summary: we can't use the script option on native solaris and Ulrich explains that the --version-script option produces less efficient code, so the best thing may be to compile with -xldscope=hidden/-fvisibility=hidden to hide everything not explicitly marked as public (i.e. default) in the source.

Here's how it works:

# /usr/bin/CC -V
CC: Sun C++ 5.10 SunOS_sparc 128228-09 2010/06/24
Usage: CC [ options ] files.  Use 'CC -flags' for details

layer.cxx

#include <iostream>
void DoThing();
__attribute__ ((visibility ("default"))) void DoLayer()
{
  printf("layer \n");
  DoThing();
}
/usr/bin/CC -xldscope=hidden -c conflict.cxx -o conflict.o
/usr/bin/CC -xldscope=hidden -c layer.cxx -o layer.o
/usr/bin/CC -G -h libconflict.so layer.o conflict.o -o libconflict.so
/usr/bin/CC -s -L. -o main.exe -lmain -lconflict

LD_LIBRARY_PATH=.
export LD_LIBRARY_PATH
./main.exe
start
work
layer
conflict
finished

layer.cxx

#include <iostream>
void DoThing();
__symbolic void DoLayer()
{
  printf("layer \n");
  DoThing();
}
/usr/bin/CC -xldscope=hidden -c layer.cxx -o layer.o
/usr/bin/CC -G -h libconflict.so layer.o conflict.o -o libconflict.so
/usr/bin/CC -s -L. -o main.exe -lmain -lconflict
./main.exe
start
work
layer
conflict
finished

layer.cxx

#include <iostream>
void DoThing();
__declspec(dllexport) void DoLayer()
{
  printf("layer \n");
  DoThing();
}
/usr/bin/CC -xldscope=hidden -c layer.cxx -o layer.o
/usr/bin/CC -G -h libconflict.so layer.o conflict.o -o libconflict.so
/usr/bin/CC -s -L. -o main.exe -lmain -lconflict
./main.exe
start
work
layer
conflict
finished

Thus, with the native solaris compiler/linker, you can use a compile setting to switch the default visibility to hidden:

/usr/bin/CC -xldscope=hidden -c layer.cxx -o layer.o

Then use any of these syntaxes to mark a particualr symbol as public:

__attribute__ ((visibility ("default"))) void DoLayer()
__symbolic                               void DoLayer()
__declspec(dllexport)                    void DoLayer()

Class Visibility

Nothing is ever easy. On Linux we see that the above export visibility statements for file-scope functions also works on entire classes. But with solaris it's not the same. Luckily it's soved by a little syntax juggling.

exports.h

#include <iostream>
void __attribute__ ((visibility ("default"))) Farewell();
class __attribute__ ((visibility ("default"))) Greeter
{
public:
  void greet();
};

exports.cxx

#include "exports.h"
void Farewell() { printf("goodbye\n"); }
void Greeter::greet() { printf("hello\n"); }

main.cxx

#include "exports.h"
int main()
{
  Greeter bob;
  bob.greet();
  Farewell();
}

Compile:

/usr/bin/CC -xldscope=hidden -c exports.cxx -o exports.o
/usr/bin/CC -G -h libexports.so exports.o -o libexports.so
/usr/bin/CC -c main.cxx -o main.o
/usr/bin/CC -L. main.o -o main.exe -lexports

What? The compile of main.exe failed!

Undefined                       first referenced
 symbol                             in file
void Greeter::greet()             main.o
ld: fatal: Symbol referencing errors. No output written to main.exe

Let's try without the xldscope flag:

/usr/bin/CC -c exports.cxx -o exports.o
/usr/bin/CC -G -h libexports.so exports.o -o libexports.so
/usr/bin/CC -c main.cxx -o main.o
/usr/bin/CC -L. main.o -o main.exe -lexports

export LD_LIBRARY_PATH=.
./main.exe

Works like a charm:

hello
goodbye

Okay, so it seems that __attribute__ ((visibility ("default"))) doesn't work on solaris when applied to an entire class. What about __declspec(dllexport)? After some thrashing, I've come to the conclusion that when you're compiling the .o for the .so, you need __declspec(dllexport) in the .h, but when you're compiling the .o for the .exe, you must not have the __declspec(dllexport) in the .h, and as far as I can tell, there's no permutation where __attribute__ ((visibility ("default"))) can be made to work. To be clear, here's my compiler version:

# CC -V
CC: Sun C++ 5.10 SunOS_sparc 128228-09 2010/06/24
Usage: CC [ options ] files.  Use 'CC -flags' for details

exports.h

#include <iostream>
void __attribute__ ((visibility ("default"))) Farewell();
class __declspec(dllexport) Greeter
{
public:
  void greet();
};

clean.h

#include <iostream>
void __attribute__ ((visibility ("default"))) Farewell();
class Greeter
{
public:
  void greet();
};

main.cxx

#include "clean.h"
int main()
{
  Greeter bob;
  bob.greet();
  Farewell();
}
/usr/bin/CC -xldscope=hidden -c exports.cxx -o exports.o
/usr/bin/CC -G -h libexports.so exports.o -o libexports.so
/usr/bin/CC -c main.cxx -o main.o
/usr/bin/CC -L. main.o -o main.exe -lexports
./main.exe

Works like a charm:

hello
goodbye

__declspec(dllexport) also works on static functions.

exports.h

#include <iostream>
void __declspec(dllexport) Farewell();
class __declspec(dllexport) Greeter
{
public:
  void greet();
};

clean.h

#include <iostream>
void Farewell();
class Greeter
{
public:
  void greet();
};
/usr/bin/CC -xldscope=hidden -c exports.cxx -o exports.o
/usr/bin/CC -G -h libexports.so exports.o -o libexports.so
/usr/bin/CC -c main.cxx -o main.o
/usr/bin/CC -L. main.o -o main.exe -lexports
./main.exe
hello
goodbye

But __attribute__ ((visibility ("default"))) does not.

exports.h

#include <iostream>
void __attribute__ ((visibility ("default"))) Farewell();
class __attribute__ ((visibility ("default"))) Greeter
{
public:
  void greet();
};
/usr/bin/CC -xldscope=hidden -c exports.cxx -o exports.o
/usr/bin/CC -G -h libexports.so exports.o -o libexports.so
/usr/bin/CC -c main.cxx -o main.o
/usr/bin/CC -L. main.o -o main.exe -lexports
Undefined                       first referenced
 symbol                             in file
void Greeter::greet()             main.o
ld: fatal: Symbol referencing errors. No output written to main.exe

So we have a means on Solaris to control symbol export by decorating a single class, but does it solve the symbol conflict problem described above? Yes. Note that due to a runtime crash related to export of statics I switched to __global instead of __declspec(dllexport), which seems to have the benifet of working on both functions and classes and doesn't cause runtime crashes when static member variables of an exported class are accessed.

Conflict Prevented With Classes

conflict.cxx

#include <iostream>
void DoThing()
{
  printf("conflict \n");
}

layer.h

class __global MyClass
{
public:
  void DoLayer();
};

layer_clean.h

class MyClass
{
public:
  void DoLayer();
};

layer.cxx

#include <iostream>
#include "layer.h"
void DoThing();
void MyClass::DoLayer()
{
  printf("layer \n");
  DoThing();
}

work.cxx

#include <iostream>
void DoThing()
{
  printf("work \n");
}

main.cxx

#include <iostream>
#include "layer_clean.h"
void DoThing();
int main()
{
  printf("start \n");
  DoThing();
  MyClass m;
  m.DoLayer();
  printf("finished \n");
  return 0;
}
/usr/bin/CC -xldscope=hidden -c conflict.cxx -o conflict.o
/usr/bin/CC -xldscope=hidden -c layer.cxx -o layer.o
/usr/bin/CC -G -h libconflict.so layer.o conflict.o -o libconflict.so

/usr/bin/CC -c work.cxx -o work.o
/usr/bin/CC -c main.cxx -o main.o
/usr/bin/CC -xar -o libmain.a work.o main.o

/usr/bin/CC -L. main.o -o main.exe -lmain -lconflict
export LD_LIBRARY_PATH=.
./main.exe
start
work
layer
conflict
finished
/usr/bin/CC -L. main.o -o main.exe -lconflict -lmain
./main.exe
start
work
layer
conflict
finished