facebook
twitter
linkedin
email

Should the order of import statements matter when importing a .so?

I am getting the following import error when trying to load a python module compiled using boost python.

ImportError: /path/to/library/libxml2.so.2: symbol gzopen64, version ZLIB_1.2.3.3 not defined in file libz.so.1 with link time reference

Strangely I don’t see this error if that is the non standard module to be imported. i.e If I import other module and then this module, it fails with import error. Not sure what’s going wrong or how to debug.

Edit:
To exactly show the issue:

$ python -c 'import json, libMYBOOST_PY_LIB' # DOES NOT WORK!!!
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: path/to/xml_library/libxml2.so: symbol gzopen64, version ZLIB_1.2.3.3 not defined in file libz.so.1 with link time reference
$ python -c 'import libMYBOOST_PY_LIB, json' # WORKS NOW!!!
$

Its not just json, few other modules also cause the same issue when imported before my module. eg. urllib2

Best answer

The order of import statements matter.

As documented in the python language reference:

Once the name of the module is known (unless otherwise specified, the term “module” will refer to both packages and modules), searching for the module or package can begin. The first place checked is sys.modules, the cache of all modules that have been imported previously. If the module is found there then it is used in step (2) of import.

Any module can change:

  • sys.modules – the cache of all modules previously imported
  • sys.path – the search path for modules

And they can change import hooks as well:

  • sys.meta_path
  • sys.path_hooks
  • sys.path_importer_cache

The import hooks can provide you the ability to load modules from zip files, any kind of archive files, from the network, etc.

import libMYBOOST_PY_LIB

This statement is going to modify sys.modules for sure, loading its dependencies into the module cache. It may modify sys.path too. It is actually very common for frameworks (e.g. boost, zope, django, requests…) to ship with batteries included / with a copy of the modules they depend on.

  • django ships with json
  • requests ships with urllib3

To see exactly what the library will load you can use:

python -v -c 'import libMYBOOST_PY_LIB'