Sphinx fails on python bindings documentation generation:
PYTHONPATH="/Users/jenkins/test/babeltrace/src/bindings/python/bt2/build/build_lib" LD_LIBRARY_PATH="/Users/jenkins/test/babeltrace/src/lib/.libs" /opt/local/bin/python3 -m sphinx -b html -E ./source build/html
Running Sphinx v2.1.2
Configuration error:
There is a programmable error in your configuration file:
Traceback (most recent call last):
File "/Users/jenkins/test/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/native_bt.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 670, in _load_unlocked
File "<frozen importlib._bootstrap>", line 583, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 1043, in create_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: dlopen(/Users/jenkins/test/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/_native_bt.cpython-37m-darwin.so, 2): Library not loaded: /usr/local/lib/libbabeltrace2.0.dylib
Referenced from: /Users/jenkins/test/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/_native_bt.cpython-37m-darwin.so
Reason: image not found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/sphinx/config.py", line 361, in eval_config_file
execfile_(filename, namespace)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/sphinx/util/pycompat.py", line 86, in execfile_
exec(code, _globals)
File "/Users/jenkins/test/babeltrace/doc/bindings/python/source/conf.py", line 3, in <module>
import bt2
File "/Users/jenkins/test/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/__init__.py", line 26, in <module>
from bt2.clock_class import *
File "/Users/jenkins/test/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/clock_class.py", line 23, in <module>
from bt2 import native_bt, object, utils
File "/Users/jenkins/test/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/native_bt.py", line 17, in <module>
_native_bt = swig_import_helper()
File "/Users/jenkins/test/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/native_bt.py", line 16, in swig_import_helper
return importlib.import_module('_native_bt')
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_native_bt'
Sphinx simply fails to find libbabeltrace2.0.dylib since that
on macOSX LD_LIBRARY_PATH have no meaning for the dynamic linker,
DYLD_LIBRARY_PATH does.
We could also use DYLD_FALLBACK_LIBRARY_PATH.
The dyld manual have this note regarding System Integrity Protection
(SIP).
Note: If System Integrity Protection is enabled, these environment
variables are ignored when executing binaries protected by System
Integrity Protection.
This patch was tested on our builder and with SIP enabled without any
problem:
ci-node-macosx-01:babeltrace jenkins$ system_profiler SPSoftwareDataType
Software:
System Software Overview:
System Version: macOS 10.14.5 (18F203)
Kernel Version: Darwin 18.6.0
Boot Volume: Macintosh HD
Boot Mode: Normal
Computer Name: ci-node-macosx-01
User Name: Jenkins CI (jenkins)
Secure Virtual Memory: Enabled
System Integrity Protection: Enabled
Time since boot: 32 days 3:36
This mostly works because the python3 executed is not flagged
restricted:
ci-node-macosx-01:~ jenkins$ ls -laO /opt/local/bin/python3
lrwxr-xr-x 1 root admin - 24 27 Jun 14:16 /opt/local/bin/python3 -> /opt/local/bin/python3.7
The same command would most probably not work if using a
"system" python.
ci-node-macosx-01:~ jenkins$ ls -laO /usr/bin/python
-rwxr-xr-x 1 root wheel restricted,compressed 66880 4 May 02:49 /usr/bin/python
I was unable to test this scenario.
User building on macOSX could face this issue in the future. I am not sure
how to mitigate this.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: Ia828ac133e2c05b614427f03064c8db02f34f741
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1652
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
PYTHON_BT2_BUILD_LIB_DIR = $(abs_top_builddir)/src/bindings/python/bt2/build/build_lib
PP = $(PYTHON_BT2_BUILD_LIB_DIR)
LLP = $(abs_top_builddir)/src/lib/.libs
-SPHINXBUILD = PYTHONPATH="$(PP)" LD_LIBRARY_PATH="$(LLP)" $(PYTHON) -m sphinx
+SPHINXBUILD = PYTHONPATH="$(PP)" LD_LIBRARY_PATH="$(LLP)" DYLD_LIBRARY_PATH="$(LLP)" $(PYTHON) -m sphinx
SPHINX_SRC = $(wildcard $(SPHINX_SOURCE_DIR)/*.rst)
$(SPHINX_HTML_TARGET): $(SPHINX_SRC)