build: make out-of-tree builds with Python bindings work on msys2
authorSimon Marchi <simon.marchi@efficios.com>
Thu, 11 Apr 2024 16:57:08 +0000 (12:57 -0400)
committerMichael Jeanson <mjeanson@efficios.com>
Wed, 17 Apr 2024 15:07:42 +0000 (11:07 -0400)
When doing an out of tree build on msys2 with
`--enable-python-bindings`, where the path to configure is absolute, I
get:

    Making all in bindings/python/bt2
    make[3]: Entering directory '/home/smarchi/build/babeltrace-mingw-w64-ucrt-x86_64/src/bindings/python/bt2'
    touch copy-static-deps.stamp
      SWIG      bt2/native_bt.c
    CC="ccache gcc" CFLAGS="-IC:/msys64/ucrt64/include/glib-2.0 -IC:/msys64/ucrt64/lib/glib-2.0/include -fvisibility=hidden -pthread -IC:/msys64/ucrt64/include/glib-2.0 -IC:/msys64/ucrt64/lib/glib-2.0/include   -Wall -Wextra -Wmissing-prototypes -Wmissing-declarations -Wnull-derefere
nce -Wundef -Wredundant-decls -Wshadow -Wjump-misses-init -Wsuggest-attribute=format -Wnested-externs -Wwrite-strings -Wformat=2 -Wstrict-aliasing -Wmissing-noreturn -Winit-self -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wsuggest-override -Wno-sign-compare -Wno-cast-functi
on-type -Wno-maybe-uninitialized -Wold-style-definition -Wstrict-prototypes -g -O2 -Wno-shadow -Wno-null-dereference -Wno-deprecated-declarations -Wno-redundant-decls -Wno-missing-field-initializers -Wno-unused-parameter -Wno-undef" CPPFLAGS="-I/home/smarchi/src/babeltrace/include -I
../../../../src -I/home/smarchi/src/babeltrace/src -include common/config.h  -I/home/smarchi/src/babeltrace/src/bindings/python/bt2/bt2" LDFLAGS="-L../../../../src/lib/.libs  -lgmodule-2.0 -lglib-2.0 -lintl -LC:/msys64/ucrt64/lib -lpython3.11  -lversion -lshlwapi -lpathcch -lbcrypt -
lm   -lgmodule-2.0 -lglib-2.0 -lintl" /ucrt64/bin/python ./setup.py build --force
    [...]
    ccache gcc -IC:/msys64/ucrt64/include/glib-2.0 -IC:/msys64/ucrt64/lib/glib-2.0/include -fvisibility=hidden -pthread -IC:/msys64/ucrt64/include/glib-2.0 -IC:/msys64/ucrt64/lib/glib-2.0/include -Wall -Wextra -Wmissing-prototypes -Wmissing-declarations -Wnull-dereference -Wundef -Wr
edundant-decls -Wshadow -Wjump-misses-init -Wsuggest-attribute=format -Wnested-externs -Wwrite-strings -Wformat=2 -Wstrict-aliasing -Wmissing-noreturn -Winit-self -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wsuggest-override -Wno-sign-compare -Wno-cast-function-type -Wno-ma
ybe-uninitialized -Wold-style-definition -Wstrict-prototypes -g -O2 -Wno-shadow -Wno-null-dereference -Wno-deprecated-declarations -Wno-redundant-decls -Wno-missing-field-initializers -Wno-unused-parameter -Wno-undef -IC:/msys64/ucrt64/include/glib-2.0 -IC:/msys64/ucrt64/lib/glib-2.0
/include -fvisibility=hidden -pthread -IC:/msys64/ucrt64/include/glib-2.0 -IC:/msys64/ucrt64/lib/glib-2.0/include -Wall -Wextra -Wmissing-prototypes -Wmissing-declarations -Wnull-dereference -Wundef -Wredundant-decls -Wshadow -Wjump-misses-init -Wsuggest-attribute=format -Wnested-ext
erns -Wwrite-strings -Wformat=2 -Wstrict-aliasing -Wmissing-noreturn -Winit-self -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wsuggest-override -Wno-sign-compare -Wno-cast-function-type -Wno-maybe-uninitialized -Wold-style-definition -Wstrict-prototypes -g -O2 -Wno-shadow -W
no-null-dereference -Wno-deprecated-declarations -Wno-redundant-decls -Wno-missing-field-initializers -Wno-unused-parameter -Wno-undef -IC:/msys64/home/smarchi/src/babeltrace/include -I../../../../src -I/home/smarchi/src/babeltrace/src -include common/config.h -I/home/smarchi/src/bab
eltrace/src/bindings/python/bt2/bt2 -IC:/msys64/ucrt64/include/python3.11 -c /home/smarchi/src/babeltrace/src/bindings/python/bt2/bt2/logging.c -o build/temp.mingw_x86_64_ucrt-3.11/home/smarchi/src/babeltrace/src/bindings/python/bt2/bt2/logging.o
    cc1.exe: warning: command-line option '-Wsuggest-override' is valid for C++/ObjC++ but not for C
    cc1.exe: fatal error: /home/smarchi/src/babeltrace/src/bindings/python/bt2/bt2/logging.c: No such file or directory
    compilation terminated.
    error: command 'C:\\msys64\\ucrt64\\bin/ccache.exe' failed with exit code 1
    make[3]: *** [Makefile:735: build-python-bindings.stamp] Error 1
    make[3]: Leaving directory '/home/smarchi/build/babeltrace-mingw-w64-ucrt-x86_64/src/bindings/python/bt2'

For 'in-tree' builds all the paths are relative and thus compatible with
native windows binaries while in an 'out-of-tree' build some of the
paths are absolute and thus incompatible. The MSYS2 shell does a lot of
'magic' conversion [1] on paths passed as arguments and through environment
variables but the heuristics are limited and it doesn't apply when
binaries are called directly without using the shell.

The first issue is the include paths in the CFLAGS and CPPFLAGS
environment variables which don't get converted because they start with
'-I'.

The second issue is that the path to 'logging.c' in the generated
'setup.py is absolute:

    sources=["bt2/native_bt.c", "/home/smarchi/src/babeltrace/src/bindings/python/bt2/bt2/logging.c"],

When distutils invokes gcc to compile this file, it spawns it directly,
not through a shell, so no automatic Unix -> Windows path conversion
takes place.

Fix both of those issues by modifying 'setup.py' to detect the MSYS2
environment and convert absolute path to Windows style using the
'cygpath' command.

[1] https://www.msys2.org/docs/filesystem-paths/

Change-Id: I9003878a6582a0dc3f4ee18b07885f332348b8bf
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/12397
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
src/bindings/python/bt2/setup.py.in

index 0d1ca724c38f910b43bb0f805bc9bb68238da148..3c81447ea80a2d8dd928f43aba69feea2bf7f980 100644 (file)
@@ -5,6 +5,8 @@
 
 import sys
 import os
+import shutil
+import subprocess
 
 # Distutils was removed in Python 3.12, use setuptools as an alternative.
 if sys.version_info >= (3, 12):
@@ -114,10 +116,44 @@ def our_get_config_vars(*args):
 sysconfig.get_config_vars = our_get_config_vars
 
 
+# Returns 'True' when running on a MinGW system.
+def is_mingw():
+    return sys.platform == "win32" and shutil.which("cygpath") != None
+
+
+# On MinGW systems run 'cygpath -m' on 'path', on other systems return 'path' as-is.
+def cygpath_m(path: str):
+    if is_mingw():
+        return subprocess.check_output(
+            'cygpath -m "{}"'.format(path), shell=True, encoding="utf-8"
+        ).strip("\n")
+
+    return path
+
+
+# On MinGW systems, check CFLAGS and CPPFLAGS for absolute include paths
+# (starts with '-I/') and convert them to valid Windows paths using cygpath.
+if is_mingw():
+    for flagvar in ["CFLAGS", "CPPFLAGS"]:
+        cur_flags = os.getenv(flagvar)
+        if cur_flags != None:
+            new_flags = ""
+            for flag in cur_flags.split():
+                if flag.startswith("-I/"):
+                    flag = "-I{}".format(cygpath_m(flag[2:]))
+
+                new_flags += " {}".format(flag)
+
+            os.environ[flagvar] = new_flags
+
+
 def main():
     babeltrace_ext = Extension(
         "bt2._native_bt",
-        sources=["bt2/native_bt.c", "@srcdir@/bt2/logging.c"],
+        sources=[
+            "bt2/native_bt.c",
+            cygpath_m("@srcdir@/bt2/logging.c"),
+        ],
         libraries=["babeltrace2", "glib-2.0"],
         extra_objects=[
             "@top_builddir@/src/autodisc/.libs/libautodisc.a",
This page took 0.026217 seconds and 4 git commands to generate.