Implement rseq numa node id field selftest

Test the NUMA node id extension rseq field. Compare it against the value
returned by the getcpu(2) system call while pinned on a specific core.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ia01752b2bdd3ab9bf5ef4f2aef12bd1704169e7a

rseq.h: implement mm_cid and node_id APIs

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic82b240e654702416936d612a03d8675f9fb869c

Update rseq.c to query ELF auxvec

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I562b07aac2c7277eb8be3a31782d402e1d693be1

Cleanup: rseq.h: add extra space around __VA_ARGS__

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Iec19b59c40b513be33e5ab3207d947c05037adb0

Add mm_cid and numa node id to rseq ABI header

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6def7df36e25d47027b3274cf325ce1beb841b75

Remove RSEQ_SKIP_FASTPATH code

This code is not currently build by the test Makefile, adds complexity,
and is not overall useful considering that the abort handling loops to
retry the fast-path.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I2bf5ead4c0314bb0f28de63d9e6b2a2943c59aa2

Fix: `void *` arithmetic compiler warning

Fixes warning "pointer of type ‘void *’ used in subtraction [-Wpointer-arith]"
which was observed on gcc 11.3.0 on Ubuntu 22.04 LTS Linux 6.3.0 RC7 x86-64.

This warning was enabled by the flag `-Wpedantic`.

Link: https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wpointer-arith
Signed-off-by: Gerrit Klein <therealthingy95@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Icfcf1d39bcde3730bf38c9ec6ed1521ed4503625

All arch: use rseq_unqual_scalar_typeof in load-acquire

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I08f8aedf09ac37fa9b06640a494f20cfbcfeb557

c++: compiler: use remove_cv and remove_reference in rseq_unqual_scalar_typeof

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I4c278af941c4e17fbef1b14bf97820100790b6f1

rseq_arm64: use rseq_unqual_scalar_typeof in load-acquire

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1cdb0b776f5c81d3cca8bb9fbd03f9549bf40d77

Revert "compiler: C++: simplify rseq_unqual_scalar_typeof implementation"

This reverts commit 38f60c8d69e147b5997f850e4f163c36b2e1c91a.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Revert "compiler: c++: use decltype"

This reverts commit 51e83f8f967435d40541b7ae07b932e5cb1c0d47.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

compiler: c++: use decltype

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I23e4d035029efd9bab6ca2b1bba13146c5eb0375

compiler: C++: simplify rseq_unqual_scalar_typeof implementation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I55b57c217e4338ebeb4947c3452de1422fe5f10c

rseq-arm64: Fix buggy load-acquire/store-release macros

The arm64 load-acquire/store-release macros from the Linux kernel rseq
selftests are buggy. Remplace them by a working implementation.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I84cf01cb0fabee89f76cc8d608d203c975397037

compiler: implement rseq_unqual_scalar_typeof

Allow defining variables and cast with a typeof which removes the
volatile and const qualitiers.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ibe7068e2c307ed47a2eb6db891e717fb7db47282

Use __asm__ __volatile__ in rseq_after_asm_goto for c11 compatibility

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I63cde85aeaf5efd03e613c255f31f1091c649790

architecture headers: add extra parentheses around macro parameters

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I34a68a8f420ce993b4d6ab21e3c1b86ad5ea4113

Tests: Introduce arch-mo unit tests

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ibef415f2c9d2c9356c00b11870057f74f89d75d5

Fix: gcc extension compiler warning

Use __VA_ARGS__ rather than GNU args... in macros for improved compiler
compatibility.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ie17e73019b1afae721a2270b0b43197c8dca8b88

Initialize librseq from `rseq_register_current_thread`

Invoke rseq_init() explicitly from rseq_register_current_thread to make
sure the library is already initialized if thread registration is done
before the library constructor is executed.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6c808ff846cd280522c3b8d50c0c64ca587261ee

Implement the REUSE specification for licensing and copyright

See "https://reuse.software/tutorial/", running the linter now returns
this :

$ reuse lint

* Bad licenses:
* Deprecated licenses:
* Licenses without file extension:
* Missing licenses:
* Unused licenses:
* Used licenses: BSD-2-Clause, FSFAP, GPL-2.0-or-later, GPL-3.0-or-later, LicenseRef-Autoconf-exception-macro, Linux-man-pages-copyleft, Linux-syscall-note, MIT
* Read errors: 0
* Files with copyright information: 52 / 52
* Files with license information: 52 / 52

Congratulations! Your project is compliant with version 3.0 of the REUSE Specification :-)

Change-Id: Ifa9f7bc1cbdce204dd8bad4b8978d90b0d7bb90a
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Relicense librseq to MIT

For simplicity, relicense all librseq (header files, library
implementation and tests) to MIT.

Header files were previously dual-licensed LGPL2.1/MIT, which is
somewhat redundant with MIT.

rseq.c was licensed under LGPL2.1. Relicence it to MIT to facilitate its
integration into statically built applications.

In order to facilitate eventual code sharing between tests and the
library implementation, relicence the tests from LGPL2.1 to MIT.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ie819e1527ddffce64e4fe76024e2728e8a723c25

rseq man page: remove _Nullable from rseq argument

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I7f19334065d588a0bf4be1c0bde2d6e70e6c2926

rseq(2) man page: Update following round of feedback and linting

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I025fe0834370700a479f6b6b38941f0ae55c38b6

Update rseq.2 man page based on comments

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I5a9848efa217a77ac2905f6ecdc93b80051194c3

Update rseq man page

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I3489828167982430093834ac9877c6d90e1c9733

Allowing querying whether libc support rseq

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I903ca816e3ebbaf9c4fff55431b50f58d0090a8c

Add RISC-V rseq support

Change-Id: If6a365a9d003ea9c0e54973aa5d931bd80d33218
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

fix: check if libc rseq support is registered

When checking for libc rseq support in the library constructor, don't
only depend on the symbols presence, check that the registration was
completed.

This targets a scenario where the libc has rseq support but it is not
wired for the current architecture in 'bits/rseq.h', we want to fallback
to our internal registration mechanism.

Change-Id: I662e537a6511fd4fd9e0650feb17e9c72ec39ddc
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Use ptrdiff_t for rseq_offset

glibc changes the type of __rseq_offset from int to ptrdiff_t before
the official 2.35 release. Adapt to this change.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Iaaef60957cf33f5536817fa92136239fe9e9ab74

Cleanup: x86: remove leftover comment

This comment is not relevant anymore with the userspace ABI introduced
by glibc 2.35.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I5b54942d84bb12ac0fe2c2246e871e86d51d3af9

x86-32: use %gs segment selector for accessing rseq thread area

Rather than use rseq_get_abi() and pass its result through a register to
the inline assembler, directly access the per-thread rseq area through a
memory reference combining the %gs segment selector, the constant offset
of the field in struct rseq, and the rseq_offset value (in a register).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ibd6fac4f2f80774ab74ae495cd0f688d789f36b8

x86-64: use %fs segment selector for accessing rseq thread area

Rather than use rseq_get_abi() and pass its result through a register to
the inline assembler, directly access the per-thread rseq area through a
memory reference combining the %fs segment selector, the constant offset
of the field in struct rseq, and the rseq_offset value (in a register).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ia0c93d32b9b36ee1ef6a737a8a5321498fc8c3bc

Remove configure check for rseq.h

Not needed anymore, using own copy.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Iec27260a112b55bc7540156b41810406795bd1c2

Remove unused ARRAY_SIZE macro

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ifa1002ff1458ced26f764660a7f4dc6eb8e259ce

Update header copyright dates

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I86ba7cc5fc899ae741374e23211af2f92115987e

Introduce rseq-abi.h

Introduce our own header for the rseq kernel ABI, so we do not depend on
Linux kernel headers.

Use this header's rseq_cs.arch.ptr field as accessor for the rseq_cs ptr
on 32-bit.

Remove the extra/ directory because the copy of the Linux kernel headers
is not needed anymore.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I0170f2bcaa7defd3a0d3f9a38115bc7d321ddde2

Fix: work-around Linux kernel rseq.h uapi endianness issue

Introduce RSEQ_CS_PTR32 to detect and work-around endianness issue for
the rseq_cs ptr32 vs padding fields on little endian 32-bit
architectures.

This does not affect the kernel implementation because the ptr32 field
is not used by the kernel.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Idee8daa16efba8a86655a1eab555f65e0fcea4ca

Fix: use long rather than int for offsets

Fixes build issue on x86-64 where the assembler expects a 64-bit
register as input. Use long type across all architectures.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I9537acf1269e5d3c0ed156e5f93a422ad5608a1b

ppc: remove incomplete undef list

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I8f17fc89f34fd671d72a9bb5bcc12ca0bed1ca81

Fix: use int rather than off_t for offsets

off_t can be a 64-bit type on 32-bit systems, leading to issues on
32-bit big endian systems (ppc32) where the inline assembler expects a
32-bit register.

There is no point in using off_t here.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I9936a1d8e9b64a51f3e1491f1a23204f1cd9c120

mips: use rseq_cs ptr 32-bit field

On big endian MIPS architectures, the padding field is before the 32-bit
ptr32 field. Access specifically to the 32-bit ptr32 field rather than
padding. It does not change the behavior on little endian.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I798e7b64b22b9c6484fc37c3a6b903b6ca1194f3

arm64: refer to rseq_cs.ptr field

Similarly to other architectures.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I61c17b4874b71ee361666734b6ac68ff591c33fe

s390/s390x: refer to rseq_cs pointer field

Does not make any different on little endian architecture, but makes it
clearer in case this code is ever ported to a big endian architecture.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6aa9b0fa4eb7ed55556dbff889104970bcc29944

arm: refer to 32-bit pointer field for rseq_cs ptr

This makes no technical difference on little-endian, but is clearer in
case this code is ever used as reference for a port to a big endian
32-bit architecture.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Iaf77c2a1b8e6d8f08be0d26ecb84686a417a07d6

Fix: ppc32: wrong rseq_cs 32-bit field pointer field on big endian

Update the ptr32 field, not padding.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I7066e7f87ac068d729aca6a11f6e9ba233b04279

Fix: ppc32: do not produce cmpdi invalid instruction

Introduce RSEQ_CMP_LONG_INT to fix this.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I02e216edde9ceed14777db2789e7297118503683

Fix: provide membarrier_private_expedited_rseq_available on all architectures

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I59afb18ba3687587b2712c3b076b170515d255bf

Test for membarrier private expedited rseq availability

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ib24ee6e84f2614b614650f391e7312121f26ab17

Fix: membarrier flags for kernels headers < 5.10

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I0c31e77a97d62140f162e058a975d47c849c5782

Test MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ

Based on Google-internal RSEQ work done by Paul Turner and Andrew
Hunter.

This patch adds a selftest for MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ.
The test quite often fails without the previous patch in this
patchset, but consistently passes with it.

[ Contribution from: Peter Oskolkov <posk@google.com>. ]

[ Modified by Mathieu Desnoyers to use rseq APIs for atomic load/store
  and memory barriers, so it compiles in C++ and with clang. ]

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ib10d52e6e933cda2df2f10c29775a79045e4425c

x86_64: Add rseq_offset_deref_addv()

This patch adds rseq_offset_deref_addv() function to
rseq-x86.h, to be used in a selftest in the next patch in the patchset.

  Once an architecture adds support for this function they should define
  "RSEQ_ARCH_HAS_OFFSET_DEREF_ADDV".

[ Contribution from Peter Oskolkov <posk@google.com>. ]

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ia32b27c909a7e41b4262dca3e86b6fb83d0b3bee

Fix: Warnings about #if checks of undefined tokens

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: If7e7f5a6a725e792fce54fee591ba880bc3766a6

Fix: work-around asm goto compiler bugs

gcc and clang each have their own compiler bugs with respect to asm
goto. Implement a work-around for compiler versions known to have those
bugs.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic745eb61163e90f668fb03707f3013e8981fb5fe

Remove arm/mips asm goto compiler work-around

The arm and mips work-around for asm goto size guess issues are not
properly documented, and lack reference to specific compiler versions,
upstream compiler bug tracker entry, and reproducer.

I can only find a loosely documented patch in my original LKML rseq post
refering to gcc < 7 on ARM, but it does not appear to be sufficient to
track the exact issue. Also, I am not sure MIPS really has the same
limitation.

Therefore, remove the work-around until we can properly document this.

Link: https://lore.kernel.org/lkml/20171121141900.18471-17-mathieu.desnoyers@efficios.com/
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ib4b151234e2255e26d26c149b8051ae024d39153

fix: dead initialization reported by scan-build

Change-Id: I7f3735f6f6332e1ba66be480ea4a7dfc5d9f84a9
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add debug output to basic_percpu_ops_test

Change-Id: Ie881e24b37b5f7debd5f98ba8d753452d31c1a36
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

configure: enable extended compiler warnings

Import the compiler warning flag detection system from liburcu and
enable extended compiler warnings.

Change-Id: I13bfbaed702c28e2816b13b78fb006462d6d5f92
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Build and run tests as C++ programs

Build and run all tests as C++ programs in addition to C.  This helps
get confidence that rseq, when used from a C++ program, behaves well.

Change-Id: I68d35d9cadcb859aaa6be4a5c620a6c21075c238
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

param test: Only provide rseq_gettid if BENCHMARK is not defined

Fixes:

param_test.c:20:21: warning: unused function 'rseq_gettid' [-Wunused-function]
static inline pid_t rseq_gettid(void)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I32a14d526263f29034366a9326b0237450088649

ppc rseq thread pointer: add extended asm

It appears that using "register void *__result asm ("r2")" is not sufficient
to guarantee that the compiler will indeed use this register. It's documented
as being just a hint that the compiler is free to optimize away. [1]

However, when used as input or output operands when calling Extended
asm, the use of the register is forced, which is our intent:

"The only supported use for this feature is to specify registers for input
and output operands when calling Extended asm (see Extended Asm). This may
be necessary if the constraints for a particular machine don't provide sufficient
control to select the desired register. To force an operand into a register,
create a local variable and specify the register name after the variable's
declaration. Then use the local variable for the asm operand and specify
any constraint letter that matches the register: [...]"

Link: https://gcc.gnu.org/onlinedocs/gcc-6.1.0/gcc/Local-Register-Variables.html
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I30ba3cc163a6e52cee5df0ba6806a4a5f9ec97d6

Introduce rseq-generic-thread-pointer.h

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ie1cf34d64626babd5d81bad5b1f00de88173f1b6

rseq thread pointer: extern C

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ibd91f0a8c831b5080a52a5e3bdab2358758068f7

Introduce RSEQ_GET_ABI_OVERRIDE

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I17c2a2f15b4e45cb95785ff03424a662ca74bbd2

Add missing rseq thread pointer headers to makefile

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I0aecd00cc91dde3b99083d12516ca8aa1a76c55f

Introduce rseq-thread-pointer.h

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I89469b2af7da6159ae3348747caab3a33d3b6118

Update librseq to co-exist with glibc rseq integration

Update librseq to co-exist with the rseq integration merged within the
glibc 2.35 release cycle.

librseq exposes its own "rseq_offset, rseq_size, rseq_flags" ABI.

Query for glibc rseq ABI (__rseq_offset, __rseq_size, __rseq_flags)
using dlsym() in a librseq library constructor. If those are found,
copy their values into rseq_offset, rseq_size, and rseq_flags.

Else, if those glibc symbols are not found, handle rseq registration
ourselves and use our own IE-model TLS to implement the rseq ABI
per-thread storage.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1220cb5c31e823e9cddf211ec56fee03b729b137

Remove rseq_deref_loadoffp placeholders

Remove those empty placeholders.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic40e350c27ccc3da4168a4107cd206da0a08b68e

Add git-review config

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic3794a3a323d0865b0ac714077c1e2aecfb0a444

Modernize autotools setup

This is part of an effort to standardise our autotools setup across
project to simplify maintenance.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Skip the tests if the rseq syscall is unavailable

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Convert test suite to the TAP test protocol

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Remove __rseq_handled declaration

Adapt to glibc's ABI.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add parametrized tests

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add basic test

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Cleanup: powerpc: Use "long" and "int" to clarify load/store macros

Using "WORD" to mean either a 32-bit or 64-bit type depending on
the architecture is misleading. The term "WORD" really means a
32-bit type in both 32-bit and 64-bit powerpc assembler. The intent
here is to wrap load/store to intptr_t into common macros for both
32-bit and 64-bit.

Rename the macros with a RSEQ_ prefix, and use the terms "INT"
for always 32-bit type, and "LONG" for architecture bitness-sized
type.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: powerpc: Add missing instruction selection "u" and "x" for load/store

Building the rseq basic test  with
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12)
Target: powerpc-linux-gnu

leads to these errors:

/tmp/ccieEWxU.s: Assembler messages:
/tmp/ccieEWxU.s:118: Error: syntax error; found `,', expected `('
/tmp/ccieEWxU.s:118: Error: junk at end of line: `,8'
/tmp/ccieEWxU.s:121: Error: syntax error; found `,', expected `('
/tmp/ccieEWxU.s:121: Error: junk at end of line: `,8'
/tmp/ccieEWxU.s:626: Error: syntax error; found `,', expected `('
/tmp/ccieEWxU.s:626: Error: junk at end of line: `,8'
/tmp/ccieEWxU.s:629: Error: syntax error; found `,', expected `('
/tmp/ccieEWxU.s:629: Error: junk at end of line: `,8'
/tmp/ccieEWxU.s:735: Error: syntax error; found `,', expected `('
/tmp/ccieEWxU.s:735: Error: junk at end of line: `,8'
/tmp/ccieEWxU.s:738: Error: syntax error; found `,', expected `('
/tmp/ccieEWxU.s:738: Error: junk at end of line: `,8'
/tmp/ccieEWxU.s:741: Error: syntax error; found `,', expected `('
/tmp/ccieEWxU.s:741: Error: junk at end of line: `,8'
Makefile:581: recipe for target 'basic_percpu_ops_test.o' failed

Based on discussion with Linux powerpc maintainers and review of
the use of the "m" operand in powerpc kernel code, add the missing
%Un%Xn (where n is operand number) to the lwz, stw, ld, and std
instructions when used with "m" operands.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Wire up basic percpu ops test

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add basic percpu ops test

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Adapt rseq registration to glibc's userspace ABI for __rseq_abi

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Update man page based on Michael Kerrisk's comments

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Update rseq.2 man page

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Wire up man pages build

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Initial import of rseq.2 man page

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Remove do_on_cpu dependencies

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Remove do_on_cpu (prototype)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Cleanup: cplusplus ifdef coding style update

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Allow using librseq APIs from C++ code

If you call librseq APIs like rseq_available and
rseq_register_current_thread from C++ code, linking fails:

    rseq.cpp:365: undefined reference to `rseq_available()'
    rseq.cpp:46: undefined reference to `rseq_register_current_thread()'

Tell the C++ compiler to treat librseq's exported APIs as having C
mangling, fixing these linker errors.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

memcpy: copy 8 bytes loop

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Bump do_on_cpu system call number to 434

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Implement percpu_fence()

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: add missing do-on-cpu-insn.h to make dist

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Move to do_on_cpu system call

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add ARM big-endian support to rseq headers

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: Thumb mode build failure on arm32

Using ".arm .inst" for the arm signature introduces build issues for
programs compiled in Thumb mode because the assembler stays in the
arm mode for the rest of the inline assembly. Revert to using a ".word"
to express the signature as data instead.

The choice of signature is a valid trap instruction on arm32 little
endian, where both code and data are little endian.

ARMv6+ big endian (BE8) generates mixed endianness code vs data:
little-endian code and big-endian data. The data value of the signature
needs to have its byte order reversed to generate the trap instruction.

Prior to ARMv6, -mbig-endian generates big-endian code and data
(which match), so the endianness of the data representation of the
signature should not be reversed. However, the choice between BE32
and BE8 is done by the linker, so we cannot know whether code and
data endianness will be mixed before the linker is invoked. So rather
than try to play tricks with the linker, the rseq signature is simply
data (not a trap instruction) prior to ARMv6 on big endian. This is
why the signature is expressed as data (.word) rather than as
instruction (.inst) in assembler.

Because a ".word" is used to emit the signature, it will be interpreted
as a literal pool by a disassembler, not as an actual instruction.
Considering that the signature is not meant to be executed except in
scenarios where the program execution is completely bogus, this should
not be an issue.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Acked-by: Will Deacon <will.deacon@arm.com>

rseq/selftests: mips: use break instruction for RSEQ_SIG

Use break as guard instruction for the restartable sequence abort
handler.

Previously, the chosen signature was simply data, based on the
assumption that it could always sit in a literal pool. However,
some compilation environments favor disabling literal pool. Therefore,
ensure the signature is a valid uncommon trap instruction.

commit 16b96b6ed87f "rseq/selftests: mips: use break instruction for
RSEQ_SIG" in the Linux kernel.

Suggested-by: Paul Burton <paul.burton@mips.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

rseq/selftests: powerpc code signature: generate valid instructions

Use "twui" as the guard instruction for the restartable sequence abort
handler.

commit 496fd0fc9f5c "rseq/selftests: powerpc code signature: generate valid
instructions" in the Linux kernel.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

rseq/selftests: aarch64 code signature: handle big-endian environment

Handle compiling with -mbig-endian on aarch64, which generates binaries
with mixed code vs data endianness (little endian code, big endian
data).

Else mismatch between code endianness for the generated signatures and
data endianness for the RSEQ_SIG parameter passed to the rseq
registration will trigger application segmentation faults when the
kernel try to abort rseq critical sections.

commit 7cd4ce2e466f "rseq/selftests: aarch64 code signature: handle
big-endian environment" in the Linux kernel.

Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

rseq/selftests: arm: use udf instruction for RSEQ_SIG

Use udf as the guard instruction for the restartable sequence abort
handler.

Previously, the chosen signature was not a valid instruction, based
on the assumption that it could always sit in a literal pool. However,
there are compilation environments in which literal pools are not
available, for instance execute-only code. Therefore, we need to
choose a signature value that is also a valid instruction.

Handle compiling with -mbig-endian on ARMv6+, which generates binaries
with mixed code vs data endianness (little endian code, big endian
data).

Else mismatch between code endianness for the generated signatures and
data endianness for the RSEQ_SIG parameter passed to the rseq
registration will trigger application segmentation faults when the
kernel try to abort rseq critical sections.

Prior to ARMv6, -mbig-endian generates big-endian code and data, so
endianness should not be reversed in that case.

commit 2b845d4b4acd "rseq/selftests: arm: use udf instruction for RSEQ_SIG"
in the Linux kernel.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>