Make use of _SC_NPROCESSORS_ONLN if it exists and fallback to
_SC_NPROCESSORS_CONF for really old operating systems. This adds
support for retrieving the number of CPUs on a few OS's such as
NetBSD, DragonFly and a few others.
PLT/GOT indirections are required in some cases. Most commonly when
calling functions from other shared libraries, but also in some
scenarios when calling functions with default symbol visibility
even within the same component on certain elf64 platforms.
On elf64 we can simply use PLT relocations for all calls to external
functions. Since the linker is able to eliminate unnecessary PLT
indirections with the final output binary being identical to non-PLT
relocations there isn't really any downside to doing so. This mimics
what regular compilers normally do for calls to external functions.
On elf32 with PIC we can use a function pointer from the GOT when
calling external functions, similar to what regular compilers do when
using -fno-plt. Since this both introduces overhead and clobbers one
register, which could potentially have been used for custom calling
conventions when calling other asm functions within the same library,
it's only performed for functions declared using 'cextern_naked'.
Prior to this change dealing with the scenario where the number of
XMM registers spilled depends on if a branch is taken or not was
complicated to handle well. There was essentially three options:
1) Always spill the largest number of XMM register. Results in
unnecessary spills.
2) Do the spilling after the branch. Results in code duplication
for the shared subset of spills.
3) Do the spilling manually. Optimal, but overly complex and vexing.
This adds an additional optional argument to the WIN64_SPILL_XMM
and WIN64_PUSH_XMM macros to make it possible to allocate space
for a certain number of registers but initially only push a subset
of those, with the option of pushing additional register later.
This makes the code much simpler (especially for adding support
for other instruction set extensions), avoids needing inline
assembly for this feature, and generally is more of the canonical
way to do this.
The CPU feature detection was added in
9c3c716882, using HWCAP_CPUID.
The argument for using that, was that HWCAP_CPUID was added much
earlier in the kernel (in Linux v4.11), while the HWCAP flags for
individual features always come later. This allows detecting support
for new CPU extensions before the kernel exposes information about
them via hwcap flags.
However in practice, there's probably quite little advantage in this.
E.g. HWCAP_SVE was added in Linux v4.15, and HWCAP2_SVE2 was added in
v5.10 - later than HWCAP_CPUID, but there's probably very little
practical cases where one would run a kernel older than that on a CPU
that supports those instructions.
Additionally, we provide our own definitions of the flag values to
check (as they are fixed constants anyway), with names not conflicting
with the ones from system headers. This reduces the number of ifdefs
needed, and allows detecting those features even if building with
userland headers that are lacking the definitions of those flags.
Also, slightly older versions of QEMU, e.g. 6.2 in Ubuntu 22.04,
do expose support for these features via HWCAP flags, but the
emulated cpuid registers are missing the bits for exposing e.g. SVE2
(This issue is fixed in later versions of QEMU though.)
Also drop the ifdef check for whether AT_HWCAP is defined; it was
added to glibc in 1997. AT_HWCAP2 was added in 2013, in glibc 2.18,
which also precedes when aarch64 was commonly used anyway, so
don't guard the use of that with an ifdef.
Automatically flag x86-64 asm object files as SHSTK-compatible.
Shadow Stack (SHSTK) is a part of Control-flow Enforcement Technology
(CET) which is a feature aimed at defending against ROP attacks by
verifying that 'call' and 'ret' instructions are correctly matched.
For well-written code this works transparently without any code changes,
as return addresses popped from the shadow stack should match return
addresses popped from the normal stack for performance reasons anyway.
Broadcasting a memory operand is a binary flag, you either broadcast
or you don't, and there's only a single possible element size for
any given instruction.
The instruction syntax however requires the broadcast semanticts
to be explicitly defined, which is an issue when using macros to
template code for multiple register widths.
Add some helper defines to alleviate the issue.
Imporve the performance of NEON functions of aarch64/deblock-a.S
by using the SVE/SVE2 instruction set. Below, the specific functions
are listed together with the improved performance results.
Command executed: ./checkasm8 --bench=deblock
Testbed: Alibaba g8y instance based on Yitian 710 CPU
Results:
deblock_chroma[1]_c: 735
deblock_chroma[1]_neon: 427
deblock_chroma[1]_sve: 353
Command executed: ./checkasm8 --bench=deblock
Testbed: AWS Graviton3
Results:
deblock_chroma[1]_c: 719
deblock_chroma[1]_neon: 442
deblock_chroma[1]_sve: 345
The sve-default-vector-length property sets the maximum vector
length in bytes; the default is 64, i.e. handling up to 512
bit vectors. In order to be able to test 1024 and 2048 bit vectors,
this has to be raised separately from setting the sve<n>=on
property.
The assembly currently uses a mixture of different styles. Don't
make all of it entirely consistent now, but try to make functions
more consistent within themselves at least.
In particular, get rid of the convention to have braces hanging
outside of the alignment line.
Some functions have the whole content indented off by one char
compared to other functions; adjust those (but retain the functions
that are self-consistent and match either of the common styles).
The assembly currently uses a mixture of different styles. Don't
make all of it entirely consistent now, but try to make functions
more consistent within themselves at least.
In particular, get rid of the convention to have braces hanging
outside of the alignment line.
We could also use HWCAP_SVE and HWCAP2_SVE2 for detecting this,
but these might not be available in all userland headers, while
HWCAP_CPUID is available much earlier.
The register ID_AA64ZFR0_EL1, which indicates if SVE2 is available,
can only be accessed if SVE is available. If not building all the
C code with SVE enabled (which could make it impossible to run on
on HW without SVE), binutils refuses to assemble an instruction
reading ID_AA64ZFR0_EL1 - but if referring to it with the technical
name S3_0_C0_C4_4, it can be assembled even without any extra
extensions enabled.
We don't expect the user to build the whole x264 codebase with
SVE/SVE2 enabled, as we only enable this feature for the assembly
files that use it, in order to have binaries that are portable
and enable the SVE codepaths at runtime if supported.
Brad Smith