CLOCK_MONOTONIC is specified as returning time "since an unspecified point in the past". On RISC OS with UnixLib this returns the time since the last hard reset, but with SharedCLibrary this returns the time since the program started - combined with the coarse resolution used internally, this almost always results in a seed of 0.
CLOCK_REALTIME meanwhile is specified as returning time since the epoch, so it should behave consistently across all platforms.
This works fine when the referenced symbol has the same prefix
as PRIVATE_PREFIX in the same file; otherwise we could also
create a macro like X() that only prepends the extern symbol
prefix but no symbol namespace prefix.
Fixes a bug where the Av1FilterLUT instance used in checkasm was not
aligned properly.
In theory, the first ALIGN macro should imply the latter alignments as well,
but I decided to mark all fields as explicitly aligned for clarity; and
because that's the precedent set in other headers.
Allows us to drop the ALIGN macro on the other usage of this struct.
Currently compilation steps use two different types of methods, manual
build directory creation and using meson setup build to directly create
the build directory. This potential makes the new user who wants to
build docs or cross compile confused about which step of compilation the
user is in. This patch aims to make these steps clear.
Based on the following comment on IRC:
"<aconz2> the `elapsed` variable in main() is read uninitialized in
synchronize and makes the first frametime with --frametime incorrect
I think. Should be initialized to 0"
Confirmed that after initializing to zero, the first line in the file
generated by --frametime is reasonable.
Rename loongarch/looprestoration_tmpl.c to loongarch/looprestoration_inner.c.
Compiling both src/looprestoration_tmpl.c and loongarch/looprestoration_tmpl.c
produces looprestoration_tmpl.c.o, causing a conflict during linking.
We have new option in clang (https://github.com/llvm/llvm-project/pull/124834)
to mark globals to be allocated in non-large sections. We can mark all globals
that are referenced from hardcoded assembly (which implicitly references globals
assuming they are in non-large sections) with this attribute to avoid running
into problems when dav1d is built with -mcmodel=medium with clang.
This patch adds a vectorised variant of the mv_projection calculation
and a faster initialisation of motion vectors for load_tmvs_neon.
Checkasm uplifts after this patch on some Neoverse and Cortex CPU cores
compared to the C reference compiled with GCC-13 and Clang-19:
GCC Clang
AWS Graviton 4: 1.62x 1.59x
Cortex-X4: 1.45x 1.46x
Cortex-X3: 1.68x 1.69x
Cortex-X1: 1.55x 1.52x
Cortex-A720: 1.54x 1.57x
Cortex-A715: 1.47x 1.55x
Cortex-A78: 1.21x 1.18x
Cortex-A76: 1.38x 1.37x
Cortex-A72: 1.08x 1.11x
Cortex-A520: 0.97x 1.18x
Cortex-A510: 0.99x 1.14x
Cortex-A55: 1.16x 1.23x
This patch increases the .text by ~660 bytes, but smaller than the
reference implementation by about 0.5 KiB.
For the bilin cases, this seems to make things marginally faster
(measured on x86_64; 7-25% faster with compiler autovectorization).
For 8tap, it doesn't make much of a difference at all.
Before: GCC Clang
mc_scaled_8tap_regular_w128_8bpc_c: 115155.5 98549.3
mc_scaled_8tap_regular_w128_8bpc_ssse3: 17936.0 18411.1
mc_scaled_bilinear_w128_8bpc_c: 40290.0 51812.9
mc_scaled_bilinear_w128_8bpc_ssse3: 18243.9 18177.0
After:
mc_scaled_8tap_regular_w128_8bpc_c: 116304.3 99453.2
mc_scaled_8tap_regular_w128_8bpc_ssse3: 18387.0 18077.3
mc_scaled_bilinear_w128_8bpc_c: 37381.4 41145.0
mc_scaled_bilinear_w128_8bpc_ssse3: 18423.8 18031.6
(Benchmarked with the seed 0; the total runtime for the scaled
benchmarks are significantly affected by the random seed.)
This reduces the stack usage of these functions from around 65 KB
each, to less than 1 KB for bilin, and around 2 KB for 8tap.
With this in place, the required stack space for dav1d should
be mostly identical across configurations; on x86_64 (both with
and without assembly), it can run with 62 KB of stack, and
on arm and aarch64, it can run with 58 KB of stack.
Switch to the same cache-friendly algorithm as was done for arm64
in 2e73051c57 and for the reference
C code in 8291a66e50.
Contrary to the arm64 implementation, this uses a main loop in C
(very similar to the one in the main C implementation in
8291a66e50) rather than assembly;
this gives a bit more overhead on the call to each function, but
it shouldn't affect the big picture much.
Performane wise, this doesn't make much of a difference - it makes
things a little bit faster on some cores, and a little bit slower
on others:
Before: Cortex A7 A8 A53 A72 A73
wiener_7tap_8bpc_neon: 269384.4 147730.7 140028.5 92662.5 92929.0
wiener_7tap_10bpc_neon: 352690.2 159970.2 169427.8 116614.9 119371.1
After:
wiener_7tap_8bpc_neon: 238328.0 157274.1 134588.6 92200.3 97619.6
wiener_7tap_10bpc_neon: 336369.3 162182.0 161954.4 125521.2 130634.0
This is mostly in line with the results on arm64 in
2e73051c57. On arm64, there was a
bit larger speedup for the 7tap case, mostly attributed to
unrolling the vertical filter (and the new filter_hv function) to
operate on 16 pixels at a time. On arm32, there's not enough
registers to do that, so we can't get such gains from unrolling.
(Reducing the unrolling on the arm64 version to match the case
on arm32 also shows similar performance numbers as on arm32 here.)
In the arm64 version, we also added separate 5tap versions of all
functions; not doing that for arm32 at this point.
This increases the binary size by 2 KB.
This doesn't have any immediate effect on how much stack space
dav1d requires in total, since the largest stack users on arm
currently are the 8tap_scaled functions.
This uses a separate function for combined horizontal and vertical
filtering, without needing to write the intermediate results
back to memory inbetween.
This mostly serves as an example for how to adjust the logic for
that case; unless we actually merge the horizontal and vertical
filtering within the _hv function, we still need space for a
7th row on the stack within that function (which means we use just
as much stack as before), but we also need one extra memcpy to
write it into the right destination.
In a build where the compiler is allowed to vectorize and inline
the wiener functions into each other, this change actually reduces
the final binary size by 4 KB, if the C version of the wiener filter
is retained.
This change makes the vectorized C code as fast as it was before
with Clang 18; on Xcode Clang 16, it's 2x slower than it was before.
Unfortunately, with GCC, this change makes the code a bit slower
again.
This increases the binary size by 9 KB, on aarch64 with Xcode Clang 16,
if the C version of the filter is retained (which it isn't
by default).
This makes the vectorized C code roughly as fast as it was before
the rewrite on GCC; on Clang it also becomes 1.3x-2.0x faster,
while still being slower than it was initially.
This reduces the stack usage of these functions (the C version)
significantly.
These C versions aren't used on architectures that already have
wiener filters implemented in assembly, but they matter both if
running e.g. with assembly disabled (e.g. for sanitizer builds),
and matter as example for how to do a cache efficient SIMD
implementation.
This roughly matches how these functions are implemented in the
aarch64 assembly (although that assembly function uses a mainloop
function written in assembly, and custom calling conventions
between the functions).
With this in place, dav1d can run with around 76 KB of stack
with assembly disabled.
This increases the binary size by around 14 KB (in the case of
aarch64 with Xcode Clang 16), unless built with (the default)
-Dtrim_dsp=true. (By default, the C version of the wiener filter
gets skipped entirely.)
On 32 bit arm, the assembly wiener function implementation still
uses large buffers on the stack though, but due to other functions
using less stack there, dav1d can still run with 72 KB of stack
there.
Unfortunately, this change also makes the functions slower, depending
on how well the compiler was able to optimize the previous version.
On GCC (which didn't manage to vectorize the functions so well before),
it becomes 1.6x-2.0x slower, while it gets 2.5x-5x slower on Clang
(where it was very well vectorized before).
Most of this performance can be gained back with later changes on
top, though.
It previously used 'pixel' which is typedefed to uint8_t in files
that aren't bitdepth-templated, but those are indices and not
pixels so that was just confusing and misleading.
f->task_thread.error can be set during flushing, not resetting this can
lead to c->task_thread.first being increased after having already submitted
a frame post flushing. That's fine if it happens on the very first frame,
but if that's the case on any subsequent frame it will incur a wrong frame
ordering.
Now that a non-first frame will be considered as such, its tasks won't be
able to execute (since they depend on a truly previous frame considered as
being after) and c->task_thread.cur will be increased past that frame, with
no way of it being reset, eventually leading to a hang.
Cameron Cawley