2026-10-31

Large System Extensions (LSE) and Atomics on AArch64

Introduction

In this blog post we’ll go into detail about how atomics are implemented on AArch64, and how this ties back into the C++ and OpenJDK runtimes. To start with, Arm has an Atomics ABI for 64-bit architecture for C/C++ that I highly recommend skimming through.

In my previous post on Exclusive Accesses and Cacheability on AArch64, I covered how exclusive operations (load exclusive and store exclusive) are implemented and how they work using exclusive monitor(s). Exclusive operations are supported on all 64-bit architectures of Arm CPUs, going all the way back to armv8-a.

A downside of exclusive operations is that operations may fail for many reasons, and may therefore have to be retried any number of times (potentially unbounded). This has the potential to become a noticeable problem on systems with many cores, as cores could interfere with the exclusive operations of other cores more often, making it hard to maintain fairness.

With the introduction of the Large System Extensions (LSE) in armv8.1-a, exclusive operations are complemented with atomic instructions that offer read-modify-write operations in a single instruction. The atomic instructions don’t rely on the use of exclusive monitor(s), and thus can’t fail spuriously in the same way, improving the performance and scalability on systems with many cores.

Below is a brief example of atomically incrementing a value (with relaxed memory ordering) using LSE and exclusive operations. This could translate to the C++ function fetch_add(1, std::memory_order_relaxed).

mov     x0, #1

# Atomic instruction (LSE)
ldadd   x0, x0, [x1]

# Exclusive operation (non-LSE)
1:
ldxr    x2, [x1]
add     x2, x2, x0
stxr    w3, x2, [x1]
cbnz    w3, 1b

We’ll mainly go into details about how this works on Linux with GCC, but macOS and Windows have similar approaches for detecting capabilities and support for run-time helpers via Clang and MSVC. In my limited testing, it seems like all Apple M-series Arm CPUs support armv8.1-a. If you’re targeting macOS/arm64, then you can most likely rely on LSE always being present.

C++ Runtime

Outline Atomics

For compilation targets and settings that do not make it clear which Arm architecture is targeted, we don’t know if the CPU that the program ends up running on supports LSE or not. We don’t always want to fall back to using exclusive operations, due to their disadvantageous behavior of potentially unbounded retries, which might starve certain threads. If the CPU supports LSE, the atomic instructions are preferable.

In order to use the best variant, GCC emits a run-time helper that detects the capabilities of the CPU and chooses either the LSE or exclusive operation variant depending on what the CPU supports. This approach is referred to as “outline atomics”.

The run-time helper checks a global variable that is cached during startup by the init_have_lse_atomics stub that GCC emits, so that the CPU’s features don’t have to be queried on every atomic operation.

Both GCC and Clang support the compiler options -moutline-atomics/-mno-outline-atomics to toggle outline atomics on/off. GCC states that the option is on by default, and is only applicable when compiling for the base armv8-a architecture. If a later revision is specified, e.g., -march=armv8.1-a/-march=armv8-a+lse, or a CPU is selected that supports LSE via -mcpu=..., then outline atomics are disabled and LSE instructions will be used directly.

Example and Disassembly

Let’s go over a minimal example where we have an atomic counter that we want to increment and read its current value.

#include <atomic>
#include <cstdint>

int main() {
    std::atomic<uint32_t> value{0};
    value.fetch_add(1); // defaults to std::memory_order_seq_cst, which
                        // translates to acquire + release semantics on
                        // Arm CPUs
    return 0;
}

Compiling this code with optimizations (-O1, otherwise we’d get a large jump-table that matches the C++ implementation) using g++ 15.2.0, and disassembling with objdump, we can see that the main function branches to a run-time helper __aarch64_ldadd4_acq_rel. The helper checks the global value that was saved in init_have_lse_atomics, and then either executes the LSE or non-LSE atomic implementation.

0000000000400740 <main>:
  // ...
  //                            x1 = &value (elided)
  400774:       52800020        mov     w0, #0x1                        // #1
  400778:       94000012        bl      4007c0 <__aarch64_ldadd4_acq_rel>
  // ...

00000000004007c0 <__aarch64_ldadd4_acq_rel>:
  // Load and check a global constant to see if LSE is enabled
  4007c0:       b0000510        adrp    x16, 4a1000 <_IO_2_1_stderr_+0x78>
  4007c4:       39642210        ldrb    w16, [x16, #2312]
  4007c8:       34000070        cbz     w16, 4007d4 <__aarch64_ldadd4_acq_rel+0x14>

  // LSE path
  4007cc:       b8e00020        ldaddal w0, w0, [x1]
  4007d0:       d65f03c0        ret

  // Non-LSE path (exclusive operations)
  4007d4:       2a0003f0        mov     w16, w0
  4007d8:       885ffc20        ldaxr   w0, [x1]
  4007dc:       0b100011        add     w17, w0, w16
  4007e0:       880ffc31        stlxr   w15, w17, [x1]
  4007e4:       35ffffaf        cbnz    w15, 4007d8 <__aarch64_ldadd4_acq_rel+0x18>
  4007e8:       d65f03c0        ret

Even though this example highlights the fetch_add operation, other atomic operations follow the same pattern for outline atomics. GCC defines the outline atomic routines in libgcc/config/aarch64/lse.S.

Strong vs. Weak Compare Exchange

A really interesting behavior that you might have noticed when using the C++ standard library’s atomic functions is that there are two implementations of compare exchange: compare_exchange_weak and compare_exchange_strong. The documentation states that:

• compare_exchange_weak is allowed to fail spuriously, that is, acts as if *this != expected even if they are equal. For example, on some platforms, compare-and-exchange is implemented using load-link/store-conditional instructions. The store-conditional instruction may fail even if the loaded value compares equal, for example because its reservation was lost. compare_exchange_weak is allowed to report this as a spurious failure, whereas compare_exchange_strong must retry or otherwise ensure that it does not return false solely for this reason.

An example of a platform that matches the behavior the documentation mentions is armv8-a, on which you need to implement compare exchange with exclusive operations, which might spuriously fail. On an LSE-enabled system (armv8.1-a), the weak and strong variants can share the same definition, since the cas LSE instruction satisfies the requirements for both variants.

Following the examples in the Atomics ABI for 64-bit architecture, we can have a look at the differences between a typical implementation of a weak and strong compare exchange. For all examples below, we consider a “relaxed” compare exchange without memory ordering requirements. Register x1 contains the address of the object, w2 contains the value, and w0 contains the expected value. The result is returned in w0.

LSE Instruction

Both variants share the same implementation, which is pretty straightforward.

compare_exchange_weak and compare_exchange_strong:

cas     w0, w2, [x1]

Exclusive Operation

In the exclusive operation variants (non-LSE), the main difference is what happens when the exclusive store operation (stxr) fails. In the weak variant, there is no check to see if it succeeded or not, so it will just continue. In the strong variant, there is a cbnz check that branches back to the beginning of the operation to retry.

The strong variant runs into exactly the problem from the introduction. Under heavy contention, stxr can fail and the thread can retry any number of times, potentially starving it. Since the cas implementation shown above doesn’t use exclusive operations, it cannot fail spuriously like the example below.

compare_exchange_weak:

mov     w4, w0
ldxr    w0, [x1]
cmp     w0, w4
b.ne    fail
stxr    w3, w2, [x1]
fail:

compare_exchange_strong:

mov     w4, w0
loop:
ldxr    w0, [x1]
cmp     w0, w4
b.ne    fail
stxr    w3, w2, [x1]
cbnz    w3, loop
fail:

Java Runtime (OpenJDK)

OpenJDK, which houses the HotSpot virtual machine, also needs to handle the possibility that a 64-bit Arm CPU does not have LSE support available. Outline atomics could be used to detect this, and would probably be an acceptable approach. Since HotSpot has the capability to Just-In-Time (JIT) compile code, it does not necessarily need a run-time check for every atomic operation. Instead, the runtime can detect the capabilities of the CPU (and any configuration by the user to choose an approach) during startup, much like the C++ runtime does, and then generate a stub that only does either LSE or exclusive operations.

Since atomic operations might need to be done before the JIT compiler is set up and ready to generate a stub, HotSpot has default assembly stubs that are always available. These default stubs must use exclusive operations, since they might run on a non-LSE Arm CPU.

The functionality below is described from the jdk-27-ga tag.

To have full control, HotSpot does not rely on the C++ standard library’s std::atomic. Instead, it uses its own layer of abstraction on top of compiler primitives/intrinsics, hand-rolled assembly stubs, or generated stubs. HotSpot exposes three kinds of atomic operations at its lowest “platform”-level:

* Support for size 4 is required, and also the pointer size, which is 8 on AArch64, so both 4 and 8 must be implemented.

Broadly speaking, there are two cases where the JDK cares about LSE and which approach is used for atomics:

1. In the runtime (non-JIT code)

   By default, the hand-rolled assembly stubs in src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.S use exclusive operations, unless __ARM_FEATURE_ATOMICS is defined, in which case hand-rolled LSE variants are used instead. The LSE branch is not portable, since it doesn’t work on armv8-a, and thus not used for production builds, only for temporary developer builds. See JDK-8282322 for more information.

   If __ARM_FEATURE_ATOMICS is undefined, then the state of the UseLSE flag is checked during startup, which is populated with whether the system has LSE or not (see #Detecting Capabilities via Auxiliary Vector), or whether the user disabled it with -XX:-UseLSE.

   If UseLSE is true, then the hand-rolled assembly stubs (which use exclusive operations) are replaced with generated LSE stubs. See src/hotspot/cpu/aarch64/stubGenerator_aarch64.cpp#L11166 for the specific implementation details.

2. In Just-in-Time compiled code

   The JIT uses a MacroAssembler to emit instructions for specific operations. Below are links to specific places in the code where the atomic operations are defined for the MacroAssembler. Two of these are done with macros that generate multiple versions of the same operation with different memory orderings (acquire, release) and sizes (1, 4, and 8 bytes).

   • ATOMIC_OP macro
   • ATOMIC_XCHG macro
   • MacroAssembler::cmpxchg

   The MacroAssembler’s implementation of “Fetch Add”, “Exchange”, and “Compare Exchange” are then used when JIT compiling code in HotSpot’s C1 and C2 JIT compilers.

While reading up on how this works, I noticed that there was some room for improvement in how 1-byte exchange is implemented on AArch64, so I submitted a patch, see JDK-8395815. Following up on that issue, I will be working on adding relaxed semantics to the exchange operations as well, which currently only support conservative memory ordering, see JDK-8396065.

Detecting Capabilities via Auxiliary Vector

Briefly mentioned in #Outline Atomics, GCC emits a function called init_have_lse_atomics on Linux for compilation targets and settings that do not make it clear which Arm architecture is targeted. That function calls into getauxval(AT_HWCAP) to see if the kernel communicates whether the CPU has “atomics” (a shorthand for LSE) available or not, and stores the result of that call in a global variable. That global variable can then be checked in any operation that needs to know whether LSE is available or not.

libgcc/config/aarch64/lse-init.c (simplified for readability):

# define AT_HWCAP      16
# define HWCAP_ATOMICS (1 << 8)

static void init_have_lse_atomics (void) {
  unsigned long hwcap = __getauxval (AT_HWCAP);
  __aarch64_have_lse_atomics = (hwcap & HWCAP_ATOMICS) != 0;
}

OpenJDK also checks the value of getauxval(AT_HWCAP) to determine whether LSE is available or not, and stores this in a bitmask that is later queried to populate the UseLSE flag.

hotspot/os_cpu/linux_aarch64/vm_version_linux_aarch64.cpp (simplified for readability):

void VM_Version::get_os_cpu_info() {
  uint64_t auxv = getauxval(AT_HWCAP);
  _features =
      // ...
      check_feature(auxv, BIT_MASK(CPU_LSE), HWCAP_ATOMICS);
      // ...
  // ...
}

CPU Capability Source

Under the hood, Linux reads the capabilities of the CPU via the ID_AA64ISAR0_EL1 system register, before eventually passing it on through several layers and finally via the auxiliary vector to a process when it is spawned.

In the ID_AA64ISAR0_EL1 system register, bits [23:20] indicate support for Atomic instructions in AArch64 state. FEAT_LSE implements the functionality identified by the value 0b0010, and FEAT_LSE128 implements the functionality identified by the value 0b0011. Starting from armv8.1-a, the value 0b0000 is not permitted, so the CPU is required to support LSE. See Arm’s documentation for ID_AA64ISAR0_EL1.

Conclusion

The whole ordeal with LSE instructions vs. exclusive operations mainly concerns portability. Since LSE is required from the armv8.1-a revision and forward, but optional in armv8-a, we can’t assume that all 64-bit Arm CPUs a program executes on will support LSE, so a portable solution is necessary.

Outline atomics is one approach to solve this, by moving the decision of which instructions to use to run-time instead of compile-time. An alternative could be to always fall back to exclusive operations, but that could run suboptimally on systems with many cores. In a runtime with a JIT compiler, like OpenJDK, you can detect what features the CPU supports during startup and only use the most optimal variant moving forward.

A really interesting case where exclusive operations can make life hard is when using the rr debugger, which cannot function with exclusive operations, and requires LSE to work. On the other hand, all Apple M-series CPUs seem to support LSE instructions, so no need for portability when targeting Mac/arm64.

I’ve been spending a lot of time digging into the Arm architecture(s) and details like this, which has been really fun. Hope you learned something new from reading this!