Is there a downside to using APX?

Why wouldn't this just be on by default?

Maybe it's a "beta", and there may be some bugs etc. I'd expect this to be on by default in future.

Applications can use ISA extensions the processor offers without any OS support, right?

WirkoApplications can use ISA extensions the processor offers without any OS support, right?

No, the OS needs to know which registers to save (and how) when switching CPU contexts between threads.

dgianstefaniIs there a downside to using APX?

Why wouldn't this just be on by default?

Maybe it's a "beta", and there may be some bugs etc. I'd expect this to be on by default in future.

It's not on by default (yet) because it's a significant change to the architecture. Since software must be recompiled to benefit from APX there can be bugs in compilers themselves. Prioritizing stability by default is the correct choice.
Not every feature of APX is yielding performance benefits in the first generation (an example). Most likely this will be corrected in future CPU generations, just like AVX-512 performance issues were.
It's good that the toggle is available in an accessible manner since historically turning off CPU features on Windows was complicated (bit-banging DWORDs in the registry and having to edit BCD come to mind).

I would've expected a Windows 12 announcement for this feature. After all, Windows 11 pretty much came about because Intel needed a new scheduler in Windows for their Alder Lake processors. But perhaps Intel isn't as important to Microsoft as it used to be.

azraelI would've expected a Windows 12 announcement for this feature.

May be when unified core comes.

ncrsSince software must be recompiled to benefit from APX there can be bugs in compilers themselves. Prioritizing stability by default is the correct choice.

From what I have seen about APX, you do not actually need to recompile to get benefits from it. That was one of the targets of APX since it, to my understanding, mostly expands how many general purpose registers there are. Recompiling may help fully leverage APX, but it should not be needed to at least get noticeable improvement.

melkor_unlimitedMay be when unified core comes.

Those have been shelved, and it's very unlikely they will ever become a thing.

XeonsNOpteronsFrom what I have seen about APX, you do not actually need to recompile to get benefits from it. That was one of the targets of APX since it, to my understanding, mostly expands how many general purpose registers there are. Recompiling may help fully leverage APX, but it should not be needed to at least get noticeable improvement.

That still requires recompilation, existing applications are unaware of them and cannot benefit. Everything on an instruction level is sadly not dynamic like that. What Intel lists as benefit is that recompilation with the compiler flag will have much more impact, while auto-vectorization with older ISA extensions usually has been very conservative. Tho simply recompiling the entire application with the flag would break compatibility with all older processors, meaning addition of dedicated code paths is still needed, and thus I expect adaption to be just as slow as AVX-512 etc.

There are some great questions and answers in this thread, but I thought it would be useful to answer some of those questions for people who don't know machine code - that is - the instructions the CPU actually use to do stuff. Because it helps you understand what's going on under the hood.

Compilers take your readable code in englishish and convert that to the instructions the CPU actually executes: machine code. We often look at machine code in assembly language which is easier to read than the actual hexadecimal code the CPU processes.

Let's take an example in pseudo code

BEGIN
DECLARE a AS INTEGER
DECLARE b AS INTEGER
DECLARE c AS INTEGER

a ← 5
b ← 10

c ← a + b

END

In x86 assembly it would look like this:

mov eax, 5 (move the value 5 to the 32 bit register A)
mov ebx, 10 (move the value 10 to the 32 bit register B)
add eax, ebx (add B to A)
push eax (put the value A on the stack to take off later, if this were a functional subroutine), or
mov ecx, eax (if you actually wanted the value in the c register. Horses for courses)

In the pseudocode, the DECLARE AS INTEGER means nothing to the CPU, but the compiler understands it can use the faster 32bit CPU registers (A, B, C, D) rather than the much slower FPU registers (actually x87 uses a FPU stack to do the math... it's a legacy from decades ago, how they work is an interesting read).

In x86 we have ONLY 4 registers that can do these kind of integer operations. (Even though there are other registers, you can't really use them in the same way, because they are stack pointers or reference registers or flags)

In x86_64 we have an additional 8 general purpose registers (R8-R16) that behave in the same way as A, B, C, D. So now we have 12 registers for doing stuff (again, there are more registers but you can't or shouldn't use them for doing stuff like this or you'll get a crash as a result of misusing stack pointers and so forth)

However, to access these additional 8 general purpose registers we need to tell the CPU to use them, and it requires an extra one byte of memory to do so. It's a special opcode prefix "REX" (0b0100) to say, what comes next needs to be interpreted differently, and now refers to a new set of instructions using these new registers. Without the REX command, the following hexadecimal code would do something entirely different!

As you can see, under x86_64 we have more flexibility but it is cumbersome. It takes more program memory to use these new registers, and consequently, a little more time unless your memory, cache and pipeline can optimise the latency of additional memory access and processing completely away. However, the gain is that now we don't have to keep recycling the primary registers A, B, C, D and storing temporary results somewhere, but have the new 8 registers to help streamline code and execution for a speedup. You need a VERY SMART compiler to take advantage of these new registers using old program codebase. Writing compilers to optimise old code is very complex. Even more so when CPUs have "errata" where things don't always behave properly. So, to get the best performance, old code often needs to be rewritten cognisant of the new architecture so it will use the new commands in the most efficient way.

The pseudocode program c ← 5+10+20+30+40+50 can only be done in x86 by breaking it down into components and cycling through the calculation step by step whereas in x86_64 you could do it without recycling registers by using EAX, EBX, R8, R9, R10, R11. It can (depending on what you are actually doing) make the program smaller and the code faster to execute because you can avoid steps associated with "marshalling the yard" or juggling just the 4 registers A, B, C, D.

However, something complied in x86_64 cannot now be run on x86. The x86 would not understand the REX command and would therefore process the subsequent commands in their original form and not in the modified, use the new registers, execution mode. It would either crash or get garbage out.

Essentially, the same thing is happening in APX. We now have an additional 16 registers to work with. But we need to tell the CPU to use them, using a special opcode or flag, and then what comes next in the hexadecimal machine code is interpreted differently. So again, a program complied for APX will not work in x86 or x86_64 that doesn't understand what to do with this opcode. And because the new registers don't exist in the old architecture, you'd either get a crash or garbage out.

So Windows 11 compiled for APX will not work on a CPU without APX. Whereas Windows 11 compiled for x86_64 will work fine on a CPU with APX but because it does not use these new registers there can
be no improvement in code execution speeds.

Now if Windows 11 is compiled to work on BOTH APX and non-APX x86_64 it needs to have different codepaths depending on whether the CPU "flag" shows if APX instruction exists or not. That means more DLLs as in APX DLLs specifically complied for APX instructions, or BIGGER DLLs that contain the separate codepaths within them.

What a PITA. So many different compiler versions of code creating so many versions of code designed to execute on different architectures. This is very different than bigger cache, faster pipeline, more efficient code execution, parallel execution, speculative branching etc. where the same complied code just runs faster or more efficiently on the new architecture.

So why is Intel doing this? Because ARM RISC has 32 registers and smashes Intel in certain applications. And Intel hopes that in those niche situations, performance gaps can be closed. There are certain applications where the go-to is now ARM. I guess Intel needs to try and address that.

Will it make noticeable performance improvement to Windows 11 for us, desktop users? I don't think so. The 10% fewer instructions that Intel is quoting is interesting but how does this correlate with performance? Might be zero for general productivity software, but it might be much higher for special server or niche applications. I'll watch this space.

But 10% fewer instructions in execution on APX CPU still needs more HDD! Since to be compatible with x86_64 the actual programs will need the original 100% + new codepath 90% = 190% in memory/disk!

references
wiki.osdev.org/CPU_Registers_x86
wiki.osdev.org/CPU_Registers_x86-64
en.wikipedia.org/wiki/REX_prefix

RedBlackAkaThat still requires recompilation, existing applications are unaware of them and cannot benefit. Everything on an instruction level is sadly not dynamic like that. What Intel lists as benefit is that recompilation with the compiler flag will have much more impact, while auto-vectorization with older ISA extensions usually has been very conservative. Tho simply recompiling the entire application with the flag would break compatibility with all older processors, meaning addition of dedicated code paths is still needed, and thus I expect adaption to be just as slow as AVX-512 etc.

Hmm, not quite what I understand. APX is almost not entirely a new instruction set as it is a standard for additional already used instructions inputs.

kondaminThose have been shelved, and it's very unlikely they will ever become a thing.

Unified cores are not shelved, they are planned around Titan Lake if I am not mistaken.

XeonsNOpteronsHmm, not quite what I understand. APX is almost not entirely a new instruction set as it is a standard for additional already used instructions inputs.


Unified cores are not shelved, they are planned around Titan Lake if I am not mistaken.

Ah, my mistake i was thinking about the royal core.
intel dumping Pcores is indeed very likely

XeonsNOpteronsFrom what I have seen about APX, you do not actually need to recompile to get benefits from it. That was one of the targets of APX since it, to my understanding, mostly expands how many general purpose registers there are. Recompiling may help fully leverage APX, but it should not be needed to at least get noticeable improvement.

This is false. APX definitely needs to recompile C/C++/etc source code (or to generate APX versions of functions in addition to the default amd64/x86-64 versions) to obtain a speedup. No recompilation means no improvements at all on a CPU supporting APX.

I think what you mean by "APX doesn't need to recompile to get benefits from it" is that a non-APX function can call an APX function without knowing that it is calling an APX function. Yes, this was a design goal of APX. In this sense you are right, but only in this sense.

• The extra 16 general-purpose 64-bit registers lower the need to spill registers to the stack frame in a larger function, and can enable loop unrolling in cases in which it wasn't previously beneficial
• 3-operand general-purpose instructions can eliminate a subsequent MOV instruction, and can lower the need to spill registers to the stack frame because the values of both input registers are preserved (in case of 2-operand instructions: value of 1 input register is preserved, the other input register is overwritten).

lemonadesodaNow if Windows 11 is compiled to work on BOTH APX and non-APX x86_64 it needs to have different codepaths depending on whether the CPU "flag" shows if APX instruction exists or not. That means more DLLs as in APX DLLs specifically complied for APX instructions, or BIGGER DLLs that contain the separate codepaths within them.

Not every program is a Windows program. Gentoo Linux (which compiles most packages from their source code during installation) will have no problems to transition to APX on the very same day the Gentoo user upgrades the CPU to Nova Lake in 2027+. It is likely that performance-oriented Linux distributions, such as Arch Linux, will start providing binary APX-only packages. Thus, bigger DLLs is a problem specific to Windows and to Linux distributions that prefer long-term stability over performance.

XeonsNOpterons..., to my understanding, mostly expands how many general purpose registers there are.

The main motivation of APX is to deal with problem the cost of branch misprediction becoming increasingly dominant and to scale further OoO designs (source), by introducing conditional load, store and compare/test instructions.

For those not too familiar with low-level programming, this means a lot of small branching logic (if/switch) can be completely eliminated resulting in larger sections of linear code, which means fewer pipeline stalls, increased front-end efficiency and higher utilization of execution ports.

This is the largest architectural overhaul of the x86 ISA in decades, yet most miss the core features of it.

The increase of general purpose registers and three-operand instructions are more low-hanging fruit to reduce e.g. register shuffling / unnecessary load/stores preventing the CPU from effectively feeding the pipeline.

I do expect though that it will take some time for compilers to harness the full potential of the new conditional instructions. The benefits of more registers and three-operand instructions will be immediate though. And I wouldn't expect Nova Lake and Diamond Rapids to be fully fine-tuned either, wait for the second or third iteration before judging it.

atomsymbolThis is false. APX definitely needs to recompile C/C++/etc source code (or to generate APX versions of functions in addition to the default amd64/x86-64 versions) to obtain a speedup. No recompilation means no improvements at all on a CPU supporting APX.

I think what you mean by "APX doesn't need to recompile to get benefits from it" is that a non-APX function can call an APX function without knowing that it is calling an APX function. Yes, this was a design goal of APX. In this sense you are right, but only in this sense.

• The extra 16 general-purpose 64-bit registers lower the need to spill registers to the stack frame in a larger function, and can enable loop unrolling in cases in which it wasn't previously beneficial
• 3-operand general-purpose instructions can eliminate a subsequent MOV instruction, and can lower the need to spill registers to the stack frame because the values of both input registers are preserved (in case of 2-operand instructions: value of 1 input register is preserved, the other input register is overwritten).

Ah, ok, thanks for the explanation that makes more sense! APX is one of the most confusing instructions from Intel yet for me to grasp lol.

efikkanThe main motivation of APX is to deal with problem the cost of branch misprediction becoming increasingly dominant and to scale further OoO designs (source), by introducing conditional load, store and compare/test instructions.

For those not too familiar with low-level programming, this means a lot of small branching logic (if/switch) can be completely eliminated resulting in larger sections of linear code, which means fewer pipeline stalls, increased front-end efficiency and higher utilization of execution ports.

This is the largest architectural overhaul of the x86 ISA in decades, yet most miss the core features of it.

The increase of general purpose registers and three-operand instructions are more low-hanging fruit to reduce e.g. register shuffling / unnecessary load/stores preventing the CPU from effectively feeding the pipeline.

I do expect though that it will take some time for compilers to harness the full potential of the new conditional instructions. The benefits of more registers and three-operand instructions will be immediate though. And I wouldn't expect Nova Lake and Diamond Rapids to be fully fine-tuned either, wait for the second or third iteration before judging it.

Thanks for your explanation as well! I doubt there will be a clock speed penalty for APX like AVX-512 or AMX, but the compilers will likely need time to properly leverage APX if I understand your and atomsymbols's explanations.

XeonsNOpteronsI doubt there will be a clock speed penalty for APX like AVX-512 or AMX.

Time will tell, but only if it's throttling due to better saturation of execution ports, which wouldn't be any kind of performance disadvantage to be clear. ;)

XeonsNOpteronsI doubt there will be a clock speed penalty for APX like AVX-512 or AMX, but the compilers will likely need time to properly leverage APX if I understand your and atomsymbols's explanations.

The main penalty of APX is in the instruction decode phase of the CPU architecture, in requiring a bit more read bandwidth from L1I cache, and in the transistors (most likely a relatively small number) that will implement the operational semantics of the APX instructions. ---- Importantly, the cost of the additional 16 general-purpose registers is virtually zero - this is because all modern x86 CPU cores already have ~200 registers in their integer physical register file and exposing additional 16 registers to the compiler/programmers costs virtually nothing in terms of transistor count.

atomsymbolThe main penalty of APX is in the instruction decode phase of the CPU architecture, in requiring a bit more read bandwidth from L1I cache, and in the transistors (most likely a relatively small number) that will implement the operational semantics of the APX instructions. ---- Importantly, the cost of the additional 16 general-purpose registers is virtually zero - this is because all modern x86 CPU cores already have ~200 registers in their integer physical register file and exposing additional 16 registers to the compiler/programmers costs virtually nothing in terms of transistor count.

I will be very interested to see how much it actually impacts performance. I got about double the solving performance in OpenFOAM with it compiled from source using AVX10.1 flags enabled versus the precompiled version which likely just leveraged AVX/AVX2 at most. The new leak of PassMark scores for 24/28 core Nova Lake SKUs show massive "lower instruction" (SSE,AMX,FMA) IPC boosts when I look at the subscores on my PassMark portal. Then again, as I noted on that post, we do not know the all-core clocks and they are likely low due to being ES or QS at best.

Would APX boost SSE/AVX/FMA throughput?

Intel seams to like APX acronym for some reason.

XeonsNOpteronsWould APX boost SSE/AVX/FMA throughput?

From the APX specification:

All instructions in the EVEX space can access the upper 16 GPRs in their memory operands.

x86 instruction encoding spaces:

• SSE instructions are in legacy encoding space
• AVX/AVX2 instructions are in VEX encoding space
• AVX-512 instructions are in EVEX encoding space
  • Additionally, AVX/AVX2 instructions have an EVEX encoding if the CPU supports AVX512VL (most AVX-512 CPUs support this)

The answer to your question is: APX improves throughput of vector instructions only if those instructions have memory operands. For example: vpaddd ymm0, ymm1, [RAX+R21] where R21 is an APX register.

lemonadesodaThere are some great questions and answers in this thread, but I thought it would be useful to answer some of those questions for people who don't know machine code - that is - the instructions the CPU actually use to do stuff. Because it helps you understand what's going on under the hood.

Compilers take your readable code in englishish and convert that to the instructions the CPU actually executes: machine code. We often look at machine code in assembly language which is easier to read than the actual hexadecimal code the CPU processes.

Let's take an example in pseudo code

BEGIN
DECLARE a AS INTEGER
DECLARE b AS INTEGER
DECLARE c AS INTEGER

a ← 5
b ← 10

c ← a + b

END

In x86 assembly it would look like this:

mov eax, 5 (move the value 5 to the 32 bit register A)
mov ebx, 10 (move the value 10 to the 32 bit register B)
add eax, ebx (add B to A)
push eax (put the value A on the stack to take off later, if this were a functional subroutine), or
mov ecx, eax (if you actually wanted the value in the c register. Horses for courses)

In the pseudocode, the DECLARE AS INTEGER means nothing to the CPU, but the compiler understands it can use the faster 32bit CPU registers (A, B, C, D) rather than the much slower FPU registers (actually x87 uses a FPU stack to do the math... it's a legacy from decades ago, how they work is an interesting read).

In x86 we have ONLY 4 registers that can do these kind of integer operations. (Even though there are other registers, you can't really use them in the same way, because they are stack pointers or reference registers or flags)

In x86_64 we have an additional 8 general purpose registers (R8-R16) that behave in the same way as A, B, C, D. So now we have 12 registers for doing stuff (again, there are more registers but you can't or shouldn't use them for doing stuff like this or you'll get a crash as a result of misusing stack pointers and so forth)

However, to access these additional 8 general purpose registers we need to tell the CPU to use them, and it requires an extra one byte of memory to do so. It's a special opcode prefix "REX" (0b0100) to say, what comes next needs to be interpreted differently, and now refers to a new set of instructions using these new registers. Without the REX command, the following hexadecimal code would do something entirely different!

As you can see, under x86_64 we have more flexibility but it is cumbersome. It takes more program memory to use these new registers, and consequently, a little more time unless your memory, cache and pipeline can optimise the latency of additional memory access and processing completely away. However, the gain is that now we don't have to keep recycling the primary registers A, B, C, D and storing temporary results somewhere, but have the new 8 registers to help streamline code and execution for a speedup. You need a VERY SMART compiler to take advantage of these new registers using old program codebase. Writing compilers to optimise old code is very complex. Even more so when CPUs have "errata" where things don't always behave properly. So, to get the best performance, old code often needs to be rewritten cognisant of the new architecture so it will use the new commands in the most efficient way.

The pseudocode program c ← 5+10+20+30+40+50 can only be done in x86 by breaking it down into components and cycling through the calculation step by step whereas in x86_64 you could do it without recycling registers by using EAX, EBX, R8, R9, R10, R11. It can (depending on what you are actually doing) make the program smaller and the code faster to execute because you can avoid steps associated with "marshalling the yard" or juggling just the 4 registers A, B, C, D.

However, something complied in x86_64 cannot now be run on x86. The x86 would not understand the REX command and would therefore process the subsequent commands in their original form and not in the modified, use the new registers, execution mode. It would either crash or get garbage out.

Essentially, the same thing is happening in APX. We now have an additional 16 registers to work with. But we need to tell the CPU to use them, using a special opcode or flag, and then what comes next in the hexadecimal machine code is interpreted differently. So again, a program complied for APX will not work in x86 or x86_64 that doesn't understand what to do with this opcode. And because the new registers don't exist in the old architecture, you'd either get a crash or garbage out.

So Windows 11 compiled for APX will not work on a CPU without APX. Whereas Windows 11 compiled for x86_64 will work fine on a CPU with APX but because it does not use these new registers there can
be no improvement in code execution speeds.

Now if Windows 11 is compiled to work on BOTH APX and non-APX x86_64 it needs to have different codepaths depending on whether the CPU "flag" shows if APX instruction exists or not. That means more DLLs as in APX DLLs specifically complied for APX instructions, or BIGGER DLLs that contain the separate codepaths within them.

What a PITA. So many different compiler versions of code creating so many versions of code designed to execute on different architectures. This is very different than bigger cache, faster pipeline, more efficient code execution, parallel execution, speculative branching etc. where the same complied code just runs faster or more efficiently on the new architecture.

So why is Intel doing this? Because ARM RISC has 32 registers and smashes Intel in certain applications. And Intel hopes that in those niche situations, performance gaps can be closed. There are certain applications where the go-to is now ARM. I guess Intel needs to try and address that.

Will it make noticeable performance improvement to Windows 11 for us, desktop users? I don't think so. The 10% fewer instructions that Intel is quoting is interesting but how does this correlate with performance? Might be zero for general productivity software, but it might be much higher for special server or niche applications. I'll watch this space.

But 10% fewer instructions in execution on APX CPU still needs more HDD! Since to be compatible with x86_64 the actual programs will need the original 100% + new codepath 90% = 190% in memory/disk!

references
wiki.osdev.org/CPU_Registers_x86
wiki.osdev.org/CPU_Registers_x86-64
en.wikipedia.org/wiki/REX_prefix

Thanks for the detailed explanation. I highly doubt we'll see APX-specific application builds on Windows. Devs have enough trouble building for x64 and arm64 as is.

AleksandarKIntel's APX is set to debut with the "Nova Lake" processors in early 2027

Do you know which generation of AMD CPUs will also have these new instructions? Zen 6? Zen 7?

jdrchI highly doubt we'll see APX-specific application builds on Windows. Devs have enough trouble building for x64 and arm64 as is.

Perhaps not from Microsoft, but at least open source productive applications and tools will probably quickly arrive, as well as some proprietary applications, e.g. DaVinci Resolve is fairly likely.

For software relying on Microsoft's compiler it will probably take a while though, they only got "experimental" support back in May. But gcc has had support for a good while already.

Compared to porting software to ARM it should be trivial though, at least in theory; just change a flag and recompile. There shouldn't be any compatibility issues or new APIs required, so aside from compiler bugs there shouldn't be logical bugs. But the faster execution may expose some underlying designs flaws in code though, i.e. race conditions etc.

efikkanPerhaps not from Microsoft, but at least open source productive applications and tools will probably quickly arrive, as well as some proprietary applications, e.g. DaVinci Resolve is fairly likely.

For software relying on Microsoft's compiler it will probably take a while though, they only got "experimental" support back in May. But gcc has had support for a good while already.

Compared to porting software to ARM it should be trivial though, at least in theory; just change a flag and recompile. There shouldn't be any compatibility issues or new APIs required, so aside from compiler bugs there shouldn't be logical bugs. But the faster execution may expose some underlying designs flaws in code though, i.e. race conditions etc.

In theory, yes. In practice, the install base is likely to be too small to be worth the effort unless AMD implements something similar.

azraelI would've expected a Windows 12 announcement for this feature. After all, Windows 11 pretty much came about because Intel needed a new scheduler in Windows for their Alder Lake processors. But perhaps Intel isn't as important to Microsoft as it used to be.

I have never heard anything about Windows 11 just been made for a scheduler.

Microsoft have been patching schedulers without a major new version for years now, Windows 11 was most likely released to obsolete old hardware and allow then to ditch the old UI.

jdrchIn theory, yes. In practice, the install base is likely to be too small to be worth the effort unless AMD implements something similar.

They will in all likelihood.

APX will have more significant benefits for user-interactive applications and games than big server workloads, so basically anything that's logic-heavy. Zen 7 will feature the much more obscure AMX, which currently mainly have "AI" uses, and requires major die space. APX by comparison is fairly easy to implement, and will likely yield even greater gains for AMD than Intel, as their current designs are wider but have a weaker front-end.