TheoneandonlyMrKWell we are talking about the 8/16 part in the Op not the single big core variants soooo, I mean yeh you do you but we're talking about what Might do us ,who needs mooooooooar cores.

Actually the "leak" from "Moore's Law is Dead" we are talking about, has many more details and possible configurations. About small workstations I already made a comment earlier.

lynx29can you explain AVX to me in quick summary, I have never understood it... I am guessing it only applies to work stations not actual gamers?

A normal computer is basically a glorified calculator. If it sees the "add" instruction, then you get "x + y". If it sees the "mul" instruction, it does "x*y".

AVX512 is a special instruction that says "add 512-bits in parallel", so you get "x0 + y0, x1+y1, x2+y2... x15+y15" (16 operations in one clock tick). AVX512 uses more power, but its more efficient than 16-individual add instructions.

Because of this weird "16 different adds" thing going on, its actually very difficult to use. Common programmers largely haven't figured out how to use it, its just not taught in universities. A lot of professional programmers study the instruction set and learn how to do it however. Video-game programmers are surprisingly focused on efficiency, and I've actually seen evidence that video games use these kinds of instructions (ex: Command and Conquer's source code was released recently, and there are all sorts of MMX routines: which was the old version of AVX way back in the 1990s)

AVX512 contains most arithmetic people want: add, subtract, and multiply. (No division!! As is common in 'high performance' instruction sets, division is so inefficient that its often just left out all together). Some video games do use AVX512, but its not very common yet.

AVX2 (256-bit, or 8-operations per instruction) was first deployed in 2014 on 4th generation Intel, and is also fully deployed on AMD Zen. As such, AVX2 (256-bit / 8-operations) is pretty standard these days.

dragontamer5788A normal computer is basically a glorified calculator. If it sees the "add" instruction, then you get "x + y". If it sees the "mul" instruction, it does "x*y".

AVX512 is a special instruction that says "add 512-bits in parallel", so you get "x0 + y0, x1+y1, x2+y2... x15+y15" (16 operations in one clock tick). AVX512 uses more power, but its more efficient than 16-individual add instructions.

AVX512 contains most arithmetic people want: add, subtract, and multiply. (No division!! As is common in 'high performance' instruction sets, division is so inefficient that its often just left out all together). Some video games do use AVX512, but its not very common yet.

AVX2 (256-bit, or 8-operations per instruction) was first deployed in 2014 on 4th generation Intel, and is also fully deployed on AMD Zen. As such, AVX2 (256-bit / 8-operations) is pretty standard these days.

so eventually they will create an AVX specific to quantum computers too, if my logic is correct... ? lol

lynx29so eventually they will create an AVX specific to quantum computers too, if my logic is correct... ? lol

Quantum is completely different. Quantum "qbits" are "entangled", causing them to be correlated with each other in ways that physics professors understand (but that almost no normal programmer understands). Some professors are studying quantum instructions just in case a practical quantum computer is ever made (in theory, it can do things like RSA factorization, or O(n) sorting / searching routines), but since its impractical and very expensive today... few people bother to learn quantum theories.

AVX is just a weird form of parallelism that's hard to use (but very, very fast and efficient).

If you have a 8-big-core processor with AVX512, you can do 16 operations per core x 8 cores == 128 operations per processor per clock tick.

These AVX instructions are nearly everywhere. So I personally think high-performance programmers should definitely learn how to use them. AVX512 is the next version Intel is making. SVE is ARM's version. GPU programs are implicitly SIMD from the ground up.

TheoneandonlyMrKWell we are talking about the 8/16 part in the Op not the single big core variants soooo, I mean yeh you do you but we're talking about what Might do us ,who needs mooooooooar cores.

Don't ya mean mooooooooooooar glue++++++++!!? :rolleyes: Raptor Lake featuring Intels finest leading edge nanometre glue technology.

Hi,
Seems this should really be named aneurysms lake lol

InVasManiDon't ya mean mooooooooooooar glue++++++++!!? :rolleyes: Raptor Lake featuring Intels finest leading edge nanometre glue technology.

I nearly shouted back "who doesn't love glue" but shit there's too many connotations there.
I'm a fan of glue if it brings new tech, tech glue obvs.

dragontamer5788A normal computer is basically a glorified calculator. If it sees the "add" instruction, then you get "x + y". If it sees the "mul" instruction, it does "x*y".

AVX512 is a special instruction that says "add 512-bits in parallel", so you get "x0 + y0, x1+y1, x2+y2... x15+y15" (16 operations in one clock tick). AVX512 uses more power, but its more efficient than 16-individual add instructions.

Because of this weird "16 different adds" thing going on, its actually very difficult to use. Common programmers largely haven't figured out how to use it, its just not taught in universities. A lot of professional programmers study the instruction set and learn how to do it however. Video-game programmers are surprisingly focused on efficiency, and I've actually seen evidence that video games use these kinds of instructions (ex: Command and Conquer's source code was released recently, and there are all sorts of MMX routines: which was the old version of AVX way back in the 1990s)

AVX512 contains most arithmetic people want: add, subtract, and multiply. (No division!! As is common in 'high performance' instruction sets, division is so inefficient that its often just left out all together). Some video games do use AVX512, but its not very common yet.

AVX2 (256-bit, or 8-operations per instruction) was first deployed in 2014 on 4th generation Intel, and is also fully deployed on AMD Zen. As such, AVX2 (256-bit / 8-operations) is pretty standard these days.

AVX is not something rare. It can be auto-generated by the compiler. If you ever turn on -O3 optimization flag (which includes -ftree-loop-vectorize option) in GCC, then you get auto vectorization for all eligible loops in your code. If you write simple loops, they will be transformed in to equivalent AVX/SSE code depending on your target architecture.

Normally, for education purpose and open source communities, -O3 is not something you see everyday. But for production usages, -O3 is almost enabled everywhere.

john_And people will be buying the wagon with the 8 horses and the 16 dogs, thinking they are getting 24 horses, while AMD will be trying to sell a cart with "only" 16 horses.

Missed opportunity there for it to have been 8 Clydesdales and 16 Miniature Ponies, while AMD remains consistent with 16 Clydesdales. 24 horses vs 16, but not all 24 are the same.

First StrikeAVX is not something rare. It can be auto-generated by the compiler. If you ever turn on -O3 optimization flag (which includes -ftree-loop-vectorize option) in GCC, then you get auto vectorization for all eligible loops in your code. If you write simple loops, they will be transformed in to equivalent AVX/SSE code depending on your target architecture.

Normally, for education purpose and open source communities, -O3 is not something you see everyday. But for production usages, -O3 is almost enabled everywhere.

Autogenerated AVX is not very good in my experience.

Good AVX code requires understanding cache alignment, array-of-structs vs struct-of-arrays and other such concepts that just aren't taught. One of the most important paradigms is prefix-sum, and that's just... not taught anywhere as far as I can tell.

Autogenerated AVX only really works on the simplest of SAXY code that a college-student can probably parallelize. Its nice that a lot of these simple cases are auto-handled by a compiler these days, but really good code requires understanding the SIMD programming model

lynx29can you explain AVX to me in quick summary, I have never understood it... I am guessing it only applies to work stations not actual gamers?

AVX like other forms of SIMD addresses data level parallelism, which is whenever you want to do the same operation on multiple entities of data. By performing multiple math operations in a single instruction, you don't just save those instructions, you also save "overhead" in terms of looping and moving data around in registers, plus you gain efficiency for the instruction cache. Done right, the performance gains could be massive. But doing it right usually requires hand written low-level code.

MusselsThink of it as hardware dedicated to a certain task, very much the same as hardware accelerated playback for say H264 on a GPU

Most of AVX is actually general purpose, only some smaller subsets addresses specific algorithms or things like "AI".

R0H1T24c ~ that's their selling point, come on it's not like the masses will know 5950x(t?) will pulverize it in MT tasks! They'll probably go OMG 24 cores, without actually looking at what the cores are capable of :shadedshu:

Most desktops are sold through the large PC manufacturers, where "specs" sell. That's the primary motivation for this hybrid core nonsense on the desktop.

BrainCruserGames use AVX all the time.

Fairly few games use AVX directly, but hopefully more will soon.

dragontamer5788Autogenerated AVX is not very good in my experience.

Good AVX code requires understanding cache alignment, array-of-structs vs struct-of-arrays and other such concepts that just aren't taught. One of the most important paradigms is prefix-sum, and that's just... not taught anywhere as far as I can tell.

Autogenerated AVX only really works on the simplest of SAXY code that a college-student can probably parallelize. Its nice that a lot of these simple cases are auto-handled by a compiler these days, but really good code requires understanding the SIMD programming model

Yes, auto vectorization does not do much, but they do carry around 3% improvement in most packages (I remember seeing that figure from some Gentoo compilation tests). That's something for free still.

First StrikeYes, auto vectorization does not do much, but they do carry around 3% improvement in most packages (I remember seeing that figure from some Gentoo compilation tests). That's something for free still.

It's much better than that. It might be 3% in average, across all kinds of software. But with more performant code, which usually is more cache optimized, we can often see 10-15% or more, so the free gains can be quite significant.
But with that being said, auto vectorization is still nothing compared to well used AVX intrinstics.

First StrikeYes, auto vectorization does not do much, but they do carry around 3% improvement in most packages (I remember seeing that figure from some Gentoo compilation tests). That's something for free still.

Except it's not for free. AVX uses more power so while you might gain some performance, you're getting it at the cost of additional power usage because more circuitry simply translates to more heat. In the case of auto-vectorization, it's likely not worth the cost if you care about power efficiency. The point of the chopped down cores is to be efficient, not fast. It is more efficient to use smaller cores for things that can't be significantly improved by AVX or where the time you have to complete the computation is long enough to be processed by a slower and cut down core. Also, implementing parallelization in the code itself is likely going to be easier to letting auto-vectorization figure it out, getting intrinsics to work well, or flat out writing assembly for those cases.

So even if a task can use AVX, depending on the use case, it might be overkill. Putting AVX on power efficient cores is practically a contradiction as well because AVX is anything but efficient in terms of power consumption because it's a very wide execution unit. Same deal with cache. SRAM takes up a lot of space and makes a good amount of heat.

AquinusExcept it's not for free. AVX uses more power so while you might gain some performance, you're getting it at the cost of additional power usage because more circuitry simply translates to more heat.

This is very inaccurate.
AVX does not use more power per computation, it's actually far more efficient. The only times AVX loads use "more power" is when they perform a lot more work that you could do without AVX.
As a matter of fact, in all x86-64 designs, all FPU operations are fed through AVX-/SSE-units. The difference is whether the AVX units are fed single or multiple pieces of data.

efikkanThis is very inaccurate.
AVX does not use more power per computation, it's actually far more efficient. The only times AVX loads use "more power" is when they perform a lot more work that you could do without AVX.
As a matter of fact, in all x86-64 designs, all FPU operations are fed through AVX. The difference is whether the AVX units are fed single or multiple pieces of data.

Sure, if you're filling up all of the FPUs involved in each AVX op. FPUs that aren't being fully used in the vector math still add heat because you don't need the full width and aren't powergated. So unless you're building it to fully load them up, you're not going to see those kinds of gains. Simply put, it's only efficient if it's been tuned to be. Auto-vectorization is really unlikely to get you that level occupancy when running AVX ops. It's only efficient if you can fill the vector.

AquinusSure, if you're filling up all of the FPUs involved in each AVX op. FPUs that aren't being fully used in the vector math still add heat because you don't need the full width and aren't powergated. So unless you're building it to fully load them up, you're not going to see those kinds of gains. Simply put, it's only efficient if it's been tuned to be. Auto-vectorization is really unlikely to get you that level occupancy when running AVX ops. It's only efficient if you can fill the vector.

The CPU will run it through the vector units (AVX units) regardless. Single FPU operations are the ones which will cause unfilled vectors, and be the least efficient. Auto vectorization will only occur when the data and operations are aligned properly, and will not get less efficiency than single FPU operations (through vector units).

efikkanThe CPU will run it through the vector units (AVX units) regardless. Single FPU operations are the ones which will cause unfilled vectors, and be the least efficient. Auto vectorization will only occur when the data and operations are aligned properly, and will not get less efficiency than single FPU operations (through vector units).

I seriously doubt that the full width is active when you're running a normal FP op because the width of the op being asked for is static. This is a case where the rest of it is probably powergated because it's known how wide the FPU(s) needs to be, but in the case of actually using AVX, the full width of the unit is active even if you can't get full occupancy of the vector that you're doing the op on. There is a difference between the two.

AquinusI seriously doubt that the full width is active when you're running a normal FP op because the width of the op being asked for is static. This is a case where the rest of it is probably powergated because it's known how wide the FPU(s) needs to be, but in the case of actually using AVX, the full width of the unit is active even if you can't get full occupancy of the vector that you're doing the op on. There is a difference between the two.

Even if it managed to power gate based on data width, it still wouldn't change the fact that 16 cycles of single floats would consume way more power than a single iteration of 16 floats, not to mentions the additional MOVs and iterations saved on top of that. So your claim of AVX being less efficient is fundamentally flawed.

efikkanit still wouldn't change the fact that 16 cycles of single floats would consume way more power than a single iteration of 16 floats

That's where you're wrong. If you can spare the time, a slim core running at lower clock speeds with less cache will actually be more efficient in terms of power used even if takes more cycles to complete because you're driving those transistors at lower speeds, lower voltages, and there are overall fewer transistors which means less leakage. You're only correct if the workload is big and for general tasks, they're not. You're not using these cores to play games. You're using them when you're using the browser or doing things that can afford the latency penalty.

I'm not arguing that AVX isn't faster or more efficient under full load. I'm saying that unless you're under full load, AVX and big cores aren't in comparison to slim cores tuned for power efficiency.

efikkanit still wouldn't change the fact that 16 cycles of single floats would consume way more power than a single iteration of 16 floats, not to mentions the additional MOVs and iterations saved on top of that.

You must have missed the part where I said:

AquinusSure, if you're filling up all of the FPUs involved in each AVX op. FPUs that aren't being fully used in the vector math still add heat because you don't need the full width and aren't powergated. So unless you're building it to fully load them up, you're not going to see those kinds of gains. Simply put, it's only efficient if it's been tuned to be. Auto-vectorization is really unlikely to get you that level occupancy when running AVX ops. It's only efficient if you can fill the vector.

If you have a really heavy workload like that, you probably are already using the high power cores. You're assuming that the vector is always 16 values wide when that's highly unlikely unless the code has been specifically tuned to use the entire width of an AVX op. The reality is that most of the time, full AVX width isn't used unless the software is tuned for it and that work isn't going to be done unless it's necessary and will benefit from it. Either way, you're running on the assumption that AVX is going to help everything and it's not. The low power cores are to handle the other cases which happen far more frequently than hitting AVX.

With that said, if you don't believe me, just look at the power consumption numbers for Apple's M1 chip and I assure you it's not just because it's ARM and on a smaller process.

I could see Intel taking the cache structure and have two identical cores rather than big LITTLE where they adjust the typical cache structure between two otherwise identical chip dies. Probably the L3 would be identical so you could get full multi thread performance across the L3 cache. The L1 and L2 could be more varied where one die might have a bigger L1 and smaller L2 while the other would have a smaller L1 and bigger L2. They could basically trade off L1/L2 cache sizes for cache latency between dies and then depending on the task pick the faster of the two up to the chip die core and threading limits of the L1/L2 caches. They might even be capable of combing together and operating at the slower of the L1/L2 cache sizes latency values though possibly limited to double that of the smaller cache size. That would still be better than a cache hit miss and using next slower cache L2/L3 however.


To summary in most instances it would likely be faster, but in a few instances it could be marginally slower for the L1 and L2 cache up to a point. That said it would be marginally slower some clock cycles, but has the by product of less waste heat in turn and make the chip dies able to turbo boost higher or longer duration the follow clock cycle so balances out a bit. It both provides some ample power down savings and heat reduction while improving the L1/L2 cache performance reasonable amount of time. Perhaps they could bake that into each die so the OS sees each core as identical, but the core itself is selective at only running half the threads when and where needed as opposed to all of them. Basically it could be like a more true hyperthreading on alternating clock cycles at times or tiny cache latency penalty in some heavily multithreaded instances though still a lot better than bigLITTLE on performance.

Calmmoarm does low power better.

Ah, so that is why it gets beaten by EPIC, or why M1 (5nm) is getting beaten by Ryzen 4000 series (meh 7nm)... oh, doh, wait... :D

BrainCruserCurrent CPUs can execute ~3 AVX or 3 normal add instructions per cycle. So the 16X data points = 16X faster.

This does not explain why one would use CPU over GPU (thousands of shaders) for it.

And in general, why do we need that big/little thing again?
ASUS gaming (!!!) laptop with (amazing) 6800m in it had 10+ hour battery life (browsing/video :), not gaming of course), Ryzen 5000 series can get close to 20 hours, so what is the point?

Keep in mind that:


citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.87.5604&rep=rep1&type=pdf#:~:text=We%20found%20that%20total%20system,components%20varies%20even%20more%20widely.&text=The%20display%20is%20the%20other,when%20the%20CPU%20is%20idle.

AquinusThat's where you're wrong. If you can spare the time, a slim core running at lower clock speeds with less cache will actually be more efficient in terms of power used even if takes more cycles to complete because you're driving those transistors at lower speeds, lower voltages, and there are overall fewer transistors which means less leakage.

You are really grasping at straws here.
Doing the same work without AVX (or other SIMD) would usually require >20x the instructions, and you want to offset that extra power required by running the core at a very low clock speed, probably making it about 100x slower, this is not a very realistic usage scenario.
The fact remains that AVX is more power efficient.

AquinusI'm not arguing that AVX isn't faster or more efficient under full load. I'm saying that unless you're under full load, AVX and big cores aren't in comparison to slim cores tuned for power efficiency.

I assume you are still talking in the context of auto-vectorizing here.
Your assumptions here about saturating the vector units is fundamentally flawed. Auto-vectorizing only happens when the data is dense and the operations in a loop easily translates to AVX operations. It's not like the compiler will take random FPU operations and stuff them together in vectors.

Auto-vectorization will not hurt your efficiency or performance, but there are some considerations;
- Sometimes the gains are negligible, because the code is too bloated, the data isn't dense and/or the operations inside the loops aren't simple enough.
- If FMA is enabled, the data produced will no longer be binary compatible, which may or may not be a problem.

AquinusWith that said, if you don't believe me, just look at the power consumption numbers for Apple's M1 chip and I assure you it's not just because it's ARM and on a smaller process.

ARM achieve efficiency with special instructions to accelerate specific workloads, and yes, ASIC will beat SIMD in efficiency, but SIMD is general purpose.

Hell, i learned a fair bit about AVX from you nerds arguing


keep it up, education is good

efikkanYou are really grasping at straws here.
Doing the same work without AVX (or other SIMD) would usually require >20x the instructions, and you want to offset that extra power required by running the core at a very low clock speed, probably making it about 100x slower, this is not a very realistic usage scenario.
The fact remains that AVX is more power efficient.

Once again, you're assuming the vector is completely occupied which is a bad assumption. I'm not saying your wrong, I'm saying your premise is bad. Also you're also assuming that the increased time to execute is going to harm performance when you have no idea if it's the bottleneck for the application. If it's running on a low power core, it's probably not, otherwise it wouldn't be running there. The reality is that the case you're describing is the case that'd already be hitting the high power cores.

efikkanI assume you are still talking in the context of auto-vectorizing here.
Your assumptions here about saturating the vector units is fundamentally flawed. Auto-vectorizing only happens when the data is dense and the operations in a loop easily translates to AVX operations. It's not like the compiler will take random FPU operations and stuff them together in vectors.

Auto-vectorization will not hurt your efficiency or performance, but there are some considerations;
- Sometimes the gains are negligible, because the code is too bloated, the data isn't dense and/or the operations inside the loops aren't simple enough.
- If FMA is enabled, the data produced will no longer be binary compatible, which may or may not be a problem.

Agreed on FMA. If you vectorize an operation though, there is a power penelty from driving the full width of AVX if you're not using the entire thing. Even half occupancy in situations where time isn't the limiting factor, a smaller, slim core is likely going to use less power. I think you underestimate how many more transistors it takes to implement AVX and the cache backing to support it. Just because auto-vectorization runs doesn't mean that it's always a perfect situation where you'll get 100% occupancy. Sure, performance might not get worse, but power consumption can.

efikkanARM achieve efficiency with special instructions to accelerate specific workloads, and yes, ASIC will beat SIMD in efficiency, but SIMD is general purpose.

...and Apple specifically made the choice to make slim cores that couldn't do everything for that case. It's not like the slim cores are ASIC circuits. That hardware is going to be in the high power cores. Once again, you're focusing on full load, high power situations. Outside of that context, it doesn't make sense to have that kind of bulk in the slim cores because you're negating the advantage they have and you might as well just have a bunch of high power cores instead, but the reality is that most CPUs aren't running full tilt most of the time and there are efficiency benefits to be had from that.

MusselsHell, i learned a fair bit about AVX from you nerds arguing


keep it up, education is good

Yes it is, but so is properly understanding the problem being solved.