I think the problem is that the motherboard's memory topology can't reliably handle 4 memory modules at this speed. Despite being validated to run at up to 4400MHz with two memory slots populated, the fastest 4-module kit that the B450-A Pro is validated for is 3733MHz which isn't far above 3600, and it's with much looser timings (CL19). The fastest 3600MHz 4x 8GB kit it's validated for is CL17.
As I understand it the motherboard shouldn't significantly affect the stability of different memory timings (which are more dependent on the CPU's memory controller), but it might be enough to push you over the edge into instability in a situation when you're very close to the limit of what the motherboard is capable of. At least in theory, changing the CPU voltage shouldn't have any effect, but based on your description it seems to, which makes me suspect that it could be temperature-related, or possibly some strange cross-CPU electrical effects. It's difficult to say without knowing more about the specifics of what you're running.
Once you've got memory and voltage settings that seem stable, you should try running a memory stability test. A version of hci memtest comes bundled with DRAM calculator for Ryzen, and you can download hci memtest or memtest86 individually. If you run a few instances with around 2000MB each, and wait for their coverage to get up to 100% of your memory (which may take an hour or so) you shouldn't get any error notifications. If it shows any errors, it means that your memory is not stable.
If you need to underclock your memory to get it stable, you could potentially reduce the memory timings to something like 3200MHz CL14-14-14-34 (and tightening some of the minor timings like tRFC can also help a lot). This would give you similar performance to 3600MHz CL16 in most situations.
If you didn't see clock fluctuations with an older Intel CPU it's possible that you had a CPU without turbo boost (e.g. an i3 7100) or that you had disabled turbo boost in BIOS or set a power limit in windows. The clock frequency of most modern CPUs is variable, unless you set a fixed clock multiplier yourself. You will usually get much better performance with a Ryzen 3000-series CPU by allowing it to boost on its own than if you set a fixed clock speed. You can gain performance from a fixed overclock on a Ryzen 5 3600 in some situations if you set the right voltage and LLC, but even if you know what you're doing the improvement is usually very minor.