Generally I'll see CPU-dependent performance drops in the castle room outside the library with the mermaid sculpture and of course in Hogsmeade but recently I found one that is horrible, probably a game bug. In the little outside courtyard area just off the aforementioned room outside the library, if you run an anticlockwise loop around the grass, as you approach the big tree to keep to it's right, the fps will TANK. Really bad. Strafe run looking to your L a bit and it's the worst. I run at 60fps VSync for smoothness and the fps will drop to the mid 20s and I've even got it to the teens. It'll recover after a few seconds to 60fps. You can continue running a loop in the courtyard and the next time you get to the same area, you'll get the identical problem again. ~20 fps which recovers. This seems to be the worst in Winter (later game). I'm now in Spring and it seems a little better but still crashes to the 30s fps.
GPU is barely being used and really CPU seems nonplussed as well so I'm not sure this is even a CPU issue? Bug/crap optimization I guess. I don't remember that problem in Summer/Autumn but in general FPS consistency in that area is pretty bad (a better CPU like a 5800X3D makes a real difference), but this is another layer of performance problem. AMD and Nvidia GPUs behave similarly as do Intel and AMD CPUs.