While OCing the CPU will give you a bit of a boost, you'll need to put at least $20 into an aftermarket cooler, and it will still be just a dual core. More and more top games released make use of quad core tech, it can make quite a difference. There's also some CPU, MB, RAM bundles this time of year that are tempting if you can afford to buy now.
I say save that cooler money and put it toward a better CPU, MB and RAM. One way you can assess how much performance each component adds to the overall system is by running the Windows Experience Index bench, but in your case just playing any resource intense made for quad game would make it obvious that the CPU is the bottleneck in that build.