Thanks! I'll try to help you as much as possible, i'm no GURU but i'll try my best to at least get ya going in the right direction
Hopefully some other's can add to the discussion.
The most important thing with OCing is to be patient, test properly & fully and read as many guide's/tutorial's as possible and of course keep temp's & volt's within spec. Unless you don't care for longevity then ball's to the wall!
There are no magic setting's. EVERY chip, board & combo is different. Only through trial/error will you achieve a stable OC
It's also a good idea to keep note's of setting's that you used and if they were stable or not. Do you have the newest BIOS for your board, if not i would highly suggest flashing to the latest one as it may increase compatibility/stability.
If i'm not mistaken you are running 4 stick's of RAM? As such it will be more difficult to OC and you'll really have to get the NB/MCh/FSB term/VTT voltage's right as you are really loading the chipset, not impossible just more difficult.
What were your temp's like with the other cooler? It look's as if you have the standard 212 which i think is not as good of a cooler as the n520 but seeing as you have both you can decide which work's better in your rig
IIRC a thing that held back those board's was the lack of MOBO cooling and as such i would reccomend getting some heatsink's for the VRM/PWR regulation circuitry (the chip's around the CPU socket) and add a fan to the NB. Swiftech/Enzotech make some nice copper sink's in various size's or you can source sink's from electronic surplus stores and customize them yourself, they are also good place's to find small server style fan's for cheap
I have used both "method's" and they have been equally as effective, just more work going the custom route.
You'll want to lock your PCIe to 100Mhz as that can cause instability if left on AUTO when upping FSB. With your volt's max Vcore i'd go with air would be around 1.35v/1.38v, i'm not exactly sure about NB/MCH volts on that board but you can start by taking them off of auto and bumping them up slowly all the while checking temp's and stability, you will have to add volt's here especially with 4 stick's of RAM.
All your changes should be small bump's all the while testing stability & temp's. If your RAM is 1066 lock it at 800, or underclock it anyways, to take it out of the equation for now untill you get the FSB stable then try bringing the RAM speed up. If you're in a "rush" to test you can try Intel burn test as it test's a lot quicker than Prime and some people say it is a better option as it REALLY stresses the OC, it will make your temp's get really high as well so it will let you know how good the cooler is doing.
Hope this help's? Be patient, read, read and then read some more! There are over 400 pages in that single thread about your specific board and i'm sure there is a wealth of info in there, and it is specific to your board. I have never OC'd a Gigabyte, never even owned one so i'm not familiar with the BIOS, if you had an Asus p45 i'd be of better assistance i guess but hey
I ended up raising my Vcore back to 1.30v as it just wasn't stable enough @ 1.28v. I stress with Prime and start with small FFT minimum 6Hrs, then large FFT minimum 6Hrs and finally Blend for 12Hrs and if i pass all those with reasonable temp's i bust out some gaming session's for "real world" testing. I seem to be doing Ok so far minus driver issues but that's a separate event