Well a front grill would help a lot, it would keep your case cooler and thats better for all your hardware. Extra giant fans would also help. Not sure it will make your overclock better by more than 100 MHz but your motherboard, videocard, ram, and other peripherals will be "happier" with a cooler ambient and more active airflow, so it seems like a good idea. The nice thing about aircooling is there are no leaks, no hassle, and even if a fan fails the system will keep running, so overall its a less stressful prospect.
There may be other ways to improve airflow without spending money. This guide might be helpful
www.overclock.net
If you want to spend a lot of money on a watercooler just to get a few hundred Mhz, I think it is a waste. I personally have gone to 5.1 on my 8600K and I cannot tell a real difference between that and 4.6 in games and just general browsing. I just picked 4.6 for my daily clock because it is the max for my stock voltage of ~1.16.
As far as delidding goes, you will see 10 to 15*C temperature reduction, but it also likely won't give you more than 100MHz because these CPU are not so much temperature limited as they are voltage limited. I also think it is a waste on an average CPU, it won't turn it into an above average CPU, it won't make it perform any better, it will just make it run slightly cooler with some real downsides.
That paltry gain comes at the expense of a lost warranty, complicated resale, and the possibility of damaging the component during modification.
Improving overall airflow however is worthwhile because it will extend the life of your components. Modern case trends are solid front bezels and suffocated motherboards.