Any time you upgrade it is going to be slower, do a clean install, it is possible and easy to do with ugprade media.
And Vista 32-bit reported 4GB after SP2, however it was still only using 3.25GB. Microsoft added that feature to make it "less confusing" for the consumer. But I guarantee you only 3.25GB was usable.
Rig1: System Specs.
Rig2: A8-5600K@4.4GHz / AsRock FM2A75 Pro4 / 8GB Corsair DDR3-1600 9-9-9-24 / HD7560D / Samsung DVD-Burner / 1.5TB WD Green + 3x3TB WD RED in RAID5
Rig3: Athlon X2 4200+ / M4A79 Deluxe / 4GB G.Skill Pi DDR2-800 4-4-4-12 / GT430 / Sony DVD-Burner / 500GB WD
Rig4: Phenom II x6 1605T @ 3.6GHz / Asus M5A99X Evo / 8GB PNY DDR3-1600 9-9-9 / GTX470 & GTX470 / Samsung DVD-Burner / 1.5TB Seagate