Want to run 100% CEP2 WUs? Here's how...

[brandonwh64]

Standard setup of CEP2 Work Units.

Create a new school profile and set just as pictured below. Once complete add this profile to the device in under device manager. Once done, you can let the WU's you have already running complete or do what I did and do a full uninstall and delete of BIONC folders in program files/data then reinstall.

WARNING! WARNING!

Brandon, thank you for this, but I'm going to add some warnings:
CEP2 makes systems run hotter. On a desktop this is probably OK, but if you're running a laptop or tablet, be very careful about temps--my laptop runs 5C hotter doing CEP2 vs SN2S while the tablet does about 7C hotter.

CEP2 is brutal on your bandwidth. Each uploaded WU is about 30MB. If you have a metered connection like I know that mjkmike does, this could get expensive very quickly. Or it could interfere with other things going on at once.

Unless you set up the RAMdisk, CEP2 thrashes your disk immensely. This is very painful with 8 WUs on a 5400RPM disk

The checkpointing system in the CEP2 WUs is the worst on WCG--ie, when the WUs are removed from memory (BOINC is closed, computer is suspended and "leave applications in memory while suspended" isn't checked) you lose comparatively a lot of work.

Now, none of these are showstopping issues, but new users in particular, please be aware of the challenges presented by a solely-CEP2 diet

If you want to use the NON disk thrashing method, look below for Justin1980's How-To!

So you want to run the Clean Energy Project Phase 2 (aka CEP2) but find your disk thrashing too much? Here's how you can get around it. As shown here the Clean Energy Project needs our help. This is an idea I came up with on my own to try to help others that believe in the CEP2 project to donate more of their resources. With a little luck someone else can post a detailed screenshot guide or PDF of instructions on how to do this setup.

This guide assumes that you are fairly knowledgeable with BOINC and WCG. You also want to maximize your CEP2 WUs for your machine.
The key is using block level caching with an extremely long time delay for writing data to the drive. The intent is that the WUs will thrash the heck out of the block level cache and thereby minimize the hard drive thrashing. Of course, data will still get written to the drive, but hopefully the writes will be more sequential.
System specs for what I used: Dual CPU, Quad Core(8 thread) with 16GB of RAM and 160GB hard drive. I used Windows 7 x64 Home Premium, the latest version of BOINC(6.12.34 x64 version) along with Supercache 5.0.524.0. Supercache can be purchased for $79.95 at www.superspeed.com. A fully functional 14 day trial version is also available at their website.

1. First, you want to set up world community grid to give you CEP2 WUs. I recommend you create a device profile. For me, I created the new "school profile". Set up a custom profile and uncheck all projects except CEP2. Set up the other characteristics of the profile as you see fit. Make sure that for project specifics you choose the number of work units for CEP2. I chose unlimited since I want 100% crunching for CEP2. Don't forget to change your device to use the new profile you created. I also set the profile to cache only 0.25 days of work because I want to cache the projects in use without caching files that are not.

Josh1980: The machine I tested this on has 8 cores(2 CPUs) and I found that I could achieve 100% utilization using just 7 of the 8 cores. The machine also handles other low CPU usage programs for me, so I can only expect that the caching program I used utilized the 8th core. Feel free to experiment and figure out how many cores work best for you.

2. Now that WCG is setup to give you all the CEP2 WUs you want it's time to setup your computer to receive these units. Install Supercache 5. After rebooting if necessary it is time to setup a partition for your BOINC data. I resized my C partition to make about 40GB of space available for BOINC. The partition for BOINC needs to be dedicated to BOINC data(you don't want other data being cached) and it needs to be big enough to not end up filling up. CEP2 uses alot more drive space than other projects. My opinion based on using a RAM drive previously is to create a partition between 2 and 4 times the number of threads you will create in GB. If you intend to run 16 threads simultaneously, I would recommend between 32 and 64GB for the BOINC partition. I'd recommend 32GB because you don't want to partition too big for the cache. Format the partition to 4k sector size NTFS.

3. Next I installed BOINC, pointing the data directory to the E drive. I do not know how you can change the data directory if BOINC is already installed. If someone posts this information I will update this post. After BOINC is installed reboot if necessary.

4. Next it is time to setup the cache. Install supercache and on installation when it asks you to setup the cache for the windows partition uncheck the box.
I determined that I wanted 80% of my RAM utilized. I've seen CEP2 WUs use as much as 300MB per thread. Since I was expecting 8 threads I figured 3GB would be utilized at the most for CEP2. The WUs use more RAM as the WUs nears completion, typically starting at 60-80MB.

Josh1980: So I setup a cache of 5GB to see how it would perform. The size of the cache can play a part in how well this performs. I setup the granularity as 4kb and set the total allocation of 5GB. I also enabled the deferred-Write Mode and set the latency to infinite. This means that the cache shouldn't try to write data to the drive until the cache is full, a manual flush command is initiated, or something else triggers it by supercache?
For a 5GB cache I found that 2GB of data was written about every 10 minutes. The hard drive would sit idle for about 10 minutes, then for about 30 seconds it would write data at 20-70MB/sec(total was typically 1-2GB). Then go idle again for approximately 10 minutes. For a 10GB cache I got about 20 minutes of idle time.

5. Now that the cache is setup, it is time to download those WUs. Using BOINC, I added the WCG project and began downloading WUs. I ended up with all CEP2 WUs. The computer then ramped up to 100% CPU utilization as the WUs were downloaded. Watching the temperatures, CPU usage, and cache I was able to determine that the cache that I setup worked very well for this project. Even during the times of hard drive activity the CPU stayed at 100% utilization and the CPU temps showed that the CPU was still being stressed by the work.


If all went well, you should see similar results to what I saw. CPU will sit at 100% utilization for about 10 minutes+, then the hard drive will write a bunch of data, then idle again.

If you use a SSD, you MAY find that the total amount of data written to the SSD will be lower using the supercache.

So why does all this stuff work? I believe that the CEP2 project continually concatenates the files on the hard drive with new data as the CEP2 WUs are completed. Because we have a large block cache we are writing all this data that is continually updated to the cache instead of thrashing the hard drive. The built-in Windows cache has a finite time before data is written to the hard drive. Being able to set up a cache that theoretically, might not write to the hard drive for a very very long time is a big bonus for us.


Special thanks to Josh1980 over at XtremeSystems

http://www.xtremesystems.org/forums/showthread.php?277686-Want-to-run-100-CEP2-WUs-Here-s-how...

 

[Norton]

Sub'd

Thanks for the link Brandon

iirc there are a couple of other methods to fill the rig with CEP2 work but I forgot them atm

 

[brandonwh64]

I did everything in the OP except that superspeed program. I currently have 8 WU of CEP2 but load is not at 100% as of yet but they all are running.

*edit*

its at 100% load now after all WU downloaded. Completion time is like 13 hours LOL

 

[Chicken Patty]

Thanks for the info

 

[adulaamin]

Thank you very much for this!

 

[Arjai]

After reading the post on xtreme, the link above, I decided to try a RamDisk cache for CEP2.

I have the ramdisk set to save every twelve minutes, made it 3GB and set it to save image on shutdown.

In order to use the RamDisk, BOINC needs to be uninstalled and re-installed, in order to point the cache to the new drive, on my system-F:. So, I set BOINC to No New Tasks and will let it run through the next 3.5-4 days of stuff, force an update and then uninstall, etc.

I also set up a new Device profile for just CEP2, to be used when I reinstall.

I have no idea how fast four CEP2 WU's will fill up my 3GB ramDisk. I am simply guessing that 12 minutes will be enough to NOT fill it. I may adjust it if the drive saves are very small, based on the drive light and Task Manager graphs.

I will keep you all posted on how this works. However, as I mentioned, I will let the BOINC cache run through what it has, about 4 days.

So, sometime on the tenth, or 11th, I will post how it is working. If anybody else is/ has doing/done this, I would appreciate your input!

Thanks

 

[Arjai]

OK, while sleeping on it, after dinner nap, I have a few questions.

RamDisk is set to save to hdd, in it's own folder, every twelve minutes ( an estimate based on some loose math from the post in xtreme.com ).

How dynamic is that 'Image' gonna be? Do I even need it to be?

Here's the deal. Currently, I am allowing BOINC 3.5 GB and it is using exactly half, 1.75 GB.
The RamDisk will be, is, 3GB.

So, the current two CEP2 Wu's running are writing to disk about twelve times a minute. I am assuming, it is writing and erasing, or replacing, or writing over, itself or the 1.75 GB would be constantly growing. No?

If this is the case, I could actually just leave the RamDisk running and simply back it up when the Laptop sleeps, or is shut off.

But, I don't think that is it. Because dude over at xtreme says his cache, of 5GB would fill up and auto dump to disk. Obviously it was filling up.

So, from what I can gather, His 'SmartCache' saved to the BOINC allocation space on his HDD/SSD. RamDisk saves an 'Image' to my C:/ drive and then reloads it when I re-start.

Problem is...This 'Image' file will be writing over itself every twelve minutes. So, are each of those updates, including the shutdown 'Image', gonna be what BOINC needs, when the job reports or, will the job even be able to keep running without an error?

So, yea. I don't EVEN know who to ask THAT question! Considering how specific it is to this one type of WU on BOINC running WCG using a RamDisk.

Either one of you fine people have been there, or I jump in feet first and see what happens!!

 

[[Ion]]

Brandon, thank you for this, but I'm going to add some warnings:
CEP2 makes systems run hotter. On a desktop this is probably OK, but if you're running a laptop or tablet, be very careful about temps--my laptop runs 5C hotter doing CEP2 vs SN2S while the tablet does about 7C hotter.

CEP2 is brutal on your bandwidth. Each uploaded WU is about 30MB. If you have a metered connection like I know that mjkmike does, this could get expensive very quickly. Or it could interfere with other things going on at once.

Unless you set up the RAMdisk, CEP2 thrashes your disk immensely. This is very painful with 8 WUs on a 5400RPM disk

The checkpointing system in the CEP2 WUs is the worst on WCG--ie, when the WUs are removed from memory (BOINC is closed, computer is suspended and "leave applications in memory while suspended" isn't checked) you lose comparatively a lot of work.

Now, none of these are showstopping issues, but new users in particular, please be aware of the challenges presented by a solely-CEP2 diet

 

[Chicken Patty]

Brandon, thank you for this, but I'm going to add some warnings:
CEP2 makes systems run hotter. On a desktop this is probably OK, but if you're running a laptop or tablet, be very careful about temps--my laptop runs 5C hotter doing CEP2 vs SN2S while the tablet does about 7C hotter.

CEP2 is brutal on your bandwidth. Each uploaded WU is about 30MB. If you have a metered connection like I know that mjkmike does, this could get expensive very quickly. Or it could interfere with other things going on at once.

Unless you set up the RAMdisk, CEP2 thrashes your disk immensely. This is very painful with 8 WUs on a 5400RPM disk

The checkpointing system in the CEP2 WUs is the worst on WCG--ie, when the WUs are removed from memory (BOINC is closed, computer is suspended and "leave applications in memory while suspended" isn't checked) you lose comparatively a lot of work.

Now, none of these are showstopping issues, but new users in particular, please be aware of the challenges presented by a solely-CEP2 diet

If you guys want I can place this in the OP or Brandon can. Up to you guys but it's definitely good info and I think everybody should view. I PERSONALLY did not know this.

 

[brandonwh64]

If you guys want I can place this in the OP or Brandon can. Up to you guys but it's definitely good info and I think everybody should view. I PERSONALLY did not know this.

I added it to the OP. I have noticed a 3 degree increase but I thought it was cause of my crappy H50

 

[Chicken Patty]

I added it to the OP. I have noticed a 3 degree increase but I thought it was cause of my crappy H50

Well now we know why Good info for sure!

Thanks ION!

 

[Arjai]

RamDisk experiment...

So, it begins.

I have formatted a 4GB partition, G:, in 4k allocations. I have pointed RamDisk to save it's 3GB cache every 12 minutes to G:.

I uninstalled BOINC, Reinstalled updated WCG to my new, CEP2, School profile. Set it all up, just now, and when it finishes downloading and starts running...

Well, in less than a half an hour, I should know if this is working.

Wish me luck!

Update1.

Well. I allocated 6 more GB to the BOINC partition. However, I think I may have a problem. The jobs, all two, were running and the RamDisk was saving to the new partition. Then, my memory use shot up, the HDD went full on and BOINC sent me a message about needing more space. So, herein lies what may be the killer of this RamDisk experiment, I can only allocate 3GB of my 8GB and still function as computer, not just a cruncher. I need both right now. That 3GB is the size of the image file it saves on the new BOINC partition. It cannot grow any larger, like I need it to do. So, even though that 3GB file is working dynamically to new info being inserted, as CEP2 is needing to do, the 3GB limitation of the reused saved image seems to be the killer.

I gonna give it another run, right now. If I get the same BOINC message, as I suspect, It is a dead experiment, due to physical RAM limitations. Because I don't know how to have RAMDisk use more of the 10GB than it's own 3GB physical size, as a save.

Update 2.
Well, I think that it is not gonna work, I actually am now SURE, since BOINC won't download any WU's now.
So, I am going to go and change BOINC, uninstall and reinstall, so it just writes to the new partition. Dump the RAMDisk. Then, in a couple days try the SuperSpeed cache using the 14 day trial. If I can make that work, I expect it to be better, I will probably download it next paycheck. Since it will be UBER useful later when I upgrade to a SSD, also.

So, there it is. A failure due to lack of Physical RAM available to me on this Laptop. Oh well, a learning experience, as well. Hopefully, a this can be useful for someone else, also. RamDisk from DataRam can work if you can set up a RamDisk that is large enough, say 6 to 10 GB, for a CEP2 cache of around 4 to 6 WU's at a time. 3GB would work, I think, if you just ran one WU at a time. That is not what I am aiming to do so, I am not going to test that, since I want to keep crunching and limit the time of experimenting. In theory though, based on this test, 1 WU would run and complete.

Anyways, like I mentioned earlier, I will try the superspeed cache, later, and update then how it works for me.

Arjai, out!

 

[sabre23]

Brandon, thank you for this, but I'm going to add some warnings:
CEP2 makes systems run hotter. On a desktop this is probably OK, but if you're running a laptop or tablet, be very careful about temps--my laptop runs 5C hotter doing CEP2 vs SN2S while the tablet does about 7C hotter.

CEP2 is brutal on your bandwidth. Each uploaded WU is about 30MB. If you have a metered connection like I know that mjkmike does, this could get expensive very quickly. Or it could interfere with other things going on at once.

Unless you set up the RAMdisk, CEP2 thrashes your disk immensely. This is very painful with 8 WUs on a 5400RPM disk

The checkpointing system in the CEP2 WUs is the worst on WCG--ie, when the WUs are removed from memory (BOINC is closed, computer is suspended and "leave applications in memory while suspended" isn't checked) you lose comparatively a lot of work.

Now, none of these are showstopping issues, but new users in particular, please be aware of the challenges presented by a solely-CEP2 diet

Will my WD Green will survive disk thrashing as it is only got 2 yrs warranty? i dont want to destroy it. I am thinking of running only 2-3 CEP at a time.
Will WD Black or intel SSD 520 will survive thrashing better?

 

[[Ion]]

Will my WD Green will survive disk thrashing as it is only got 2 yrs warranty? i dont want to destroy it. I am thinking of running only 2-3 CEP at a time.
Will WD Black or intel SSD 520 will survive thrashing better?

A standard hard drive will be totally fine. The "thrashing" impacts day-to-day performance on slow drives more than reliability of the drive itself. The SSD should be fine, too--it's a modern drive and I'm confident that Intel has put work into making the flash chips better stand up to repeated writes.

 

[manofthem]

I still haven't changed over yet but I need to do this soon. All this talk about CEP2 being much harder to run has made my enthusiasm wane a bit; I'm expecting issues tbh. But thanks brandonwh64 and [Ion]

 

[Norton]

I still haven't changed over yet but I need to do this soon. All this talk about CEP2 being much harder to run has out made my enthusiasm wane a bit; I'm expected issues tbh. But thanks brandonwh64 and [Ion]

I wouldn't worry about it too much- they run fine on nearly all of my rigs

The file size of the upload (over 45MB) is a bit of an issue in one location due to having a small internet connection.

Try just increasing the # of work units you can run to 8 with no other changes and see how you do

 

[torgoth]

So I've created the profile, how do I set it in the BOINC Manager?
edit: nevermind I figured it out, hope it works!

 

[Mindweaver]

Nice thread brandon! Good job!

 

[TRWOV]

For the record, CEP2 can use as much as 400MB of RAM per WU. They usually hover below the 300MB mark but from time to time I get WUs that take >380MB. I have disabled CEP2 on the X4 620 for this reason (it runs W7 Starter).

 

[Peter1986C]

Thanks for the note TRWOV.

 

[manofthem]

So I change my main rig over to CEP2 now. After downloading the WUs and after they started running, the CPU stayed around 10-30% for a bit, and it just finally shot up to 99%, where is seems to be staying now.

Does that sound alright?

 

[brandonwh64]

So I change my main rig over to CEP2 now. After downloading the WUs and after they started running, the CPU stayed around 10-30% for a bit, and it just finally shot up to 99%, where is seems to be staying now.

Does that sound alright?

Yep sounds just like mine

 

[manofthem]

Yep sounds just like mine

Thanks, good to know. And thanks for the OP guide, followed it as closely as possible, other than the one program
I hope it continues to play nicely

 

[[Ion]]

So I change my main rig over to CEP2 now. After downloading the WUs and after they started running, the CPU stayed around 10-30% for a bit, and it just finally shot up to 99%, where is seems to be staying now.

Does that sound alright?

Yup, one of the quirks is that it has to load a bunch of stuff from disk and spool up to speed--I suspect that if you had looked at the HDD light, the drive would have been very busy

 

[manofthem]

Yup, one of the quirks is that it has to load a bunch of stuff from disk and spool up to speed--I suspect that if you had looked at the HDD light, the drive would have been very busy

Yeah figured as such. Since it all smoothed out and went to 100%, I'm thinking it should be fine.

Also, since I read about a lot of disk trashing of whatevs, I installed to the HDD, the programfiles and programdata folders, so hopefully it'll do less on the ssd, as I'd hate to loose performance on my 830, as it is getting older now. Maybe I'll give it a test after the CEP2 Challenges