FAST team stats... announcemnt

[cine.chris]

Announcement, but not like an Nvidia or AMD announcement.
It's actually working, currently tweaking the pieces to get the pages updated on the webserver
Sample from yesterday's EOD end-of-day:
TPU sample page
Rank shown is your True Active Project Rank, with abandons removed.
Note: this is different from Production Rank which is usually a 7 day average of point production.
I'll post an update with the live link soon.
The current project stats, with the new NEW FAH servers, are updating hourly.
So, expect some glitches with all the new moving pieces.
Yes, it needs a team index.
Anyway, you guys are the first to see it!

 

[phill]

Great work there @cine.chris !!

 

[cine.chris]

Let me know if you are seeing updates, ~ on the hour...

Team Data

 

[Jacky_BEL]

Seems to work OK.

 

[cine.chris]

Seems to work OK.

Yes, except, adding to their other server problems their downloads/timestamps are now in a constant state of drift & appear to not be linked to a cron process.
Like yesterday, their update drifted past the final update.
My post on the Folding Forums got no responses so, I guess that nobody is concerned with the science of stats... just folding proteins.
I've now shifted my cron task 25 minutes since starting this project and it will cross the hour, which forced me to shuffle the zero hour.
My only option is to update every 15 minutes until this is fixed.
Fortunately, my process is efficient, running in ~2min, which considering that you are dealing with two sets of 2.5M records is impressive.
The credit goes to Python & Pandas, not my crude skill set.
Anyway... the page is beautiful & fast on a phone. No tracking, no ads, no click-bait swill, no graphics, just pure HTML & Jquery.

Rev2 image from my phone, which will be live soon!

 

[Jacky_BEL]

I have noticed on the https://apps.foldingathome.org/cpu recent CPU page that reporting the "last returned" seems to happen every 10 minutes, at the 06 16 26 36 46 and 56th minute.

 

[cine.chris]

The FAH updates have stalled... Uggh.
They could be returning to a 3hr update rate or just broken, I'll have to wait & see.
Lot's of problems with their infrastructure lately...

 

[Jacky_BEL]

Yes, some of my machines have errors in the log file, trying to contact the collection server.
All machines are now running though.
But the F@H recent CPU page has stopped updating also.

Timestamp on the last F@h Fast Teampage URL is Sat Feb 27 04:04:26 GMT 2021
and on my F@h donor page it is 2021-02-27 04:47:21

I hope maybe the DC-stats page now gets a reset somehow too

Results are available again.

 

[phill]

It seems that Extreme and Free DC had an issue today.. Normally it's just Free-DC... That seems to never work 100% well for FAH... WCG and Rosetta seems to work ok tho.... FAH not even close

 

[Jacky_BEL]

The app from @cine.chris seems to suffer the same error as DC-stats right now with exagerated numbers.
Maybe the reporting timing issue as Chris stated.

Anyway, I guess it is no simple task to track all those folding results.

 

[cine.chris]

That's what it should do with no EOD end-of-day stats for the prior day. (by-design)
So, it shows the two days combined in this case.
Will Jacky_BEL make 30Million ???
I also set the EOD an hour earlier, which is US EST.
As, EOD is a synthetic construct anyway. I've always thought it should be GMT.
When everything is working correctly, there will be a collection of Team EOD files at:
http://fahtech.com/EOD/ , but not working yet.

 

[phill]

I'd consider using the Extreme Forum one, I think that one actually does update properly for the day and I've not noticed it being down for more than a day but that does tend to have a lot more accurate information, it just doesn't show milestones sadly....

 

[cine.chris]

Folding data all comes from the same source.
With the shifting download update time, now moving at 3.6 min/day. There is a likely hood that people could be pulling different data. As they only provide cumulative totals, not hourly or daily, it's the same data everywhere. The key variable now is how the floating update time is managed & tracked. My tract is to follow that point and update data when the data changes and not follow the clock, as the next update could be 1 min away & missed.
Of course, one hour slightly off is still much better than every 3 hrs.
My current validation is to make sure that the end/start data between days matches & no data is lost in the first hour stats.
Again, for everyone, the final data is what Folding says it is. It's all the same data.... if downloaded at the same time.
That's one reason I'll be logging the EOD stats. Right now, everything is essentially a chalkboard and erased in a few days.
Currently, I have published RANK data that goes back to 20190101.

Index of /rank

Try & find data that really describes the behavior of the Folding Project over the years... it doesn't exist.
Currently, I have data back to Mid-2013, so more historical perspectives are what I want to provide and a framework to understand how group projects like this behave.

 

[Jacky_BEL]

Now it all starts to make more sense to me.
EOC seem to have found a good workaround, but it only updates every 3 hours.

I have bookmarked the app in my browser

 

[cine.chris]

With the restart, the stats are still drifting about 3.6 min/day.
I'll be trailing the drift.
The updates will cross the hour point again sometime tomorrow if there is no reset.
So, that last update will be just past the EOD... it'll actually be the 00-hour for the next day.

 

[Jacky_BEL]

What is the meaning of time-slot:16 and Next:02 ?

 

[cine.chris]

16th-hour updated at 2 minutes
2 is the minutes from the time when python wrote the file.
And a likely guess that the next update will occur at 2 minutes, which is a safe guess for ~4 days.
If I don't get the tracker working it skips 15 minutes.

I need to finish the tracker as the updates will cross the hour soon! (see timestamp log below)
Those BASH shell script if's and arithmetic take a lot of testing and validation as I've worked with them much.
And I guess I'm a clumsy programmer.
Getting the opportunity to test these outlying conditions is hard to find or waiting another 4-5 days in this case.

20210228-170001
Sun Feb 28 21:55:59 GMT 2021
TimeStamp Trigger: PROCESSING

 

[Jacky_BEL]

Thank you for taking the time to explain.
It's been a long time since I did some programming. I think you are doing a great job here!

 

[cine.chris]

@Jacky_BEL Thanks.
Also, Time-Slot, I wanted to separate the updates from a clock-based event as they are continuing to drift.
Just a mental cue to expect update events to be fluid.

@Jacky_BEL Looking at #'s.
I show a later total update from Folding, but the daily base has shifted & daily #'s are too low.
Need to find that error.
It was correct at hour zero.

I checked the numbers for three donors, by hand, based on the GMT 5:53 data file, the EOD for St Louis CST GMT-6.
File Header: Sun Feb 28 05:53:19 GMT 2021
and my web#s were dead-on exact for daily and the current totals were a later update than showing on EOC,
I was glancing at the 24/7data on EOC, thinking they didn't match.
EOC is likely still on a 3Hr update cycle and would have to be compared after the 3 hr update which takes ~20min.

@XZero450 must've found some additional gear, I have my edit station running today and he still passed me.

Top300

FAH RANK Data

I need to change the colors and rank scaling for this gang & add team

 

[Jacky_BEL]

Okay,
So 1hr. and 3hr. updates are in fact seperate updates then?

 

[cine.chris]

Okay,
So 1hr. and 3hr. updates are in fact seperate updates then?

Same data, just the time, between when the updates occur.
My understanding of updates...
Historically, it had been hourly, at the end of the hour.
Apparently, during the Covid work surge, FAH went to a 3hr update cycle.
Then recently, it returned to an almost hourly update rate.

@Jacky_BEL
The update tracker is working.
An amazing <5min from timestamp to page publish.
About 1-2 min is lost in the update actually being available for download, based on data I reviewed this morning.
Based on that, I added a 3min buffer to the tracker...
so, it can't get much faster.
It also transitioned the hour rollover without issue.
This scheduling is tight, I'm expecting some hiccups.

 

[cine.chris]

20210304-220001
sleeping 16 minutes
Fri Mar 05 03:13:51 GMT 2021 <== time when this line is written, 1st of 2.5million records that gets compressed, put online
TimeStamp Trigger: PROCESSING
---- 221747 ---- <========== time after the last page was transferred to web server (Dreamhost, Virginia, USA)
An impressive sub 4min update time. Timestamp to Page Publish on the website.
Update File TimeStamp: 13:51
Page Publish completion: 17:47

 

[cine.chris]

The FAH STATS have stalled... again.
Numbers do not reset as there wasn't an EOD (end-of-day) points update.
========================
STATS feed has resumed...

Also, missing data from two servers has been added to the database, so you might get an extra points bump too.
This is also was the reason the updates were stalled.

 

[Jacky_BEL]

The numbers are showing in the EOC , DC and your stats, but the Donor stats Folding@home stats report and Folding@home Recent CPUs are no longer updated.
They are still stuck at 2021-03-06 21:35:45 for me.

 

[cine.chris]

They're working on replacing the old stat pages.
I only know what I've seen mentioned on Discord or in the Forums.
I got the BASH date code working yesterday, turns out it's simple & powerful. I'd never had to do much shell scripting in the past.
So... stall detection, update timing, will be much easier, more reliable.
@Jacky_BEL
Found a link for you:

Folding@home | Statistics

Folding@home Statistics

statsbeta.foldingathome.org

There is an OS tab there...