XML stats dumped TOO often?

Message boards : Number crunching : XML stats dumped TOO often?
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile [B^S] Zain Upton
Avatar

Send message
Joined: 2 Mar 05
Posts: 8
Credit: 694
RAC: 0
Message 349 - Posted: 31 Mar 2005, 8:48:41 UTC

Hey Janus and crew,

First thanks for all the continued hard work... its great to see BURP developing.

Currently you have the BURP XML dump set to hourly... I dont know if that is what your after but it just may be TOO frequently.

Most stats developers scripts check the timestamp hourly... so it means the XML is also being downloaded hourly. With the small XML this is fine... but for the future may be hazourdous.

Anyway... just wanted to point it out.

Zain
---

<img src='http://www.boincsynergy.com/images/stats/comb-62.jpg'><br>Teamless? Drop by <a href='http://www.boincsynergy.com'>BOINC Synergy</a> and look around. Also checkout our famous <a href='http://www.boincsynergy.com/stats'>BOINC World Stats</a>
ID: 349 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 351 - Posted: 31 Mar 2005, 13:57:50 UTC - in response to Message 349.  

Dump frequencies will be adjusted along the run. The plan is to dump as frequently as possible without disturbing the general performance of website and database servers.
ID: 351 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 352 - Posted: 31 Mar 2005, 16:55:07 UTC - in response to Message 351.  

... and since the servers are pretty powerfull I'm really only worried about network usage - which can be overcome by using the same mirroring system as for other downloads.

Actually I would much prefer stats sites to gather the info from XML rather than scraping it from the HTML pages - since scraping loads the systems more heavily and uses more bandwidth.
ID: 352 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : XML stats dumped TOO often?