Message boards :
Number crunching :
Payday
Message board moderation
Author | Message |
---|---|
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
A while ago BOINC changed the credit granting algorithm from the original BOINC credit algorithm to CreditNew. CreditNew has a fairly complex way of calculating just how much credit should be granted for every workunit. It also has the unfortunate assumption that either the workunit workload is known in advance OR said workload does not change too much from workunit to workunit, across applications and so on. BURP has crazy freakout monster workunits intermixed with tiny workunits - and even uses the same application for those WUs. We have almost no clue how long a workunit will take until it has been rendered, even within a single session the workunits may be different in length by several orders of magnitude. At first it was assumed that CreditNew would eventually settle. However, over time, it has become increasingly clear that CreditNew does not perform well for the kind of workload, applications and situation we have here at BURP - in fact the credit system has become the target of much ridicule and confusion. Payday: "Payday" is a development subproject of BURP. The main objective is to retroactively perform a correction on all granted credit since the introduction of CreditNew and subsequently replace or accompany CreditNew as the main credit granting algorithm for BURP. Secondary objectives are to make it way simpler to understand than CreditNew, make it credit-stable both in short-term and long-term measurements and at the same time make it fairly transparent to the community what is going on. It will never be perfect, but it will very likely become a bit better than CreditNew. The basic way that credit will be calculated with Payday is this: Amount of work done per hour * hours it took * cobblestone factor = your credit Oldschool BOINCers will find this very similar to the original credit system - and this is no coincidence since that system served as inspiration for Payday. Payday, as stated above, is still under development, but a large part of the algorithm framework is already in beta testing at the moment. Here's an early sample from Session 1405: ![]() The graph shows frame 0 at the left all the way to frame 315 to the right. The seemingly randomly placed red dots are the credit that was granted by CreditNew and the green dots are the workunit workload in gigaflop-hours estimated by the Payday algorithm. In a perfect world a larger workload would give more credit. You will notice that the workload increases slightly as Frank (the flying squirrel) jumps towards the camera and that the workload then decreases rapidly in the final frames when Frank and the branch are no longer in view. As you recall from above, with Payday the granted credit will be changed from the red dots to a cobblestone factor times the green dots. That particular factor has not been determined yet, but it will be set to something that somewhat matches an average BOINC project. Overall analysis matches with the example graph above and shows that there will be a general increase in credit. For some sessions CreditNew completely missed the target for some reason - those sessions will have a very large increase in granted credit. In addition to granting the right amount of credit the intention is to also grant credit to failed workunits where the volunteer had given the workunit a fair chance of succeeding (in other words workunits that fail immediately are not included here but some of the workunits that fail partway through are). BURP is very much an experimental project and hence part of the workunits can be expected to fail - in fact sometimes this is how the project improves and moves forward. Some particularly problematic sessions have already had credit manually granted for this kind of workunits, with Payday all sessions will be covered. It is important to note that no change in granted credit has been carried out yet by the Payday credit system, and it will be a while still before it will be unleashed. TL;DR: Don't worry, credit will be given where credit is due - ...eventually. It's complicated. |
zombie67 [MM] Project donor Send message Joined: 9 Dec 06 Posts: 93 Credit: 2,492,267 RAC: 649 |
|
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
Have a look at the graphs for all the sessions since introduction of CreditNew. Similarly to the graph in my previous post these graphs have the BOINC cobblestone credit as red and the Payday flop-hour estimate in green. There is no scale on either credit nor flophour graphs since the credit factor for the Payday credit system is yet to be determined. Each graph links to the session that it belongs to. There is quite a lot of graphs and the forum layout really doesn't like presenting them all in one page - the easiest way to view a single one is probably to right click it and select "View Image" or something like that. If you hover over a graph your browser will probably show the link and the ID. An interesting instance is the graph for 1490 and 1491 - for some odd reason CreditNew has granted an identical, low, amount of credit to >80% of the workunits and given the remaining ones a much higher amount. Payday shows a more consistent credit flow throughout those sessions. If you find anything peculiar, please point out which session it happened in so that the issue can be worked out. Here we go - the list has been divided into hundreds to keep it a bit more easy to navigate: 11xx sessions: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() 12xx sessions: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() 13xx sessions: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() 14xx sessions: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() 15xx sessions: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() 16xx sessions: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() TL;DR: Credit will be given - soon™ - it's still a little complicated. |
![]() Project donor Send message Joined: 11 Apr 05 Posts: 403 Credit: 2,189,214 RAC: 7 |
Looks quite interesting. While most of the PayDay graphs surely will give out more I'm surprised that there are not less of the sessions which have a lower Payday graph than what CreditNew officially gave, like 1171 and 1195 for example. Also the current 1436 - CreditNew does give out way more credits here than PayDay... interesting. This will sureley be fun when it's executed. Life is Science, and Science rules. To the universe and beyond Proud member of BOINC@Heidelberg My BOINC-Stats ![]() |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
The graphs do not have a common scale yet, they only show distribution and deviation. So in other words the green line is in a different scale as compared to the red line. What that scale is (the credit factor) is yet to be determined. If someone has a good resource on how much 1 hour of floating point operations is worth in BOINC cobblestones then please do share. Of course this is a complex question since each computer does more or less floating point operations in any given time. So far I've seen the project matrix over at BOINCStats trying to deal with this issue but it doesn't quite spell it out - it merely compares different projects to each other. |
John P. Myers Send message Joined: 10 Aug 11 Posts: 2 Credit: 53,826 RAC: 0 |
Well...to be honest i quit crunching this project because of the way credits were being handled. Credit New is a known joke and of the projects that bothered to adopt it in the first place, many have reverted back to the way their credit system used to be. It's entirely unfair and grossly inaccurate. As for me specifically, this WU is still haunting me: Workunit: 1776233 Created: 24 Oct 2012 | 0:44:40 UTC Sent: 24 Oct 2012 | 1:12:19 UTC Received: 27 Oct 2012 | 21:10:33 UTC Server state: Over Outcome: Success Client state: Done Exit status: 0 (0x0) Computer ID: 51505 Report deadline: 18 Dec 2012 | 0:05:39 UTC Run time: 295,918.32 CPU time: 547,745.17 Validate state: Valid Credit: 203.63 Are you F@#&ing kidding me! Application version: SunflowerBlender v4.81 (mt) lol, but anyway, to your question. We'll use one of my computers as the example, and this is data every project has access to. At this link: http://burp.renderfarming.net/show_host_detail.php?hostid=51509 is information on a user's computer, specifically the measured float and measured integer speeds. Using those values, you could determine the number of cobblestones required to crunch a WU based on the returned CPU time. I say only to use CPU time, even for single-threaded apps, because run time can vary when a system is doing something else besides crunching the WU. For example, when a virus scan occurs, it may require the use of the core BURP was being crunched on. This would stop the CPU time clock from ticking, but the run time clock would continue adding up, artificially inflating the results. In this way, a given WU would pay the same credits regardless of what system it was crunched on, but a faster system would complete more WUs in a period of time, earning more credits during that time than a slower CPU. This wasn't always the case with Credit New which is why most everyone felt it was unfair. It discouraged upgrading, or even crunching for BOINC altogether. Credits are the only thing we have to show our accomplishments. |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
Yes, session 1491 is one of the sessions where the credit system borked up a lot. You are probably one of the red dots around the bottom of this credit graph: ![]() You would be receiving somewhere around 4500+ credit for that one I guess. I haven't run the numbers. |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
Just while I remember to post about it: Payday will not affect "Recent Average Credit" of users, teams or hosts because the adjustments date back in time, it will only change the total amount of credit for each user, host and team. If we adopt Payday instead of CreditNew as a credit system here in the future then it will also adjust the RAC. Also, the graphs posted in this thread are now in scale. The red and green dots can be compared. The credit factor has been calculated based on the definition of a cobblestone credit unit as being 1/200th of a days work on a single-core 1 GFLOPS/GIOPS machine. Since such a machine doesn't actually exist and certainly never signed up on BURP we can't actually verify it directly - and neither does it make much sense (most computers today have much higher IOPS than FLOPS). When looking at other projects some of them deviate as much as an order of magnitude from this definition. In other words it is kinda arbitrary what other projects do, but for the purpose of Payday it works fine. A dry-run was made last weekend and it came out looking good. The next step is to hook in the code that actually updates credit for everyone and then run it. The run will take several days, starting with just a few sessions at first and then ramping up to full speed. The next Monday maintenance already has other tasks allocated for it (scaling back down to normal operations since most of the Big Buck Bunny rush is over). We're probably looking at the middle of February before it starts. |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
Payday is this week. We're starting slow. The first session to be corrected for credit was 1112. Around 66000 additional credit was given out - around 3000 credit was given to workunit instances that did not validate entirely or were originally rejected/failed for other reasons. For example this session had issues with disk parameters and unzipping on Linux clients. ![]() |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
Sessions 1114 through 1144 have been credited. Around 0.3M more credit. |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
Up to and including 1234, around 0.4M more credit. |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
To 1337 - 2.7M credit |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
1434 - 7M credit |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
1500, 12M |
zombie67 [MM] Project donor Send message Joined: 9 Dec 06 Posts: 93 Credit: 2,492,267 RAC: 649 |
|
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
To 1572, some sessions not included yet. An additional 0.5M credit |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
To, and including, 1660 - around 5M more credit |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
To 1800, around 2.5M more credit |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
To 1825 |
![]() Volunteer moderator Project administrator ![]() Send message Joined: 16 Jun 04 Posts: 4571 Credit: 2,100,463 RAC: 8 |
To 1959, around 2.5M more credit. This also concludes the first (and largest) pass of 3. The next pass is going to focus on multi-part sessions. |