Payday


Advanced search

Message boards : Number crunching : Payday

1 · 2 · Next
Author Message
Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 11728 - Posted: 5 Apr 2013, 19:38:41 UTC
Last modified: 5 Apr 2013, 20:44:27 UTC

A while ago BOINC changed the credit granting algorithm from the original BOINC credit algorithm to CreditNew. CreditNew has a fairly complex way of calculating just how much credit should be granted for every workunit. It also has the unfortunate assumption that either the workunit workload is known in advance OR said workload does not change too much from workunit to workunit, across applications and so on.
BURP has crazy freakout monster workunits intermixed with tiny workunits - and even uses the same application for those WUs. We have almost no clue how long a workunit will take until it has been rendered, even within a single session the workunits may be different in length by several orders of magnitude.
At first it was assumed that CreditNew would eventually settle. However, over time, it has become increasingly clear that CreditNew does not perform well for the kind of workload, applications and situation we have here at BURP - in fact the credit system has become the target of much ridicule and confusion.

Payday:

"Payday" is a development subproject of BURP. The main objective is to retroactively perform a correction on all granted credit since the introduction of CreditNew and subsequently replace or accompany CreditNew as the main credit granting algorithm for BURP. Secondary objectives are to make it way simpler to understand than CreditNew, make it credit-stable both in short-term and long-term measurements and at the same time make it fairly transparent to the community what is going on.

It will never be perfect, but it will very likely become a bit better than CreditNew.

The basic way that credit will be calculated with Payday is this:
Amount of work done per hour * hours it took * cobblestone factor = your credit
Oldschool BOINCers will find this very similar to the original credit system - and this is no coincidence since that system served as inspiration for Payday.

Payday, as stated above, is still under development, but a large part of the algorithm framework is already in beta testing at the moment. Here's an early sample from Session 1405:

The graph shows frame 0 at the left all the way to frame 315 to the right. The seemingly randomly placed red dots are the credit that was granted by CreditNew and the green dots are the workunit workload in gigaflop-hours estimated by the Payday algorithm. In a perfect world a larger workload would give more credit.
You will notice that the workload increases slightly as Frank (the flying squirrel) jumps towards the camera and that the workload then decreases rapidly in the final frames when Frank and the branch are no longer in view.

As you recall from above, with Payday the granted credit will be changed from the red dots to a cobblestone factor times the green dots. That particular factor has not been determined yet, but it will be set to something that somewhat matches an average BOINC project.

Overall analysis matches with the example graph above and shows that there will be a general increase in credit. For some sessions CreditNew completely missed the target for some reason - those sessions will have a very large increase in granted credit.

In addition to granting the right amount of credit the intention is to also grant credit to failed workunits where the volunteer had given the workunit a fair chance of succeeding (in other words workunits that fail immediately are not included here but some of the workunits that fail partway through are). BURP is very much an experimental project and hence part of the workunits can be expected to fail - in fact sometimes this is how the project improves and moves forward. Some particularly problematic sessions have already had credit manually granted for this kind of workunits, with Payday all sessions will be covered.

It is important to note that no change in granted credit has been carried out yet by the Payday credit system, and it will be a while still before it will be unleashed.

TL;DR:
Don't worry, credit will be given where credit is due - ...eventually. It's complicated.

zombie67 [MM]
Project donor
Avatar
Send message
Joined: 9 Dec 06
Posts: 90
Credit: 2,118,099
RAC: 0
Message 11777 - Posted: 18 Apr 2013, 2:38:42 UTC - in response to Message 11728.

Awesome!
____________
Dublin, California
Team: SETI.USA

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 11912 - Posted: 29 Jun 2013, 16:44:42 UTC - in response to Message 11728.
Last modified: 29 Jun 2013, 17:05:57 UTC

Have a look at the graphs for all the sessions since introduction of CreditNew. Similarly to the graph in my previous post these graphs have the BOINC cobblestone credit as red and the Payday flop-hour estimate in green. There is no scale on either credit nor flophour graphs since the credit factor for the Payday credit system is yet to be determined.

Each graph links to the session that it belongs to. There is quite a lot of graphs and the forum layout really doesn't like presenting them all in one page - the easiest way to view a single one is probably to right click it and select "View Image" or something like that. If you hover over a graph your browser will probably show the link and the ID.

An interesting instance is the graph for 1490 and 1491 - for some odd reason CreditNew has granted an identical, low, amount of credit to >80% of the workunits and given the remaining ones a much higher amount. Payday shows a more consistent credit flow throughout those sessions.

If you find anything peculiar, please point out which session it happened in so that the issue can be worked out.

Here we go - the list has been divided into hundreds to keep it a bit more easy to navigate:

11xx sessions:


12xx sessions:


13xx sessions:


14xx sessions:


15xx sessions:


16xx sessions:


TL;DR:
Credit will be given - soon™ - it's still a little complicated.

Profile DoctorNow
Project donor
Avatar
Send message
Joined: 11 Apr 05
Posts: 392
Credit: 2,168,338
RAC: 25
Message 11913 - Posted: 29 Jun 2013, 18:23:49 UTC

Looks quite interesting. While most of the PayDay graphs surely will give out more I'm surprised that there are not less of the sessions which have a lower Payday graph than what CreditNew officially gave, like 1171 and 1195 for example.
Also the current 1436 - CreditNew does give out way more credits here than PayDay... interesting.
This will sureley be fun when it's executed.
____________
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 11914 - Posted: 29 Jun 2013, 22:03:13 UTC - in response to Message 11913.
Last modified: 29 Jun 2013, 22:28:47 UTC

The graphs do not have a common scale yet, they only show distribution and deviation. So in other words the green line is in a different scale as compared to the red line. What that scale is (the credit factor) is yet to be determined.

If someone has a good resource on how much 1 hour of floating point operations is worth in BOINC cobblestones then please do share. Of course this is a complex question since each computer does more or less floating point operations in any given time.
So far I've seen the project matrix over at BOINCStats trying to deal with this issue but it doesn't quite spell it out - it merely compares different projects to each other.

John P. Myers
Send message
Joined: 10 Aug 11
Posts: 2
Credit: 53,826
RAC: 0
Message 11953 - Posted: 29 Jul 2013, 8:02:31 UTC - in response to Message 11914.

Well...to be honest i quit crunching this project because of the way credits were being handled. Credit New is a known joke and of the projects that bothered to adopt it in the first place, many have reverted back to the way their credit system used to be. It's entirely unfair and grossly inaccurate.

As for me specifically, this WU is still haunting me:
Workunit: 1776233
Created: 24 Oct 2012 | 0:44:40 UTC
Sent: 24 Oct 2012 | 1:12:19 UTC
Received: 27 Oct 2012 | 21:10:33 UTC
Server state: Over
Outcome: Success
Client state: Done
Exit status: 0 (0x0)
Computer ID: 51505
Report deadline: 18 Dec 2012 | 0:05:39 UTC
Run time: 295,918.32
CPU time: 547,745.17
Validate state: Valid
Credit: 203.63 Are you F@#&ing kidding me!
Application version: SunflowerBlender v4.81 (mt)


lol, but anyway, to your question.

We'll use one of my computers as the example, and this is data every project has access to.

At this link: http://burp.renderfarming.net/show_host_detail.php?hostid=51509 is information on a user's computer, specifically the measured float and measured integer speeds. Using those values, you could determine the number of cobblestones required to crunch a WU based on the returned CPU time.

I say only to use CPU time, even for single-threaded apps, because run time can vary when a system is doing something else besides crunching the WU. For example, when a virus scan occurs, it may require the use of the core BURP was being crunched on. This would stop the CPU time clock from ticking, but the run time clock would continue adding up, artificially inflating the results.

In this way, a given WU would pay the same credits regardless of what system it was crunched on, but a faster system would complete more WUs in a period of time, earning more credits during that time than a slower CPU. This wasn't always the case with Credit New which is why most everyone felt it was unfair. It discouraged upgrading, or even crunching for BOINC altogether. Credits are the only thing we have to show our accomplishments.

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12274 - Posted: 19 Jan 2014, 15:32:43 UTC - in response to Message 11953.
Last modified: 19 Jan 2014, 16:41:13 UTC

Yes, session 1491 is one of the sessions where the credit system borked up a lot. You are probably one of the red dots around the bottom of this credit graph:

You would be receiving somewhere around 4500+ credit for that one I guess. I haven't run the numbers.

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12326 - Posted: 29 Jan 2014, 19:10:50 UTC
Last modified: 29 Jan 2014, 19:51:20 UTC

Just while I remember to post about it:
Payday will not affect "Recent Average Credit" of users, teams or hosts because the adjustments date back in time, it will only change the total amount of credit for each user, host and team. If we adopt Payday instead of CreditNew as a credit system here in the future then it will also adjust the RAC.

Also, the graphs posted in this thread are now in scale. The red and green dots can be compared. The credit factor has been calculated based on the definition of a cobblestone credit unit as being 1/200th of a days work on a single-core 1 GFLOPS/GIOPS machine. Since such a machine doesn't actually exist and certainly never signed up on BURP we can't actually verify it directly - and neither does it make much sense (most computers today have much higher IOPS than FLOPS). When looking at other projects some of them deviate as much as an order of magnitude from this definition. In other words it is kinda arbitrary what other projects do, but for the purpose of Payday it works fine.

A dry-run was made last weekend and it came out looking good. The next step is to hook in the code that actually updates credit for everyone and then run it.
The run will take several days, starting with just a few sessions at first and then ramping up to full speed.

The next Monday maintenance already has other tasks allocated for it (scaling back down to normal operations since most of the Big Buck Bunny rush is over). We're probably looking at the middle of February before it starts.

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12348 - Posted: 9 Feb 2014, 21:34:51 UTC - in response to Message 12326.
Last modified: 9 Feb 2014, 22:00:40 UTC

Payday is this week.

We're starting slow. The first session to be corrected for credit was 1112. Around 66000 additional credit was given out - around 3000 credit was given to workunit instances that did not validate entirely or were originally rejected/failed for other reasons. For example this session had issues with disk parameters and unzipping on Linux clients.

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12352 - Posted: 10 Feb 2014, 18:17:05 UTC - in response to Message 12348.
Last modified: 10 Feb 2014, 18:21:05 UTC

Sessions 1114 through 1144 have been credited. Around 0.3M more credit.

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12353 - Posted: 11 Feb 2014, 20:29:42 UTC - in response to Message 12352.

Up to and including 1234, around 0.4M more credit.

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12354 - Posted: 12 Feb 2014, 18:36:18 UTC - in response to Message 12353.

To 1337 - 2.7M credit

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12355 - Posted: 13 Feb 2014, 18:54:21 UTC - in response to Message 12354.

1434 - 7M credit

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12356 - Posted: 16 Feb 2014, 12:04:02 UTC - in response to Message 12355.

1500, 12M

zombie67 [MM]
Project donor
Avatar
Send message
Joined: 9 Dec 06
Posts: 90
Credit: 2,118,099
RAC: 0
Message 12360 - Posted: 17 Feb 2014, 13:54:20 UTC

This is great! Thanks again!
____________
Dublin, California
Team: SETI.USA

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12361 - Posted: 17 Feb 2014, 20:07:30 UTC - in response to Message 12360.

To 1572, some sessions not included yet. An additional 0.5M credit

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12362 - Posted: 18 Feb 2014, 20:01:59 UTC - in response to Message 12361.

To, and including, 1660 - around 5M more credit

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12367 - Posted: 19 Feb 2014, 20:55:35 UTC - in response to Message 12365.

To 1800, around 2.5M more credit

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12372 - Posted: 20 Feb 2014, 20:53:16 UTC - in response to Message 12367.

To 1825

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4483
Credit: 2,094,806
RAC: 0
Message 12378 - Posted: 22 Feb 2014, 12:34:37 UTC
Last modified: 22 Feb 2014, 12:34:53 UTC

To 1959, around 2.5M more credit. This also concludes the first (and largest) pass of 3. The next pass is going to focus on multi-part sessions.

1 · 2 · Next
Post to thread

Message boards : Number crunching : Payday