Sun 08 May 2016 08:09:47 BST | BURP | This computer has finished a daily quota of 532 tasks


Advanced search

Message boards : Number crunching : Sun 08 May 2016 08:09:47 BST | BURP | This computer has finished a daily quota of 532 tasks

Author Message
Woodles
Send message
Joined: 28 Jan 16
Posts: 4
Credit: 2,583,225
RAC: 259
Message 14425 - Posted: 8 May 2016, 7:16:58 UTC
Last modified: 8 May 2016, 7:31:55 UTC

With the current tasks only taking a few seconds each this happens in less than two hours yet I managed to crunch 7500 workunits until work stopped being sent late last night. So I assume this limit was introduced/enforced then?

Now whenever I request work I get the above message even though my last workunit was sent and returned yesterday so I can't do any work today.

As a secondary question, is there any way of increasing the daily quota for these short workunits?

[edit]
I have noticed that my last returned workunit errored with "finish file present too long" but it was the only error yesterday, is that why the limit was imposed? And why isn't it reset for today?
[/edit]

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4467
Credit: 2,094,806
RAC: 0
Message 14426 - Posted: 8 May 2016, 12:16:53 UTC - in response to Message 14425.
Last modified: 8 May 2016, 12:22:06 UTC

We hit a somewhat silly bottleneck in the backend system, turns out that the system that archives incoming completed frames has a scheduling resolution measured in whole seconds. To put it simply: the max rendering speed right now for a single session is 1fps and we rendered way faster than that once the current session picked up speed at around the 25% completed mark.
It will take a little while to fix this issue as the service scheduling system is shared between many of the backend server components. I will keep it running at 1fps for now, the current session will probably be done before the fix is ready anyways.
The temporary quota is a feature that kicks in to protect everyone when the system detects an issue with workunit distribution like this. As the backlog clears the quota will also be back to normal.

Woodles
Send message
Joined: 28 Jan 16
Posts: 4
Credit: 2,583,225
RAC: 259
Message 14427 - Posted: 8 May 2016, 21:52:17 UTC - in response to Message 14426.
Last modified: 8 May 2016, 21:52:34 UTC

Thanks for that Janus, I'll just leave things as they are then.

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4467
Credit: 2,094,806
RAC: 0
Message 14435 - Posted: 10 May 2016, 20:28:25 UTC
Last modified: 10 May 2016, 20:43:03 UTC

The new backend software runs faster at around 4.2fps now - and the fix also had a very positive influence on the validator which is now blazingly fast - but we are hitting some other bottlenecks. Essentially the clients are performing a DDoS attack on two of our service ports, namely the file upload handler and the scheduler. Both of these are BOINC services that aren't particularly multi-threaded right now on the server. The requests start queuing up and eventually block human visitors to the website until the web request gets through the queue to open a connection.
I've added a TODO item about this but it will not be fixed during this session.

We may have to start throttling again a bit.


Post to thread

Message boards : Number crunching : Sun 08 May 2016 08:09:47 BST | BURP | This computer has finished a daily quota of 532 tasks