The database overload issue during 135

Message boards : Server backend and mirrors : The database overload issue during 135
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4568
Credit: 2,100,409
RAC: 50
Message 1478 - Posted: 3 Aug 2005, 9:35:51 UTC

During session 135 the database (more precisely the result table) suddenly decided to commit suicide and caused sustained disk activity with about 90MB/s reads while eating up all available system memory.

The reason was that the table had outgrown the amount of RAM available on the server system and a number of ineffective queries required all of the table to be able to fit in RAM in order to work at all.

At some point the disk and memory activity simply became too much for the database server and it started dropping connections.

I have been spending some time trying to track down some of these ineffective database queries. At least one very nasty one, that is triggered every time someone reports back a result, has been isolated and will eventually be changed.
Another one was the one that updated the number of waiting parts and the current renderspeed for use in the project status box in the lower part of the menu on the website. This one will be rewritten to make better use of SQL estimation functions as well as caching. Instead of updating it live it will only be updated every 10 mins in the future.

This is still an ongoing process, and it is bound to take some time to get everything to work again.

-------

"What about the results for session 135?"
All the workunits for 135 will be deleted and reissued when the system is ready again.

"What about my credits?"
You will keep the credits that you have got already. You shouldn't expect to get credits for any 135 work you haven't already gotten credit for.

"But I still have results waiting to be turned in for 135, will I get credit for that?"
No, probably not. All session 135 workunits will be aborted and eventually they will time out on your client as well.

"Should I reset the project in the client then?"
Nope. At least not right now.
ID: 1478 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Server backend and mirrors : The database overload issue during 135