Memory consumption leads to thrashing

Message boards : Number crunching : Memory consumption leads to thrashing
Message board moderation

To post messages, you must log in.

AuthorMessage
Stanislav Sokolov

Send message
Joined: 18 Apr 05
Posts: 4
Credit: 103,121
RAC: 0
Message 8623 - Posted: 24 Aug 2008, 15:07:59 UTC

I have a quad core Q6600 CPU and 4GB of memory on Vista 64bit. I have recently noticed that machine started thrashing memory against the swap file. On closer inspection I discovered that BURP was the culprit. It loaded up to 4 WUs, though only two were running, each WU having a 1.7GB working set. Even with two active WUs, it sums up to 3.4GB of memory needed to BURP, add a third task from another BOINC project (I only allow to use 3 cores as Vista shows signs of priority inversion if all 4 cores are used) and it leaves me with no free memory for my own work.
Is it possible to instruct BURP to only download (or run) once single task?
I have currently suspended the who project as I had to pause individual BURP tasks to allow only one task to run.
ID: 8623 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4544
Credit: 2,097,282
RAC: 193
Message 8634 - Posted: 26 Aug 2008, 14:01:50 UTC - in response to Message 8623.  

You will find settings to control both memory and CPU usage on your account page under General preferences.

Which version of BOINC did you use when this issue happened?
ID: 8634 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Stanislav Sokolov

Send message
Joined: 18 Apr 05
Posts: 4
Credit: 103,121
RAC: 0
Message 8636 - Posted: 26 Aug 2008, 19:37:13 UTC
Last modified: 26 Aug 2008, 19:40:42 UTC

I use 5.10.45.

Even though I specify that WUs should be evicted from memory, I could have up to 4 blender processes show in Task Manager, even though only 1 or 2 were actually running. Total memory consumption was over 3GB and well over the setting of 35% of 4GB memory.

Here are my settings:
Do work while computer is running on batteries?
(matters only for portable computers) no
Do work while computer is in use? yes
Do work only between the hours of (no restriction)
Leave applications in memory while suspended?
(suspended applications will consume swap space if \'yes\') no
Switch between applications every
(recommended: 60 minutes) 120 minutes
On multiprocessors, use at most 3 processors
Use at most
Enforced by version 5.6 and greater 100 percent of CPU time
Disk and memory usage
Use at most 2 GB disk space
Leave at least
(Values smaller than 0.001 are ignored) 0.01 GB disk space free
Use at most 80% of total disk space
Write to disk at most every 120 seconds
Use at most 25% of page file (swap space)
Use at most
Enforced by version 5.8 and greater 35% of memory when computer is in use
Use at most
Enforced by version 5.8 and greater 45% of memory when computer is idle
Network usage
Computer is connected to the Internet about every
(Leave blank or 0 if always connected.
BOINC will try to maintain at least this much work.) 0 days
Maintain enough work for an additional
(Requires 5.10+ client.) 0.5 days
Confirm before connecting to Internet?
(matters only if you have a modem, ISDN or VPN connection) no
Disconnect when done?
(matters only if you have a modem, ISDN or VPN connection) no
Maximum download rate: 200 KB/s
Maximum upload rate: 35 KB/s
Use network only between the hours of
Enforced by versions 4.46 and greater (no restriction)
Skip image file verification?
Check this ONLY if your Internet provider modifies image files (UMTS does this, for example).
Skipping verification reduces the security of BOINC. no
ID: 8636 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4544
Credit: 2,097,282
RAC: 193
Message 8637 - Posted: 26 Aug 2008, 20:48:16 UTC - in response to Message 8636.  

Even though I specify that WUs should be evicted from memory, I could have up to 4 blender processes show in Task Manager, even though only 1 or 2 were actually running. Total memory consumption was over 3GB and well over the setting of 35% of 4GB memory.

Here are my settings:
Use at most 25% of page file (swap space)
Use at most
Enforced by version 5.8 and greater 35% of memory when computer is in use
Use at most
Enforced by version 5.8 and greater 45% of memory when computer is idle

How much swap space do you have? Is it dynamically allocated?
ID: 8637 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Stanislav Sokolov

Send message
Joined: 18 Apr 05
Posts: 4
Credit: 103,121
RAC: 0
Message 8638 - Posted: 30 Aug 2008, 7:51:45 UTC - in response to Message 8637.  

Even though I specify that WUs should be evicted from memory, I could have up to 4 blender processes show in Task Manager, even though only 1 or 2 were actually running. Total memory consumption was over 3GB and well over the setting of 35% of 4GB memory.

Here are my settings:
Use at most 25% of page file (swap space)
Use at most
Enforced by version 5.8 and greater 35% of memory when computer is in use
Use at most
Enforced by version 5.8 and greater 45% of memory when computer is idle

How much swap space do you have? Is it dynamically allocated?


Statically allocated 6141MB (Roughly 6GB) on the fastest RAID striped partition.
ID: 8638 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4544
Credit: 2,097,282
RAC: 193
Message 8642 - Posted: 31 Aug 2008, 9:26:03 UTC - in response to Message 8638.  
Last modified: 31 Aug 2008, 9:26:44 UTC

Ah ok. You wouldn\'t happen to remember how much of the swap space was allocated to the different programs at the time of the incident - and whether these were the ones actually running (ie. actively using the CPU).

The setting for \"switch between programs\" is rather low for that machine when not allowing programs to stay in memory while switching. When coupled with the higher I/O activity just when starting a new workunit this may actually have caused trashing just because of the starting WUs.

This may sound like a funny suggestion, but I\'d actually suggest you to increase the \"switch between\" period and allow programs to stay in memory while switching. This should (theoretically) decrease trashing since science applications are allowed to finish their work before being switched out and at the same time they can gracefully use the swap space when switching in and out rather than having to generate a lot of I/O activity to restart (which in case of BURP can be quite substantial for large sessions like the one which was running when this happened).

I\'m still not exactly sure how you could end up with a process with a working space of 1.7GB since the hardcoded limit was 1.6GB for that session. Maybe something went wrong during the shutdown of those workunits and they were left running in idle state.

In the beta client 2 things have been changed to try to overcome such issues:
1) I/O when starting has been cut in half and the initial memory footprint lowered considerably by doing the unpacking and file I/O once rather than twice for each WU primary input file.
2) Slightly better memory management and a 2-level monitoring of memory usage should ensure that clients never allocate more memory than the limit. This should already be the case but the current client has some very old 1-level code where it may be failing.
ID: 8642 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Stanislav Sokolov

Send message
Joined: 18 Apr 05
Posts: 4
Credit: 103,121
RAC: 0
Message 8649 - Posted: 31 Aug 2008, 18:18:59 UTC - in response to Message 8642.  

Thank you for the suggestions. I will try the suggested settings and if they do not help, I\'ll post a screenshot of the Task Manager status.
ID: 8649 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Memory consumption leads to thrashing