CPU wasting

Message boards : Problems and Help : CPU wasting
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
yik

Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11833 - Posted: 8 May 2013, 5:21:56 UTC

I have a PC with totally 32thread and one BURP task holds all 32thread. However, it actually uses up 25% CPU (about 8thread of 32) and I think those BURP tasks can be limited for 8thread per task.
How should I do it? I don't want wasting CPU resource any more so I have stopped all BURP tasks...

Thanks for reply~
ID: 11833 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Speedy

Send message
Joined: 25 May 06
Posts: 206
Credit: 674,709
RAC: 1
Message 11834 - Posted: 8 May 2013, 6:24:06 UTC
Last modified: 8 May 2013, 6:27:35 UTC

I understand where you are coming from two a point with CPU usage been low. I have a 12 thread I7 980 X running at 3.43 GHz (stock 3.33 GHz) task's run under full load using about 66% of my CPU which is about 7.92 thread's. At times my CPU usage can be as low as 8% . I find this happens at the beginning and the end of tasks a task from session 1436 takes roughly 4.6 hours. Out of interest may I ask how long it takes for your machine to return a task and how fast your threads are?
If we did decrease the number of cores that each task uses it will increase the overall time it takes to complete a task/session
It would be nice if you are able to stay around and help finish sunflower sessions. However I wish you well in what ever you decide to do.

Have a crunching good day!!
Live in NZ y not join Smile City?
ID: 11834 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DoctorNow
Project donor
Avatar

Send message
Joined: 11 Apr 05
Posts: 403
Credit: 2,182,867
RAC: 68
Message 11835 - Posted: 8 May 2013, 9:03:56 UTC - in response to Message 11833.  
Last modified: 8 May 2013, 9:05:56 UTC

I have a PC with totally 32thread and one BURP task holds all 32thread. However, it actually uses up 25% CPU (about 8thread of 32) and I think those BURP tasks can be limited for 8thread per task.

Well, would be a wonder when the Sunflower-app wouldn't use all available cores. But probably there's a limit for coding a multicore-app. From my experience and looking at a lot of results since beginning of the Sunflower-event I guess it can at least use 12 cores, maybe 16 is the limit.
As Speedy already mentioned note that the app not always uses all cores depending on the processing step, if you watch over the usage you can sometimes see a dropdown, especially at the beginning and ending of a WU.


How should I do it? I don't want wasting CPU resource any more so I have stopped all BURP tasks...

Well, there is no way that you can change about the cpu usage from your side. It's an app-feature/issue that all cores are occupied.
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats
ID: 11835 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
yik

Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11836 - Posted: 8 May 2013, 12:01:25 UTC

My 32thread PC running at 2.7GHz finished a task from session 1436 for 5.7hours.
It seems my PC couldn't take the advantage of 32thread for the BURP task....

I still have the interest to help finish sunflower sessions and hope that there will be some improvement for my PC to run several BURP tasks at the same time~

Thanks for your reply~!
ID: 11836 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
yik

Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11837 - Posted: 9 May 2013, 6:08:51 UTC

I try running a BURP task again and I am sure that it constantly uses 25% CPU and still holds my 32thread in the Boinc and other tasks just waiting to run ~~
ID: 11837 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Speedy

Send message
Joined: 25 May 06
Posts: 206
Credit: 674,709
RAC: 1
Message 11838 - Posted: 9 May 2013, 8:40:16 UTC

Pleased to hear that you are running tasks again. Are the other tasks waiting to run from this project? If not I would assume that the other tasks have a longer deadline? Is it possible for you to unhide your hosts)? (I completely understand if you cannot do this) so we can see the ram your host has and how long tasks take and the credit you get given. I would be very interested to know if it is possible to run more than one sunflower task at a time if you have the capacity. In the instant we are talking about the capacity is there

Have a crunching good day!!
Live in NZ y not join Smile City?
ID: 11838 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
yik

Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11839 - Posted: 9 May 2013, 9:33:28 UTC - in response to Message 11838.  

yes~ Only one BURP task was running and the others were waiting to run from this project.

All tasks were also SunflowerBlender and the deadline of them were almost the same.

I am sorry that I don't want unhiding my hosts at this moment but I can give you information of my PC at much as possible for improving the performance of the BURP project~

My 32thread PC is running DDR3 (4+4+8+8)24GB Ram.
Total credit 3,689
Average credit 346.08
Operating System Microsoft Windows 7
Ultimate x64 Edition, Service Pack 1, (06.01.7601.00)
BOINC version 7.0.64
Memory 24547.54 MB
Cache 256 KB
Measured floating point speed 2557.59 million ops/sec
Measured integer speed 9832.54 million ops/sec


It had finished two tasks from Session 1436.
1) Run 20,669.75s CPUTime 151,338.44s Credit 1,156.19
2) Run 14,750.48s CPUTime 105,602.97s Credit 2,521.91

The tasks from Session 1436 usually used about 5GB Ram so I am sure that my PC can run more at a moment.

Hope these info. are useful~
ID: 11839 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Speedy

Send message
Joined: 25 May 06
Posts: 206
Credit: 674,709
RAC: 1
Message 11840 - Posted: 9 May 2013, 22:06:04 UTC

Thank you for information. You have a very nice computer.

Have a crunching good day!!
Live in NZ y not join Smile City?
ID: 11840 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DoctorNow
Project donor
Avatar

Send message
Joined: 11 Apr 05
Posts: 403
Credit: 2,182,867
RAC: 68
Message 11841 - Posted: 10 May 2013, 6:19:37 UTC - in response to Message 11837.  

I try running a BURP task again and I am sure that it constantly uses 25% CPU and still holds my 32thread in the Boinc and other tasks just waiting to run ~~

Hm, this is indeed a little bad when you have 32 cores and 24 GB RAM available.
I don't know, I thought the app does use as much as cores are there, but obviously not then.

Janus, do you know exactly about how many cores SunflowerBlender can really use?
If there's a limit the developers should do something about that not all cores are occupied when a computer has more cores as the app can use.
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats
ID: 11841 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4556
Credit: 2,097,282
RAC: 0
Message 11842 - Posted: 11 May 2013, 22:09:11 UTC - in response to Message 11841.  

I try running a BURP task again and I am sure that it constantly uses 25% CPU and still holds my 32thread in the Boinc and other tasks just waiting to run ~~

Janus, do you know exactly about how many cores SunflowerBlender can really use?

Nope, and that's actually an interesting question. On the machines I have access to Blender will start drowning the memory I/O controllers when it runs at around 10 or so cores. It is quite likely that running two BURP units alongside each other on so many cores is also a bad idea (for the same reason) and that mixing with something with a smaller memory footprint (like SETI or Einstein) would be a better idea.

I have no clue why the default multiprocessing plan class is set to 64 cpus. It means that you have to have 65 cpus to run to units alongside each other. A better number would probably be 16. This was added to the "Stuff-to-fix-after-Sunflower"-list.
ID: 11842 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DoctorNow
Project donor
Avatar

Send message
Joined: 11 Apr 05
Posts: 403
Credit: 2,182,867
RAC: 68
Message 11844 - Posted: 13 May 2013, 4:43:20 UTC - in response to Message 11842.  
Last modified: 13 May 2013, 4:45:07 UTC

Nope, and that's actually an interesting question. On the machines I have access to Blender will start drowning the memory I/O controllers when it runs at around 10 or so cores. It is quite likely that running two BURP units alongside each other on so many cores is also a bad idea (for the same reason) and that mixing with something with a smaller memory footprint (like SETI or Einstein) would be a better idea.

I see, so it looks like there's a hardware limit here...

I have no clue why the default multiprocessing plan class is set to 64 cpus. It means that you have to have 65 cpus to run to units alongside each other. A better number would probably be 16. This was added to the "Stuff-to-fix-after-Sunflower"-list.

Hm, plan class, I see... I think I have an idea to give yik something interesting.
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats
ID: 11844 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DoctorNow
Project donor
Avatar

Send message
Joined: 11 Apr 05
Posts: 403
Credit: 2,182,867
RAC: 68
Message 11845 - Posted: 14 May 2013, 6:24:14 UTC - in response to Message 11844.  
Last modified: 14 May 2013, 6:49:03 UTC

Posted a bit too early, I'm still testing... ;-)
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats
ID: 11845 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DoctorNow
Project donor
Avatar

Send message
Joined: 11 Apr 05
Posts: 403
Credit: 2,182,867
RAC: 68
Message 11846 - Posted: 14 May 2013, 7:55:02 UTC
Last modified: 14 May 2013, 8:32:07 UTC

Okay, after a bit digging in the app_info.xml-stuff again I found out it was easier than I thought, not even the plan_class is needed as I thought first. Here's an app_info.xml which can restrict how much cpus SunflowerBlender can use:

<app_info>
<app>
<name>sunflowerblender</name>
<user_friendly_name>SunflowerBlender</user_friendly_name>
<non_cpu_intensive>0</non_cpu_intensive>
</app>
<file_info>
<name>sunflowerblender_4.81_windows_x86_64__mt</name>
<min_cpus>4</min_cpus>
<nbytes>6371008.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
<executable/>
</file_info>
<file_info>
<name>sunflowerblender_4.81_windows_x86_64__mt.zip</name>
<nbytes>56122395.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
</file_info>
<file_info>
<name>infozip_license</name>
<nbytes>3041.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
</file_info>
<file_info>
<name>windows_zip.exe</name>
<nbytes>135168.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
<executable/>
</file_info>
<file_info>
<name>windows_unzip.exe</name>
<nbytes>167936.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
<executable/>
</file_info>
<app_version>
<app_name>sunflowerblender</app_name>
<version_num>481</version_num>
<avg_ncpus>X</avg_ncpus>
<max_ncpus>X</max_ncpus>

<file_ref>
<file_name>sunflowerblender_4.81_windows_x86_64__mt</file_name>
<main_program/>
</file_ref>
<file_ref>
<file_name>sunflowerblender_4.81_windows_x86_64__mt.zip</file_name>
</file_ref>
<file_ref>
<file_name>infozip_license</file_name>
</file_ref>
<file_ref>
<file_name>windows_zip.exe</file_name>
</file_ref>
<file_ref>
<file_name>windows_unzip.exe</file_name>
</file_ref>
</app_version>
</app_info>

yik, in case you don't know what to do with this:
Create a file named "app_info.xml" in the BURP project folder used by BOINC.
Copy & paste the above lines in that, save and restart BOINC.
Note the two bold lines. Here you can set the amount of cores you want to use from the app. The X should have the same number.
Since 10 seems to be a hardware limit I suggest a usage from 6 to 8, but it's up to you. The rest of your 32 cores can do anything else then. :-)
Disadvantage of this: you don't get normal Blender-WUs anymore, the file would need a little extension then.
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats
ID: 11846 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
yik

Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11847 - Posted: 14 May 2013, 15:25:56 UTC - in response to Message 11846.  

It seems intersting...
But after I created the app_info.xml file, the new tasks become forever downloading~
How should I do?
ID: 11847 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DoctorNow
Project donor
Avatar

Send message
Joined: 11 Apr 05
Posts: 403
Credit: 2,182,867
RAC: 68
Message 11848 - Posted: 14 May 2013, 17:51:45 UTC - in response to Message 11847.  

From what session are the tasks?
Some can take a while to download (depending on your internet connection anyway) - probably it is finished meanwhile. ;-)
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats
ID: 11848 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
yik

Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11850 - Posted: 16 May 2013, 12:23:36 UTC - in response to Message 11848.  
Last modified: 16 May 2013, 12:24:19 UTC

http://burp.renderfarming.net/session.php?id=1436

^This

After all items on the transfer list were downloaded, status of all tasks were still "downloading".
I had put the 5 files in the project folder but it didn't start rendering as I didn't see any blender program were running from the Windows Task Manager.

Are there any way to get the BURP tasks start rendering?
ID: 11850 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DoctorNow
Project donor
Avatar

Send message
Joined: 11 Apr 05
Posts: 403
Credit: 2,182,867
RAC: 68
Message 11852 - Posted: 17 May 2013, 1:34:09 UTC - in response to Message 11850.  

After all items on the transfer list were downloaded, status of all tasks were still "downloading".

Hm, a bit strange. I don't think it has anything to do with the app_info.xml but who knows...
Can you post the event log here? Relevant should be only the parts from the BOINC-start with all the settings and the parts from where you started downloading tasks.
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats
ID: 11852 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
yik

Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11853 - Posted: 17 May 2013, 6:42:20 UTC - in response to Message 11852.  

After all items on the transfer list were downloaded, status of all tasks were still "downloading".

Hm, a bit strange. I don't think it has anything to do with the app_info.xml but who knows...
Can you post the event log here? Relevant should be only the parts from the BOINC-start with all the settings and the parts from where you started downloading tasks.

17/5/2013 14:33:36 |  | No config file found - using defaults
17/5/2013 14:33:36 |  | Starting BOINC client version 7.0.64 for windows_x86_64
17/5/2013 14:33:36 |  | log flags: file_xfer, sched_ops, task
17/5/2013 14:33:36 |  | Libraries: libcurl/7.25.0 OpenSSL/1.0.1 zlib/1.2.6
17/5/2013 14:33:36 |  | Data directory: D:\BOINC\aXeon
17/5/2013 14:33:36 |  | Running under account yik
17/5/2013 14:33:36 |  | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt aes syscall nx lm vmx smx tm2 dca pbe
17/5/2013 14:33:36 |  | OS: Microsoft Windows 7: Ultimate x64 Edition, Service Pack 1, (06.01.7601.00)
17/5/2013 14:33:36 |  | Memory: 23.97 GB physical, 24.07 GB virtual
17/5/2013 14:33:36 |  | Disk: 931.51 GB total, 612.05 GB free
17/5/2013 14:33:36 |  | CUDA: NVIDIA GPU 0: GeForce GTX 660 (driver version 314.22, CUDA version 5.0, compute capability 3.0, 2048MB, 1578MB available, 2058 GFLOPS peak)
17/5/2013 14:33:36 |  | OpenCL: NVIDIA GPU 0: GeForce GTX 660 (driver version 314.22, device version OpenCL 1.1 CUDA, 2048MB, 1578MB available, 2058 GFLOPS peak)
17/5/2013 14:33:36 | BURP | Found app_info.xml; using anonymous platform
17/5/2013 14:33:36 | BURP | URL http://burp.renderfarming.net/; Computer ID 58822; resource share 100
17/5/2013 14:33:36 | BURP | General prefs: from BURP (last modified 08-May-2013 11:20:22)
17/5/2013 14:33:36 | BURP | Host location: none
17/5/2013 14:33:36 | BURP | General prefs: using your defaults
17/5/2013 14:33:36 |  | Reading preferences override file
17/5/2013 14:33:36 |  | Preferences:
17/5/2013 14:33:36 |  | max memory usage when active: 15955.90MB
17/5/2013 14:33:36 |  | max memory usage when idle: 18410.66MB
17/5/2013 14:33:36 |  | max disk usage: 100.00GB
17/5/2013 14:33:36 |  | suspend work if non-BOINC CPU load exceeds 12 %
17/5/2013 14:33:36 |  | (to change preferences, visit a project web site or select Preferences in the Manager)
17/5/2013 14:33:36 |  | File projects/burp.renderfarming.net/sunflowerblender_4.81_windows_x86_64__mt.zip has wrong size: expected 56122395, got 10637372
17/5/2013 14:33:36 |  | Not using a proxy
17/5/2013 14:33:37 | BURP | Started download of file1882709
17/5/2013 14:33:40 | BURP | update requested by user
17/5/2013 14:33:42 | BURP | Sending scheduler request: Requested by user.
17/5/2013 14:33:42 | BURP | Not requesting tasks: project is not highest priority
17/5/2013 14:33:45 | BURP | Scheduler request completed
17/5/2013 14:34:00 | BURP | Sending scheduler request: To fetch work.
17/5/2013 14:34:00 | BURP | Requesting new tasks for CPU
17/5/2013 14:34:01 | BURP | Scheduler request completed: got 4 new tasks
17/5/2013 14:38:15 | BURP | Finished download of file1882709
ID: 11853 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DoctorNow
Project donor
Avatar

Send message
Joined: 11 Apr 05
Posts: 403
Credit: 2,182,867
RAC: 68
Message 11854 - Posted: 17 May 2013, 9:04:37 UTC - in response to Message 11853.  
Last modified: 17 May 2013, 9:26:35 UTC

17/5/2013 14:33:36 | | File projects/burp.renderfarming.net/sunflowerblender_4.81_windows_x86_64__mt.zip has wrong size: expected 56122395, got 10637372

Oops, this is probably the culprit. Shows me that I forgot a little thing, sorry.
Please delete all the lines in the app_info.xml which start with <nbytes> and <max_nbytes> and restart. That should do it.
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats
ID: 11854 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
yik

Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11855 - Posted: 19 May 2013, 3:07:31 UTC - in response to Message 11854.  

17/5/2013 14:33:36 | | File projects/burp.renderfarming.net/sunflowerblender_4.81_windows_x86_64__mt.zip has wrong size: expected 56122395, got 10637372

Oops, this is probably the culprit. Shows me that I forgot a little thing, sorry.
Please delete all the lines in the app_info.xml which start with <nbytes> and <max_nbytes> and restart. That should do it.



Unfortunately, I got into the 3% CPU usage issue even using app_info for RenderFarm.fi got this issue.
All local tasks didn't use more than 3% CPU and one of the BURP task needed render ing for more than 13hrs last night and I had to sleep so I turned off my computer and I don't know how long will it take for using 3% CPU to render a 4K 3D picture@@
ID: 11855 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Problems and Help : CPU wasting