CPU wasting


Advanced search

Message boards : Problems and Help : CPU wasting

1 · 2 · Next
Author Message
yik
Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11833 - Posted: 8 May 2013, 5:21:56 UTC

I have a PC with totally 32thread and one BURP task holds all 32thread. However, it actually uses up 25% CPU (about 8thread of 32) and I think those BURP tasks can be limited for 8thread per task.
How should I do it? I don't want wasting CPU resource any more so I have stopped all BURP tasks...

Thanks for reply~

Speedy
Send message
Joined: 25 May 06
Posts: 206
Credit: 670,351
RAC: 189
Message 11834 - Posted: 8 May 2013, 6:24:06 UTC
Last modified: 8 May 2013, 6:27:35 UTC

I understand where you are coming from two a point with CPU usage been low. I have a 12 thread I7 980 X running at 3.43 GHz (stock 3.33 GHz) task's run under full load using about 66% of my CPU which is about 7.92 thread's. At times my CPU usage can be as low as 8% . I find this happens at the beginning and the end of tasks a task from session 1436 takes roughly 4.6 hours. Out of interest may I ask how long it takes for your machine to return a task and how fast your threads are?
If we did decrease the number of cores that each task uses it will increase the overall time it takes to complete a task/session
It would be nice if you are able to stay around and help finish sunflower sessions. However I wish you well in what ever you decide to do.
____________

Have a crunching good day!!
Live in NZ y not join Smile City?

Profile DoctorNow
Project donor
Avatar
Send message
Joined: 11 Apr 05
Posts: 392
Credit: 2,168,338
RAC: 3
Message 11835 - Posted: 8 May 2013, 9:03:56 UTC - in response to Message 11833.
Last modified: 8 May 2013, 9:05:56 UTC

I have a PC with totally 32thread and one BURP task holds all 32thread. However, it actually uses up 25% CPU (about 8thread of 32) and I think those BURP tasks can be limited for 8thread per task.

Well, would be a wonder when the Sunflower-app wouldn't use all available cores. But probably there's a limit for coding a multicore-app. From my experience and looking at a lot of results since beginning of the Sunflower-event I guess it can at least use 12 cores, maybe 16 is the limit.
As Speedy already mentioned note that the app not always uses all cores depending on the processing step, if you watch over the usage you can sometimes see a dropdown, especially at the beginning and ending of a WU.


How should I do it? I don't want wasting CPU resource any more so I have stopped all BURP tasks...

Well, there is no way that you can change about the cpu usage from your side. It's an app-feature/issue that all cores are occupied.
____________
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats

yik
Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11836 - Posted: 8 May 2013, 12:01:25 UTC

My 32thread PC running at 2.7GHz finished a task from session 1436 for 5.7hours.
It seems my PC couldn't take the advantage of 32thread for the BURP task....

I still have the interest to help finish sunflower sessions and hope that there will be some improvement for my PC to run several BURP tasks at the same time~

Thanks for your reply~!

yik
Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11837 - Posted: 9 May 2013, 6:08:51 UTC

I try running a BURP task again and I am sure that it constantly uses 25% CPU and still holds my 32thread in the Boinc and other tasks just waiting to run ~~

Speedy
Send message
Joined: 25 May 06
Posts: 206
Credit: 670,351
RAC: 189
Message 11838 - Posted: 9 May 2013, 8:40:16 UTC

Pleased to hear that you are running tasks again. Are the other tasks waiting to run from this project? If not I would assume that the other tasks have a longer deadline? Is it possible for you to unhide your hosts)? (I completely understand if you cannot do this) so we can see the ram your host has and how long tasks take and the credit you get given. I would be very interested to know if it is possible to run more than one sunflower task at a time if you have the capacity. In the instant we are talking about the capacity is there
____________

Have a crunching good day!!
Live in NZ y not join Smile City?

yik
Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11839 - Posted: 9 May 2013, 9:33:28 UTC - in response to Message 11838.

yes~ Only one BURP task was running and the others were waiting to run from this project.

All tasks were also SunflowerBlender and the deadline of them were almost the same.

I am sorry that I don't want unhiding my hosts at this moment but I can give you information of my PC at much as possible for improving the performance of the BURP project~

My 32thread PC is running DDR3 (4+4+8+8)24GB Ram.
Total credit 3,689
Average credit 346.08
Operating System Microsoft Windows 7
Ultimate x64 Edition, Service Pack 1, (06.01.7601.00)
BOINC version 7.0.64
Memory 24547.54 MB
Cache 256 KB
Measured floating point speed 2557.59 million ops/sec
Measured integer speed 9832.54 million ops/sec


It had finished two tasks from Session 1436.
1) Run 20,669.75s CPUTime 151,338.44s Credit 1,156.19
2) Run 14,750.48s CPUTime 105,602.97s Credit 2,521.91

The tasks from Session 1436 usually used about 5GB Ram so I am sure that my PC can run more at a moment.

Hope these info. are useful~

Speedy
Send message
Joined: 25 May 06
Posts: 206
Credit: 670,351
RAC: 189
Message 11840 - Posted: 9 May 2013, 22:06:04 UTC

Thank you for information. You have a very nice computer.
____________

Have a crunching good day!!
Live in NZ y not join Smile City?

Profile DoctorNow
Project donor
Avatar
Send message
Joined: 11 Apr 05
Posts: 392
Credit: 2,168,338
RAC: 3
Message 11841 - Posted: 10 May 2013, 6:19:37 UTC - in response to Message 11837.

I try running a BURP task again and I am sure that it constantly uses 25% CPU and still holds my 32thread in the Boinc and other tasks just waiting to run ~~

Hm, this is indeed a little bad when you have 32 cores and 24 GB RAM available.
I don't know, I thought the app does use as much as cores are there, but obviously not then.

Janus, do you know exactly about how many cores SunflowerBlender can really use?
If there's a limit the developers should do something about that not all cores are occupied when a computer has more cores as the app can use.
____________
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats

Profile Janus
Volunteer moderator
Project administrator
Avatar
Send message
Joined: 16 Jun 04
Posts: 4487
Credit: 2,094,806
RAC: 0
Message 11842 - Posted: 11 May 2013, 22:09:11 UTC - in response to Message 11841.

I try running a BURP task again and I am sure that it constantly uses 25% CPU and still holds my 32thread in the Boinc and other tasks just waiting to run ~~

Janus, do you know exactly about how many cores SunflowerBlender can really use?

Nope, and that's actually an interesting question. On the machines I have access to Blender will start drowning the memory I/O controllers when it runs at around 10 or so cores. It is quite likely that running two BURP units alongside each other on so many cores is also a bad idea (for the same reason) and that mixing with something with a smaller memory footprint (like SETI or Einstein) would be a better idea.

I have no clue why the default multiprocessing plan class is set to 64 cpus. It means that you have to have 65 cpus to run to units alongside each other. A better number would probably be 16. This was added to the "Stuff-to-fix-after-Sunflower"-list.

Profile DoctorNow
Project donor
Avatar
Send message
Joined: 11 Apr 05
Posts: 392
Credit: 2,168,338
RAC: 3
Message 11844 - Posted: 13 May 2013, 4:43:20 UTC - in response to Message 11842.
Last modified: 13 May 2013, 4:45:07 UTC

Nope, and that's actually an interesting question. On the machines I have access to Blender will start drowning the memory I/O controllers when it runs at around 10 or so cores. It is quite likely that running two BURP units alongside each other on so many cores is also a bad idea (for the same reason) and that mixing with something with a smaller memory footprint (like SETI or Einstein) would be a better idea.

I see, so it looks like there's a hardware limit here...

I have no clue why the default multiprocessing plan class is set to 64 cpus. It means that you have to have 65 cpus to run to units alongside each other. A better number would probably be 16. This was added to the "Stuff-to-fix-after-Sunflower"-list.

Hm, plan class, I see... I think I have an idea to give yik something interesting.
____________
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats

Profile DoctorNow
Project donor
Avatar
Send message
Joined: 11 Apr 05
Posts: 392
Credit: 2,168,338
RAC: 3
Message 11845 - Posted: 14 May 2013, 6:24:14 UTC - in response to Message 11844.
Last modified: 14 May 2013, 6:49:03 UTC

Posted a bit too early, I'm still testing... ;-)
____________
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats

Profile DoctorNow
Project donor
Avatar
Send message
Joined: 11 Apr 05
Posts: 392
Credit: 2,168,338
RAC: 3
Message 11846 - Posted: 14 May 2013, 7:55:02 UTC
Last modified: 14 May 2013, 8:32:07 UTC

Okay, after a bit digging in the app_info.xml-stuff again I found out it was easier than I thought, not even the plan_class is needed as I thought first. Here's an app_info.xml which can restrict how much cpus SunflowerBlender can use:

<app_info>
<app>
<name>sunflowerblender</name>
<user_friendly_name>SunflowerBlender</user_friendly_name>
<non_cpu_intensive>0</non_cpu_intensive>
</app>
<file_info>
<name>sunflowerblender_4.81_windows_x86_64__mt</name>
<min_cpus>4</min_cpus>
<nbytes>6371008.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
<executable/>
</file_info>
<file_info>
<name>sunflowerblender_4.81_windows_x86_64__mt.zip</name>
<nbytes>56122395.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
</file_info>
<file_info>
<name>infozip_license</name>
<nbytes>3041.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
</file_info>
<file_info>
<name>windows_zip.exe</name>
<nbytes>135168.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
<executable/>
</file_info>
<file_info>
<name>windows_unzip.exe</name>
<nbytes>167936.000000</nbytes>
<max_nbytes>0.000000</max_nbytes>
<status>1</status>
<executable/>
</file_info>
<app_version>
<app_name>sunflowerblender</app_name>
<version_num>481</version_num>
<avg_ncpus>X</avg_ncpus>
<max_ncpus>X</max_ncpus>

<file_ref>
<file_name>sunflowerblender_4.81_windows_x86_64__mt</file_name>
<main_program/>
</file_ref>
<file_ref>
<file_name>sunflowerblender_4.81_windows_x86_64__mt.zip</file_name>
</file_ref>
<file_ref>
<file_name>infozip_license</file_name>
</file_ref>
<file_ref>
<file_name>windows_zip.exe</file_name>
</file_ref>
<file_ref>
<file_name>windows_unzip.exe</file_name>
</file_ref>
</app_version>
</app_info>

yik, in case you don't know what to do with this:
Create a file named "app_info.xml" in the BURP project folder used by BOINC.
Copy & paste the above lines in that, save and restart BOINC.
Note the two bold lines. Here you can set the amount of cores you want to use from the app. The X should have the same number.
Since 10 seems to be a hardware limit I suggest a usage from 6 to 8, but it's up to you. The rest of your 32 cores can do anything else then. :-)
Disadvantage of this: you don't get normal Blender-WUs anymore, the file would need a little extension then.
____________
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats

yik
Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11847 - Posted: 14 May 2013, 15:25:56 UTC - in response to Message 11846.

It seems intersting...
But after I created the app_info.xml file, the new tasks become forever downloading~
How should I do?

Profile DoctorNow
Project donor
Avatar
Send message
Joined: 11 Apr 05
Posts: 392
Credit: 2,168,338
RAC: 3
Message 11848 - Posted: 14 May 2013, 17:51:45 UTC - in response to Message 11847.

From what session are the tasks?
Some can take a while to download (depending on your internet connection anyway) - probably it is finished meanwhile. ;-)
____________
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats

yik
Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11850 - Posted: 16 May 2013, 12:23:36 UTC - in response to Message 11848.
Last modified: 16 May 2013, 12:24:19 UTC

http://burp.renderfarming.net/session.php?id=1436

^This

After all items on the transfer list were downloaded, status of all tasks were still "downloading".
I had put the 5 files in the project folder but it didn't start rendering as I didn't see any blender program were running from the Windows Task Manager.

Are there any way to get the BURP tasks start rendering?

Profile DoctorNow
Project donor
Avatar
Send message
Joined: 11 Apr 05
Posts: 392
Credit: 2,168,338
RAC: 3
Message 11852 - Posted: 17 May 2013, 1:34:09 UTC - in response to Message 11850.

After all items on the transfer list were downloaded, status of all tasks were still "downloading".

Hm, a bit strange. I don't think it has anything to do with the app_info.xml but who knows...
Can you post the event log here? Relevant should be only the parts from the BOINC-start with all the settings and the parts from where you started downloading tasks.
____________
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats

yik
Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11853 - Posted: 17 May 2013, 6:42:20 UTC - in response to Message 11852.

After all items on the transfer list were downloaded, status of all tasks were still "downloading".

Hm, a bit strange. I don't think it has anything to do with the app_info.xml but who knows...
Can you post the event log here? Relevant should be only the parts from the BOINC-start with all the settings and the parts from where you started downloading tasks.

17/5/2013 14:33:36 | | No config file found - using defaults 17/5/2013 14:33:36 | | Starting BOINC client version 7.0.64 for windows_x86_64 17/5/2013 14:33:36 | | log flags: file_xfer, sched_ops, task 17/5/2013 14:33:36 | | Libraries: libcurl/7.25.0 OpenSSL/1.0.1 zlib/1.2.6 17/5/2013 14:33:36 | | Data directory: D:\BOINC\aXeon 17/5/2013 14:33:36 | | Running under account yik 17/5/2013 14:33:36 | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt aes syscall nx lm vmx smx tm2 dca pbe 17/5/2013 14:33:36 | | OS: Microsoft Windows 7: Ultimate x64 Edition, Service Pack 1, (06.01.7601.00) 17/5/2013 14:33:36 | | Memory: 23.97 GB physical, 24.07 GB virtual 17/5/2013 14:33:36 | | Disk: 931.51 GB total, 612.05 GB free 17/5/2013 14:33:36 | | CUDA: NVIDIA GPU 0: GeForce GTX 660 (driver version 314.22, CUDA version 5.0, compute capability 3.0, 2048MB, 1578MB available, 2058 GFLOPS peak) 17/5/2013 14:33:36 | | OpenCL: NVIDIA GPU 0: GeForce GTX 660 (driver version 314.22, device version OpenCL 1.1 CUDA, 2048MB, 1578MB available, 2058 GFLOPS peak) 17/5/2013 14:33:36 | BURP | Found app_info.xml; using anonymous platform 17/5/2013 14:33:36 | BURP | URL http://burp.renderfarming.net/; Computer ID 58822; resource share 100 17/5/2013 14:33:36 | BURP | General prefs: from BURP (last modified 08-May-2013 11:20:22) 17/5/2013 14:33:36 | BURP | Host location: none 17/5/2013 14:33:36 | BURP | General prefs: using your defaults 17/5/2013 14:33:36 | | Reading preferences override file 17/5/2013 14:33:36 | | Preferences: 17/5/2013 14:33:36 | | max memory usage when active: 15955.90MB 17/5/2013 14:33:36 | | max memory usage when idle: 18410.66MB 17/5/2013 14:33:36 | | max disk usage: 100.00GB 17/5/2013 14:33:36 | | suspend work if non-BOINC CPU load exceeds 12 % 17/5/2013 14:33:36 | | (to change preferences, visit a project web site or select Preferences in the Manager) 17/5/2013 14:33:36 | | File projects/burp.renderfarming.net/sunflowerblender_4.81_windows_x86_64__mt.zip has wrong size: expected 56122395, got 10637372 17/5/2013 14:33:36 | | Not using a proxy 17/5/2013 14:33:37 | BURP | Started download of file1882709 17/5/2013 14:33:40 | BURP | update requested by user 17/5/2013 14:33:42 | BURP | Sending scheduler request: Requested by user. 17/5/2013 14:33:42 | BURP | Not requesting tasks: project is not highest priority 17/5/2013 14:33:45 | BURP | Scheduler request completed 17/5/2013 14:34:00 | BURP | Sending scheduler request: To fetch work. 17/5/2013 14:34:00 | BURP | Requesting new tasks for CPU 17/5/2013 14:34:01 | BURP | Scheduler request completed: got 4 new tasks 17/5/2013 14:38:15 | BURP | Finished download of file1882709

Profile DoctorNow
Project donor
Avatar
Send message
Joined: 11 Apr 05
Posts: 392
Credit: 2,168,338
RAC: 3
Message 11854 - Posted: 17 May 2013, 9:04:37 UTC - in response to Message 11853.
Last modified: 17 May 2013, 9:26:35 UTC

17/5/2013 14:33:36 | | File projects/burp.renderfarming.net/sunflowerblender_4.81_windows_x86_64__mt.zip has wrong size: expected 56122395, got 10637372

Oops, this is probably the culprit. Shows me that I forgot a little thing, sorry.
Please delete all the lines in the app_info.xml which start with <nbytes> and <max_nbytes> and restart. That should do it.
____________
Life is Science, and Science rules. To the universe and beyond
Proud member of BOINC@Heidelberg
My BOINC-Stats

yik
Send message
Joined: 27 Dec 12
Posts: 18
Credit: 236,230
RAC: 0
Message 11855 - Posted: 19 May 2013, 3:07:31 UTC - in response to Message 11854.

17/5/2013 14:33:36 | | File projects/burp.renderfarming.net/sunflowerblender_4.81_windows_x86_64__mt.zip has wrong size: expected 56122395, got 10637372

Oops, this is probably the culprit. Shows me that I forgot a little thing, sorry.
Please delete all the lines in the app_info.xml which start with <nbytes> and <max_nbytes> and restart. That should do it.



Unfortunately, I got into the 3% CPU usage issue even using app_info for RenderFarm.fi got this issue.
All local tasks didn't use more than 3% CPU and one of the BURP task needed render ing for more than 13hrs last night and I had to sleep so I turned off my computer and I don't know how long will it take for using 3% CPU to render a 4K 3D picture@@

1 · 2 · Next
Post to thread

Message boards : Problems and Help : CPU wasting