Mirror software version 1.93 upgrade

Message boards : Server backend and mirrors : Mirror software version 1.93 upgrade
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4571
Credit: 2,100,463
RAC: 8
Message 1952 - Posted: 20 Nov 2005, 11:42:17 UTC

The mirror software has now been updated to version 1.93. You can download the new version from here:
http://burp.boinc.dk/development/burp_mirror.tar.gz

Changes in this new version include:
- More errors are caught and displayed as fatal errors
- Debug is now being piped to a file instead of the http stream
- The mirrors send their ID as part of http requests so that the main server can make decisions based on whether it is a mirror or a normal user accessing a file.
- Headers are no longer sent if a mirror-mirror connection is used. This saves a little bandwidth and avoids errors on some servers.
- Missing layout files are silently ignored, although a debug message is printed in the error log.

There are still a few known issues left:
1) Some servers have problems doing the md5 sum of files
2) (low priority) Some servers do not print the file lists but otherwise function normally.

To install the new version simply overwrite the existing files with the new ones from the tar.gz-archive. If you do this there is no need to re-register at the main burp site.
If you experience any problems upgrading, or have any suggestions or ideas, please post here.
ID: 1952 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Admire

Send message
Joined: 23 Mar 05
Posts: 108
Credit: 94,266
RAC: 0
Message 1954 - Posted: 20 Nov 2005, 14:37:02 UTC

The folders are not created when you extract the files.
When life is to big to tackle, kick it in the shins.
ID: 1954 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
FreeLarry

Send message
Joined: 10 Oct 04
Posts: 42
Credit: 1,689,701
RAC: 0
Message 1958 - Posted: 21 Nov 2005, 10:35:30 UTC - in response to Message 1954.  

The folders are not created when you extract the files.


?? Had no problems unzipping in windows using ultimate zip program. The directories were created and then i copy and pasted the files to existing mirror directory.

Did change the time from 80 in mirror.php to 14400 as windows seems to need longer for file xfers operations. That was what i finally had to resort to during the last run.

Larry
ID: 1958 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Bananeweizen

Send message
Joined: 22 Jul 05
Posts: 3
Credit: 5,028
RAC: 0
Message 1960 - Posted: 22 Nov 2005, 11:27:36 UTC

I\'ve installed the upgrade and I took the chance to investigate my mirror (\"Rechenkraft.net\"). The mirror overview says it\'s active and has transferred about 1 GB already, but the mirror details page says that _all_ data files are in state \"missing\". I checked that against the files on the server and can say that this is wrong. I verified that file 5937 (blender_4.20_i686-pc-linux-gnu.zip) is on my mirror and has the correct size.
Can I do something to manually start a synchronization of my mirror with the server?

And another issue: On my mirror site http://217.160.138.71/mirror/burp/mirror.php 3 files are listed as type \"ERROR: corrupted section header size\". That string can be found in their corresponding .info files, so could this be a problem on your side?

Ciao, Michael.
ID: 1960 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
baracutio
Project donor

Send message
Joined: 29 Mar 05
Posts: 96
Credit: 174,604
RAC: 0
Message 1961 - Posted: 23 Nov 2005, 1:45:39 UTC

@bananeweizen:
you have really upgraded the mirror software to version 1.93? the page with your mirror details shows this:
Mirror -> Rechenkraft.net BURP mirror
Mirror software version -> BOINC-mirror v1.92

so something is going wrong there... maybe the website is not up to date?!

greets from germany;)



mfg bara
ID: 1961 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
malacy

Send message
Joined: 27 Apr 05
Posts: 18
Credit: 9,800
RAC: 0
Message 1962 - Posted: 23 Nov 2005, 5:58:50 UTC - in response to Message 1961.  

likewise: my mirror had some port routing problems (trying to run something behind apache that also wanted to be on port 80 caused my burp instance to jump to :810 for a while. it\'s back together now, upgraded to 1.93. could you start a forced mirror update? thanks :)
ID: 1962 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
FreeLarry

Send message
Joined: 10 Oct 04
Posts: 42
Credit: 1,689,701
RAC: 0
Message 1963 - Posted: 23 Nov 2005, 8:29:10 UTC - in response to Message 1961.  

@bananeweizen:
you have really upgraded the mirror software to version 1.93? the page with your mirror details shows this:
Mirror -> Rechenkraft.net BURP mirror
Mirror software version -> BOINC-mirror v1.92

so something is going wrong there... maybe the website is not up to date?!

greets from germany;)



mfg bara



Like wise my page hasn\'t updated to reflect 1.93, also noted that burp on that machine no longer seems able to connect to burp.boinc.dk to fetch master file. it is able to reach site thru the 5.2.7 boinc link on project page tho.

Larry

ID: 1963 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Admire

Send message
Joined: 23 Mar 05
Posts: 108
Credit: 94,266
RAC: 0
Message 1966 - Posted: 23 Nov 2005, 15:14:09 UTC

My mirror is also reporting the wrong ver. after I updated it.
When life is to big to tackle, kick it in the shins.
ID: 1966 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4571
Credit: 2,100,463
RAC: 8
Message 1969 - Posted: 23 Nov 2005, 17:29:06 UTC
Last modified: 23 Nov 2005, 17:31:31 UTC

Hi there everyone.

First of all I\'d like to point out that the mirrors are not yet on the 10min update cycle yet, so they are only updated every once in a while.
If you suspect that the data shown on the master server is out of date please check the \"Last Ack\" time shown on the frontpage on the master. This indicates when your mirror information was last updated. I just ran a forced update a few moments ago.

This leads me to the second point: Some mirrors did not \"survive\" the big update when all the mirrors were resynced (see the recent news) due to errors in the mirror software. Most of those errors should have been fixed in this version (1.93). However, in an attempt to properly debug this I will manually go through the update of the problematic mirrors to see if any errors pop up. I can\'t tell you exactly when, but it will be during the next couple of weeks.
The affected mirrors are all those with high ack-times as well as all deactivated mirrors.
While doing this I will also purge all non-existant mirrors (if your mirror happens to be unlucky enough to be offline when I test it you can always reregister).

About \"ERROR: corrupted section header size\" - this is a known error that doesn\'t affect the mirrors at all. It is because the linux installation file was compressed using UPX and somehow this causes the filetype to become a tad difficult to determine =)

I hope to have a much larger percentage of the currently deactivated mirrors online soon. Time will tell, though, perhaps more debugging is needed.
ID: 1969 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4571
Credit: 2,100,463
RAC: 8
Message 1975 - Posted: 23 Nov 2005, 19:42:31 UTC
Last modified: 23 Nov 2005, 19:45:40 UTC

As part of this upgrade you can now see the latest sync-log for your mirror (available under details on the master). This log can help you identify issues like permission problems, wrong server configuration etc. These errors will show up as fatal errors (usually at the very bottom of the page).

The sync logs are mostly random jibberish, but they do contain a lot of good information when it comes to debugging mirrors.

Not all mirrors have been contacted since this feature was turned on, so some mirrors may return \"No such file\" when pressing the sync-log link.

If the tag shows at the bottom of the file, everything is probably quite fine with that particular mirror.
ID: 1975 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
FreeLarry

Send message
Joined: 10 Oct 04
Posts: 42
Credit: 1,689,701
RAC: 0
Message 1976 - Posted: 23 Nov 2005, 20:02:13 UTC - in response to Message 1975.  

As part of this upgrade you can now see the latest sync-log for your mirror (available under details on the master). This log can help you identify issues like permission problems, wrong server configuration etc. These errors will show up as fatal errors (usually at the very bottom of the page).

The sync logs are mostly random jibberish, but they do contain a lot of good information when it comes to debugging mirrors.

Not all mirrors have been contacted since this feature was turned on, so some mirrors may return \"No such file\" when pressing the sync-log link.

If the tag <synced/> shows at the bottom of the file, everything is probably quite fine with that particular mirror.


just had looked at mine before your post - looks good. Noted in one of the logs something about a .keep file with unknown extentsion that i\'m baffled about and molehill still will not fetch a master file for boinc for burp.

Larry
ID: 1976 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
PovAddict
Avatar

Send message
Joined: 25 Apr 05
Posts: 347
Credit: 4,618
RAC: 0
Message 1991 - Posted: 25 Nov 2005, 1:42:39 UTC

Janus,

I did something stupid while updating and I lost my config.php and keys.

Do I need to register my mirror again? If answer is yes, please REMOVE my mirror so you don\'t get that unused entry in the list.


ID: 1991 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
FreeLarry

Send message
Joined: 10 Oct 04
Posts: 42
Credit: 1,689,701
RAC: 0
Message 1997 - Posted: 26 Nov 2005, 9:49:45 UTC - in response to Message 1976.  


molehill still will not fetch a master file for boinc for burp.


Well i finally got it to communicate with your site. spomething to do with the boinc manager i think as had to exit boinc and restart before it would connect to your site.

Larry
ID: 1997 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4571
Credit: 2,100,463
RAC: 8
Message 1999 - Posted: 26 Nov 2005, 11:22:04 UTC - in response to Message 1991.  

Theoretically it will reuse the mirror entry if you use the same URL and name. Haven\'t tested it yet though.

Let me know how it works.
ID: 1999 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
PovAddict
Avatar

Send message
Joined: 25 Apr 05
Posts: 347
Credit: 4,618
RAC: 0
Message 2004 - Posted: 27 Nov 2005, 1:22:53 UTC

I can\'t register again. I deleted all related to BURP mirror I had in my DocumentRoot, downloaded the .tar again and extracted it into web/burp_mirror/ (I set Apache with an alias so it is /burp/).

Then I followed the registration and right here I get an error \"Unable to handle request\"

The content found at the URL (http://povaddict.dnsalias.com/burp/mirror.php) does not seem to match that of a BURP mirror in the setup phase, please inspect and correct it. You may need to delete your mirror configuration file if you have already created such a file.

HTTP/1.1 200 OK
Date: Sun, 27 Nov 2005 01:20:39 GMT
Server: Apache/2.0.54 (Win32) PHP/5.0.3 SVN/1.2.3 DAV/2
X-Powered-By: PHP/5.0.3
Content-Length: 72
Connection: close
Content-Type: text/html

1.93 - Fatal Error: No configuration file could be found at ./config.php

This seems contradictory: \"You may need to delete your mirror configuration file\" and the error is because there isn\'t one!

ID: 2004 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
baracutio
Project donor

Send message
Joined: 29 Mar 05
Posts: 96
Credit: 174,604
RAC: 0
Message 2008 - Posted: 27 Nov 2005, 20:03:04 UTC

why is the status of some mirrors \'waiting\'? do they wait for the next resynch with the main servers?



mfg bara
ID: 2008 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4571
Credit: 2,100,463
RAC: 8
Message 2013 - Posted: 28 Nov 2005, 10:11:03 UTC
Last modified: 28 Nov 2005, 10:11:21 UTC

Interesting error, PovAddict. It runs a bit deeper than just a missing config file.
For now you can add the text \"BOINC-mirror\" right at the first byte of the file (mirror.php) when you need to get past the first check. Remember to remove it again after you pass the check.
I will have to release a new mirrror version to fix this.

The reason why some mirrors are waiting is because they are waiting to become activated. This is an ongoing process that takes a little time because each of them has to be evaluated and (currently, because we are still in pre-alpha) some of them face problems that need to be corrected in the code.
ID: 2013 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
PovAddict
Avatar

Send message
Joined: 25 Apr 05
Posts: 347
Credit: 4,618
RAC: 0
Message 2019 - Posted: 28 Nov 2005, 19:00:54 UTC - in response to Message 2013.  

Interesting error, PovAddict. It runs a bit deeper than just a missing config file.
For now you can add the text \"BOINC-mirror\" right at the first byte of the file (mirror.php) when you need to get past the first check. Remember to remove it again after you pass the check.
I will have to release a new mirrror version to fix this.



Glad to help. This is the exact reason why I wanted to register a mirror. My computer is not on 24/7 and I have onlt 128k upstream. But I\'m helping find bugs!

Finished the process without problems. I added BOINC-mirror, did the first test. I set the same name and URL to the same as before (as you said in a previous post). If it didn\'t work, delete manually the other mirror entry, I think it will make a big mess if it\'s still there.

I removed the text from the file then and got the config files. I\'ll let you know if I have any problems.
ID: 2019 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Peter M. Nielsen
Project donor

Send message
Joined: 14 Mar 05
Posts: 130
Credit: 307
RAC: 0
Message 2031 - Posted: 30 Nov 2005, 17:33:31 UTC - in response to Message 1954.  

The folders are not created when you extract the files.

Hej Janus

After upgrading my Faramir-mirror errors out with a md5-checksum error. Any ideas why?

- Peter
ID: 2031 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4571
Credit: 2,100,463
RAC: 8
Message 2039 - Posted: 2 Dec 2005, 0:59:07 UTC - in response to Message 2031.  

After upgrading my Faramir-mirror errors out with a md5-checksum error. Any ideas why?

Hm... A stream error while trying to fetch the correct files from the main server. A particularly nasty one as well - there\'s no workaround for this one so you\'ll have to wait for the next release.
We are really getting somewhere in the bugcatching league this release.
ID: 2039 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Server backend and mirrors : Mirror software version 1.93 upgrade