mirror traffic estimtion

Message boards : Server backend and mirrors : mirror traffic estimtion
Message board moderation

To post messages, you must log in.

AuthorMessage
|MatMan|

Send message
Joined: 22 Nov 05
Posts: 16
Credit: 1,896,265
RAC: 0
Message 7165 - Posted: 20 Dec 2007, 9:47:57 UTC

According to the mirror stats my mirror should have transferred around 7 GB from yesterday (19th december) to today (20th december), as can be seen here.

According to my ISPs traffic stats at most 0.5 GB have been actually transferred. As this is a huge difference you might want to look into it.

Could it be related to the torrent downloads?
ID: 7165 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7166 - Posted: 20 Dec 2007, 11:25:17 UTC

Yes, this difference is due to the way that the traffic is meassured. Currently BURP overestimates the used bandwidth because all traffic is measures in a way that counts a started download as a full download. Your ISP measures the actual number of bytes that were transfered - which may be less in case of aborted downloads.
Originally the mirror software tried to measure the actual number of bytes delivered over the network, but this number turned out to be highly inaccurate (in the opposite direction) because not all of the apache servers supported the way that the code hooked into the webserver shutdown functions (the functions called when a user aborts a download).
It was decided that it was better to slightly overshoot the measurement rather than underestimate it (in the latter case it would open up the risk of spending more network bandwidth than what the mirror owner allows). This is a temporary solution until a better (possibly Java-based or available in both Java and PHP) mirror system is developed.

Another interesting fact that has turned up is that there is a portion of the mirrors that do not meet the uptime requirements at all. This also calls for a new kind of more flexible software to control and run the mirror system.
ID: 7166 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
|MatMan|

Send message
Joined: 22 Nov 05
Posts: 16
Credit: 1,896,265
RAC: 0
Message 7169 - Posted: 21 Dec 2007, 15:37:29 UTC - in response to Message 7166.  
Last modified: 21 Dec 2007, 15:40:08 UTC

Another interesting fact that has turned up is that there is a portion of the mirrors that do not meet the uptime requirements at all. This also calls for a new kind of more flexible software to control and run the mirror system.

Do you mean my mirror was down so the downloads failed? I think it should be quite reliable & fast.

Even if some downloads failed it would require 100s or 1000s of failed downloads to get somewhere near 7 GB traffic and all that in about 2 hours time. For the rest of the month the mirror actually transferred around 4 GB. So this is nearly twice as much false-traffic in a very short timeframe -> can\'t be caused by a short downtime of the server at that time (while I think it wasn\'t down anyway).

You are right, it is nothing critical at the moment. But because there was such a big difference between estimated and actual traffic I thought it was worth mentioning it.
ID: 7169 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Janus
Volunteer moderator
Project administrator
Avatar

Send message
Joined: 16 Jun 04
Posts: 4574
Credit: 2,100,463
RAC: 8
Message 7170 - Posted: 21 Dec 2007, 15:49:31 UTC - in response to Message 7169.  

Do you mean my mirror was down so the downloads failed? I think it should be quite reliable & fast.

No, I was reffering to downloads that were started but then cancelled by the user or program.
ID: 7170 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Server backend and mirrors : mirror traffic estimtion