8 replies [Last post]
KnightWolfJK
KnightWolfJK's picture
Offline
Enthusiast
Washington, DC
Last seen: 7 years 6 weeks ago
Washington, DC
Timezone: GMT-5
Joined: 2003-06-10
Posts: 210
Points: 0

This is so off-topic it ain't even funny...

I've recently joined a group of people who are determined to donate some or all of their bandwidth for the hosting of Halo movies for halo.bungie.org. I've been a long time member, it's a great community, and I'm certain it's a good cause. Laughing out loud

They used to push out 3-5gb daily- so far we've amassed enough hosts to total 145gb/month but the trick is to distribute it.

From what I can tell, they've worked out a system to incrementally serve out a new download link for each visitor to the page, theoretically sharing the downloads equally.

The problem I see is that some hosts have 5gb/month limits while others have 30gb/month limits. I've installed some basic bandwidth monitoring scripts on my 3 servers, but can ANYONE suggest how to monitor bandwidth of multiple files on multiple servers from one location? COTS software or complicated installs aren't an option- I'm working with a bunch of random peeps and a quick setup would be best.

I'm thinking either a) a database (standard flatfile, mysql, etc ??)containing the URL's to all the videos and someway to monitor them from one site or b) some type of script on each server that will gather it's bandwidth info in a log file while a script on another server culls all the info and displays it in one spot...

Any of that make any sense? I'm open to any and all suggestions and criticism... And thanks for tolerating something so off-topic. Smile

Diplomacy is the art of letting someone have your way

Tags:
paCkeTroUTer
paCkeTroUTer's picture
Offline
Enthusiast
Melbourne, Australia
Last seen: 1 year 11 weeks ago
Melbourne, Australia
Timezone: GMT+10
Joined: 2003-06-27
Posts: 241
Points: 2

Monitor bandwidth for files across multiple hosts?

One way I would approach this is by:

  1. Calculating the combined total available bandwidth on all servers at a particular point of time(or on clients visit to the download section).
  2. Get the % ratio of available bandwidths for each server.
  3. Calculate from the list of files on each server how much bandwidth they would serve.
  4. Divide that bandwidth by the number of days left in the month.
  5. Search for files on each Server with their file size totalling(approximately) the available bandwidth for the day.
  6. Record those files in a database table with links.
  7. When performing the above Search(point 5) you will have to exclude the files which have already been recorded in the table for that session of the user.
  8. Loop through the next server until you get a list of links to all the files on the different servers.

hmm does this make sense ? Or have I gone way off to your problem ?

[/]

http//melbourne.ug.php.net

KnightWolfJK
KnightWolfJK's picture
Offline
Enthusiast
Washington, DC
Last seen: 7 years 6 weeks ago
Washington, DC
Timezone: GMT-5
Joined: 2003-06-10
Posts: 210
Points: 0

Sounds great...

It makes a lot of sense, and I appreciate your input... Alas, I'm still too n00b to get this going from scratch. Any chance you know of something like this in existence anywhere? My chances of succes are much higher if I can modify something that already exists...

Thanks again

Diplomacy is the art of letting someone have your way

paCkeTroUTer
paCkeTroUTer's picture
Offline
Enthusiast
Melbourne, Australia
Last seen: 1 year 11 weeks ago
Melbourne, Australia
Timezone: GMT+10
Joined: 2003-06-27
Posts: 241
Points: 2

Re: Sounds great...

KnightWolfJK wrote:
It makes a lot of sense, and I appreciate your input... Alas, I'm still too n00b to get this going from scratch. Any chance you know of something like this in existence anywhere? My chances of succes are much higher if I can modify something that already exists...

Thanks again

I am not sure you will find one similar already made as its something someone would custom make(but you never know someone similar to your requirements must have already made one Wink ). Perhaps when I finish my current project(sig's Wink ) , I can give you a hand in developing it. How soon do you need it ?

Anybody else for some comments on my previous post ? Is it a possibilty to achieve what I said ?.... hey ofcourse "Necessity is the Mother of Invention !"

http//melbourne.ug.php.net

KnightWolfJK
KnightWolfJK's picture
Offline
Enthusiast
Washington, DC
Last seen: 7 years 6 weeks ago
Washington, DC
Timezone: GMT-5
Joined: 2003-06-10
Posts: 210
Points: 0

Yesterweek?

Well, technically speaking we started serving out the videos last night, so bandwidth monitoring would come in handy at the present moment.

I have a half-a$$ monitoring page for my three servers set up, but it's rudimentary and based on my best guess of the amount of bandwidth each user will be served. It takes $max_bandwidth divided by $amount_of_bandwidth_per_user, then chooses a gif file to display the approximate amount of bandwidth served. I picked it up at hotscripts and modified it a tad, but as I mentioned it's not really a solution, just a guess.

I found a great script, Advance Bandwith, at perlonline.com. Unfortunately it's $99 per server license and we're currently using 7 servers...

Thanks again packet

Diplomacy is the art of letting someone have your way

paCkeTroUTer
paCkeTroUTer's picture
Offline
Enthusiast
Melbourne, Australia
Last seen: 1 year 11 weeks ago
Melbourne, Australia
Timezone: GMT+10
Joined: 2003-06-27
Posts: 241
Points: 2

Monitor bandwidth for files across multiple hosts?

hey KnightWolfJK, any luck yet ? My PC just crashed..... so I am now going to install XP from scratch on a new disk. Sad its a pain starting all-over again.

http//melbourne.ug.php.net

KnightWolfJK
KnightWolfJK's picture
Offline
Enthusiast
Washington, DC
Last seen: 7 years 6 weeks ago
Washington, DC
Timezone: GMT-5
Joined: 2003-06-10
Posts: 210
Points: 0

Not yet

Nah man, hit a brick wall for the time-being.

Good luck with the rebuild, that's always good fun

Diplomacy is the art of letting someone have your way

Hawk
Offline
Regular
Soton, UK
Last seen: 11 years 1 week ago
Soton, UK
Joined: 2003-07-29
Posts: 7
Points: 0

Monitor bandwidth for files across multiple hosts?

If the servers have suitable control panel software on them such as Ensim, then U could dedicate a site to the downloading from that server and Ensim could measure it ..... don;t know how U would read the values from another server though Sad

BTW: this month I was 600Gb below my pre-paid quota limit !!!!!

I've just started a web hositng company and we are using Ensim Pro 3.5 for Red Hat Linux to mangane and monitor everything.

A better solution maybe to use a PHP page to download each file and get it to read from the actual file and echo it to the browser, it could count how much it passes-through and update your central server accordingly ( also via a php script ).

I could quite easily knock the script up for you if your interested.

Please note .. it will consume more CPU time than a plain download ... so if thats a factor then maybe don't do so, but the impact shouldn't be much.
.... Just ensure output buffering is off !!!!! and U'll have to set the time-to-live in php.ini to be suitable for the script to not get killed during execution.

KnightWolfJK
KnightWolfJK's picture
Offline
Enthusiast
Washington, DC
Last seen: 7 years 6 weeks ago
Washington, DC
Timezone: GMT-5
Joined: 2003-06-10
Posts: 210
Points: 0

Monitor bandwidth for files across multiple hosts?

Unfortunately, I think we're working with too many different server flavors. While about 90% of the servers are *nix-based, not all servers have php installed, and it isn't an option to so do (for example, my cox.net account).

Thanks, I appreciate your feedback

Diplomacy is the art of letting someone have your way