FTP Server for Fast Map Download

[css]

@xlfearlx
Try apache: http://httpd.apache.org/download.cgi

It's not that hard to set up.

@dualcore & others
The in-game download is - what I've heard - limited to the "rate" of the cvar settings. Some say 30000 is the maximum value there, so it means 30k/s is the maximum download speed. Anyway the in-game download is extremely slow. Now if xlfearlx has 2 Mbps upload, he gets faster uploads by setting up local web server serving the files.

The downside of having your own fastdl url on a low-bandwidth server is that one map download will saturate the connection and it will be laggy on the server for a while.

[xlfearlx07]

ty css, I'll give apache a try. It would be even more awesome if you could provide a link to a guide or write one yourself for me... Otherwise I'm sure I'll find a guide on how to use it somewhere.

What you said about the in-game download being limited to rates (30k/s) makes alot of seance. I wounder why steam or mani hasn't found a way to fix that, thus removing the need for fast download servers. Probably due to the lag on the game server that it would understandably create while the map downloads are taking place. I greatly appreciate your response css, it made the most seance of all the answers I've gotten thus far. No offense to everyone else, he just explained it well.

[Nisd]

If you are on windows i actually suggest you to add the feature IIS as it is very simple to set up with Virtual directories

[xlfearlx07]

Nisd:
Good idea thanks. ( :

css: I installed and have a decent grasp on how to set up an apache server. You weren't joking when you said it's easy to set up. I didn't even use a guide haha

[css]

Great. Now you can write the tutorial for someone else

[xlfearlx07]

(10-16-2009, 06:45 PM)css Wrote:  Great. Now you can write the tutorial for someone else

I've been set up. LOL Well if I get the fast download working I might just do a tutorial. ( : Thanks again man, or girl. You're name doesn't exactly give you away haha

[Drocona]

Oh you nerds

[Advanced]: Set up sv_downloadurl for faster downloads

1: sv_downloadurl is for HTTP
2: Make sure there is NO trailing slash, it won't work
3: It's not smart running a fastDL server from the same connection the gameserver is on
4: It's even less smart doing the above on your limited home connection
5: FastDL will be faster regardless of where you host it

Running a fastDL server from the same connection the gameserver is on is BAD because:
1 => When someone connects to the server and initiates a download, ALL bandwidth available will be used to try and MAX OUT that clients download speed. If that guy is on a 100mbit connection and your gameserver is too, the client WILL download with 100mbit and your server will lag to death (cause theres no bandwidth available)
2 => Same goes for home connection, only available bandwidth hardly is enough to support ONLY the gameserver.
3 => File transfers cost CPU power, that CPU power SHOULD be with the gameserver but it will be used up by the filetransfer since that has higher priorities.

Also, direct gameserver downloads on the Source engine games are limited on: 30KB/s download
Orangebox engine games are limited on 100KB/s download

The above values are the MAXIMUM, they are however directly related to your sv_maxrate settings. sv_maxrate 10000 will make it 10KB/s max. sv_maxrate 0 will make it the max values stated above.

[xlfearlx07]

Dracona:
I appreciate the extra info on the slash (which I already knew) but other than the expensive router a game server business would have, my home network is a very powerful and runs without a hiccup when the server is full and up to 3 people are surfing the web. I'm sure it could handle more but I don't have any more computers to test that theory. As for it being bad to set up the dl server on the same connection as the game server, that's what priority settings are for, so that is not an issue. How do you think the game server companies do it. Anyway, I'll be sure to post letting everyone know how well my fast download server performed, if it even works.

[Drocona]

I wasn't pointing at being able to "handle it". Even with priority settings and one hell of a connection, you will eventually have the chaos moment of having 4 people connecting all wanting to download from the fastdl server completely sapping every single bit/sec out of your connection. Which results in the server not getting enough, also keep in mind, it's important to leave at least 20% of your bandwidth unused to avoid pings rising to ~80ms instead of normal 10-40ms.
I'm pretty certain that when my neighbour and me join your server the same time you connection is maxed out cause we are pulling 50+mbit/sec each, up to 150mbit total

Even if you are not maxing your connection, the ping WILL rise and it doesn't go unnoticed.

About the gameserver companies, I'm 100% certain that every GSP that actually knows what they are doing have at least 1 seperate server for the fastdl hosting of their clients.

[css]

(10-17-2009, 02:33 AM)Drocona Wrote:  About the gameserver companies, I'm 100% certain that every GSP that actually knows what they are doing have at least 1 seperate server for the fastdl hosting of their clients.

No. You're wrong.

Do you think GSPs have separate connection for servers and downloads? Do you think GSPs first buy Co-location / 10 racks / 500 Mbit for $15000/mo and then buy another DL server for $500 just to avoid having their bandwidth saturated by 4 players?

I'm hosting server with 100 Mbit connection and I'm hosting web server on the same server. The server is popular and the website is popular. The connection is never the bottleneck.

You can see my signature and give it a try. Join when there is custom map and download all you can. If you succeed doing the 100 Mbit = 12.5 MB / s, you'll download the 23 MB de_nightfever.bsp.bz2 in about 2 seconds. I bet it takes longer than that for TCP connection to even do the "slow start" to the actual speed.

Get real. If you have 100 Mbps connection you can host your fastdl server with no problems on the same server as your game server.

Feel free to try to download stuff from our web to prove your point that you're capable of saturating the web and killing our connection. We have 100 Mbps. In theory it's possible to use all the traffic with 100 Mbps connection, but in practice it never happens.

Xlfearlx's case is totally different because he knows he's on home connection. He can accept being laggy for about 5-10 seconds while players download the map in the beginning. After that there is no lag because everybody have the map.

Quote:3 => File transfers cost CPU power, that CPU power SHOULD be with the gameserver but it will be used up by the filetransfer since that has higher priorities.

This is not true. It's ridiculous. How much do you think file transfers take CPU? Have you ever upgraded your CPU because your downloads were too slow? Doh.

[Drocona]

(10-17-2009, 05:11 AM)css Wrote:  No. You're wrong.

Do you think GSPs have separate connection for servers and downloads? Do you think GSPs first buy Co-location / 10 racks / 500 Mbit for $15000/mo and then buy another DL server for $500 just to avoid having their bandwidth saturated by 4 players?

Just like a webhosting provider (big) You spread the services over several servers. One app server handles MySQL etc, the other does the HTTP/FTP and another does the E-Mail.
Of course this is a "perfect world" setup and I know that the GSP's that really go for quality instead of quantity work this way.
It really doesn't cost that much for a simple download server, an unmetered 5mbit works fine. It is even in my original businessplan for a GSP.

(10-17-2009, 05:11 AM)css Wrote:  I'm hosting server with 100 Mbit connection and I'm hosting web server on the same server. The server is popular and the website is popular. The connection is never the bottleneck.
<snip>
Get real. If you have 100 Mbps connection you can host your fastdl server with no problems on the same server as your game server.

<snip>
In theory it's possible to use all the traffic with 100 Mbps connection, but in practice it never happens.

I agree that 1-2 years back it was indeed "theory", now the day though most people are on 20mbit+ connections (We are getting ~120mbit here at my house soon, currently at 50mbit) and can fully utilise that amount of bandwidth. If I have 2 or 3 people from my street download at the same time from a single server hosting both the fastDL and the gameserver I can assure you that the gameserver wil lag like hell and the connection is maxed out, giving the downloading people 50mbit or 33mbit respectively.

(10-17-2009, 05:11 AM)css Wrote:  Xlfearlx's case is totally different because he knows he's on home connection. He can accept being laggy for about 5-10 seconds while players download the map in the beginning. After that there is no lag because everybody have the map.

I agree with this, I was speaking in a general way. It most likely works fine for him, but bigger servers bring more people that need to download. Also on a home connection of ~2mbit I'm sure 1 downloader will max that connection with ease, hence why I said it in the first place.

(10-17-2009, 05:11 AM)css Wrote:  This is not true. It's ridiculous. How much do you think file transfers take CPU? Have you ever upgraded your CPU because your downloads were too slow? Doh.

Ever tried to transfer a 50GB Blu ray over Gbit? When I'm transferring at 90-100MB/s one core of my Q9450 reaches a constant 25% usage, if you have a busy server running on a core it will be noticeable, also with smaller amounts of traffic. On an old P3 1.0GHz, the CPU will max out at 18MB/s even.

[loopyman]

So much for discussing an FTP server for fast downloads.

I have tried it before with no luck, I believe you can only use the HTTP protocol for fast downloads.

[xlfearlx07]

I enjoyed the "epic text war". I still have to side with css, but yeah, hopefully I'll have some time tomorrow to try the http server. I have set it all up, just have to change the sv_downloadurl to my http server. I'll report back as soon as I have tested it.

Loopyman,
I'm afraid you're right, FTP doesn't seem to register at all for an srcds server. I still wish steam didn't limit the download speed via maxcmdrate.

[loopyman]

I believe maxrate dictates the download speed to an extent...

But yea, you can run a http server off the same servers as your gameservers, its just about limiting how much bandwidth the HTTP can use, there's ways to limit it, I believe its something you have to tweak in apache.

I haven't tried anything else, I only use IIS or Apache for my web stuff.

[xlfearlx07]

I tested the http fast download server and the results are GREAT! The following results were between the free http web server host I used and my home http web server:

The smaller maps (1mb), I saw the download speeds go up about 1 second... So instead of the regular 3 second download time it was 4 second download time.

The largest map file (40mb) I saw the download speeds go up about 30 seconds. So instead of the original 30 second download time, it was 60 second download time.

I found the results to be pretty good. Seeing as how we don't play many 40mb maps we shouldn't notice the difference in download speed too much. I have yet to test my new fast download server when the game server is full though so I'll report back in a month or so letting you know if it lagged at all when the server was full and people were downloading maps at the same time.