Skipping lag?

[gentvdinner]

Im having weird issues with source. I run a server hosting co for CS 1.6 and CS Source game servers. Our 1.6 servers run fine, our sources, sometimes not so much. As your playing a weird Skipp happens. where your running and all of a sudden your like 10 paces behind where you were. The server is a dual Xeon 2.8GHZ with 2GB Ram and running Red Hat fedora core 2. At our heaviest skipping times ive logged on to 1.6 and noticed beautifull smooth gameplay. Also note, CPU Usage seems normal with servers no more than 20-25% usage full. We only are currently running 4 source servers at this time. Note that 2 of these arent even packed when we experience this. Also, pings don't spike or rise, no loss or choke either. Any ideas?

[arnoeldchen]

This is exactly what I have to face on my Dual Opteron Server. Pings are fine, CPUs are low (30-50 percent), network uplink is multi-carrier and has excellent quality, but I have these "warps". Distro is also a FC2. Did you find out something meanwhile?

Arnold

[cryotek]

I would try renicing the process, find the PID of your hlds_i686 or hlds_amd and renicing it to -20
for example, say your hlds binary was 12456
you would then run

Code:

renice -20 12456

IIRC you can only do this as root.

Renicing it with -20 should make it use more resources.

You can also look into setting the sys_ticrate when you start up the server, i recall Alfred, one of the devs at valve, saying a ticrate of 33 or 66 should be enough for Source, to do this, add -sys_ticrate 33 to your startup line.
Another thing you might wanna consider, is, if possible, reducing the size of your kernel by removing code that isn't needed, this will also free up some memory.

Let me know how this runs.

[arnoeldchen]

Server already uses a tickrate of 66 (syntax is -tickrate), fps_max of 1000 and is reniced with -10 by a cronjob (more doesn't help the case, because it would superseed system events). Free mem is a 1,4-1,5 GB. No luck so far.

Arnold

[cryotek]

Are you running x86 version of FC2? or amd64 version?

[arnoeldchen]

Server runs:

Linux 2.6.10-1.771_FC2smp #1 SMP Mon Mar 28 01:05:47 EST 2005 x86_64 x86_64 x86_64 GNU/Linux

free
total used free shared buffers cached
Mem: 2055912 1839000 216912 0 167556 1059980
-/+ buffers/cache: 611464 1444448
Swap: 1020116 0 1020116

top - 12:25:45 up 3 days, 5:40, 1 user, load average: 0.51, 0.43, 0.38
Tasks: 82 total, 1 running, 81 sleeping, 0 stopped, 0 zombie
Cpu0 : 48.0% us, 0.3% sy, 0.0% ni, 51.0% id, 0.0% wa, 0.0% hi, 0.7% si
Cpu1 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 2055912k total, 1839512k used, 216400k free, 167564k buffers
Swap: 1020116k total, 0k used, 1020116k free, 1059972k cached

I really have no clue what causes these lags. As many people claim to have them it's most presumably the source server itself. But among those who complain there are a lot of guys with FC2 as a distro and SMP machines (real SMP or HT).

Arnold

[cryotek]

I haven't a clue either, if i use linux, I use Gentoo, and i never have a problem with it.

[gentvdinner]

I have FC2 and plenty of ram. I hardly have any 1.6 servers running anymore. I never used the ticrate option. what exactly does that do for source? is this option only for hlds? My hlds servers run fine, and are already reniced at -20. My source is set for default of 0. I have dual xeon 2.8s with 2gb of ram. at least a gig is free at all times.

[efk]

I'm not that familiar with FC but if its anything like redhat, it installs everything under the sun. You may want to go through and turn off services that you're not using and see if that makes a difference. Generally, this includes all the RPC stuff that most people dont use anymore and any other things you may not be using.

Just a thought.

[gentvdinner]

yeah there IS alot of crap on. Ive turned alot of stuff i dont use off. The thing is, cpu usage really doesnt get high to me. I have one server @ 20 players full and maybe i have 20% cpu usage for it. Should that deem you bouncing all over the place in a source game, esp when theirs great pings, no loss or choke and everyone has it at once. Its weird. My 1.6 server doesnt do that. And our last remaining client whose 1.6 who hasnt switched to source yet has no complaints. Im stumped.

[cryotek]

I meant to say renice your srcds processes. Try setting -ticrate 66 i can't remember EXACTLY what it does, i'll get back to you on that. have yuo tried shutting down ALL servers and running just 1 source server to see how it runs?

I would also try going through the kernel source and weeding out stuff you don't need and rebuild your kernel. the smaller your kernel is, the less memory required to cache it.

[efk]

CPU and memory usage matters, but at the instant people are seing the lag it might be spiking up. Have you just sat and watched top and noted an application jumping your resource usage? Perhaps theres a cronjob thats running every so often that makes your server "skip." You might want to have people make a note of when the skips happen, is it every 10 minutes, or is it a bit more random? This might give you some insight as to whats happening.

Its not all about cpu and memory usage, you have to worry about system I/O as a whole. Multitasking in a *nix environment gets a bit sticky when you're relating it to an application that requires performance like a gaming server, so saving resources where you can will definately benefit you in the long run. Renicing an application is certainly one way to do it, but if you "lean up" your install of FC i would imagine that renicing the application isn't necessary.

Also you might want to compare notes with other FC users to see common occurances that might be contributing to the problem. IE, what kernel modules you have loaded, kernel options, fs type - etc. Perhaps srcds just doesn't like a specific kernel option or filesystem type.

[gentvdinner]

I Apreciate these tips. Ive reniced all my sources to -10 and set all my hlds processes to 20 which is lowest. Noones reported lag on those server and my source servers run alot smoother than they did before. Which is making for alot of happier customers. Ill also watch top for a longer period of time and see whats popping up to cause this spiking. I mean ive had over 100 users on at one time on this server and theres no lag or spikes, except in some sort of interval which would def say to me a system process or cronjob is popping in under severe load.

[gentvdinner]

Heres a list of my daily cronjobs i found:

/etc/cron.daily/yum.cron
/etc/cron.daily/00-logwatch
/etc/cron.daily/00-makewhatis.cron
/etc/cron.daily/rpm
/etc/cron.daily/0anacron
/etc/cron.daily/logrotate
/etc/cron.daily/gcp
/etc/cron.daily/tmpwatch
/etc/cron.daily/slocate.cron
/etc/cron.daily/prelink

Any ideas on what half of this crap is and what could be disabled?

[cryotek]

yum.cron i assume will run yum to update your files. slocate updates your file cache for using locate logrotate should be sel explanatory. technically, you can probably disable all of these. but they really wouldn't interfere if they only run once a day.