System Admin - Level 2

System Admin - Level 2

How to find the Kernal version from sheel
# uname -r

How to find the apache version from sheel
[root@srv1 ~]# httpd -v
Server version: Apache/2.0.52
Server built: Nov 30 2004 11:22:20


How to search for certain terms in your Domlogs, using SSH.

for files in /usr/local/apache/domlogs/*; do grep "wget" $files; done;



-OR-

cd /usr/local/apache/domlogs
grep wget *
grep lynx *
grep curl *



Replace wget with other file names/terms you might want to search for.

If that takes too long, try doing it one by one:

grep wget a*
grep wget b*
grep wget c*
grep wget d*
grep wget e*
grep wget f*
grep wget g*
grep wget h*
grep wget i*
grep wget j*
grep wget k*
grep wget l*
grep wget m*
grep wget n*
grep wget o*
grep wget p*
grep wget q*
grep wget r*
grep wget s*
grep wget t*
grep wget v*
grep wget w*
grep wget x*
grep wget y*
grep wget z*



Alternatively, if you get an error like "Argument list too long":

for i in `ls /usr/local/apache/domlogs|grep -v 'bytes_log'`; do echo "checking on $i" && grep wget /usr/local/apache/domlogs/$i && grep lynx /usr/local/apache/domlogs/$i && grep curl /usr/local/apache/domlogs/$i; done > /root/grep-domlogs-results.txt


Then simply take a look at this file /root/grep-domlogs-results.txt
to ssh servers by nothingless on Jan 21, 2006
Looking up recent dictionary attacks
Use the code below to look up what words were used in recent dictionary attacks using SSH.

grep "dictionary attack" /var/log/exim_mainlog

to ssh servers by nothingless on Jan 21, 2006
Looking into DOS and DDOS Attacks
A good guide to what to do when your server is attacked.

top -d2
netstat -nap | grep SYN | wc -l
netstat -nap | less



If there are many httpd processes showing up after step 1, you might be under attack. If you get high numbers for the second one, you are almost definitely under attack. Use the third one to see the IP addresses, and then ban them from the server:

iptables -A INPUT -s ip.address -j DROP



Also try the following for fixing stuff:

cd /dev/shm
ls



And delete anything that's not supposed to be there.

locate bindz
locate botnet.txt
locate dc
locate ex0.pl
locate kaiten
locate r0nin
locate udp.pl
locate ...
lsof | grep .,
locate mybot

to ssh servers by nothingless on Jan 21, 2006
Ban IPs from a server
Use the code below to permanently ban an IP address from accessing your server.

iptables -A INPUT -s ip.address -j DROP

to ssh servers by nothingless on Jan 21, 2006
How to tail logs

tail -200 /var/log/exim_mainlog
tail -200 /usr/local/apache/logs/error_log



To watch the log get updated in real time:

tail -f /var/log/messages

to ssh servers by nothingless on Jan 21, 2006
How To Manually Update Cpanel

/scripts/upcp

/scripts/upcp --force

to ssh servers by nothingless on Jan 21, 2006
How To Restart Services
Restart Apache:

service httpd restart



Restart Services:

service chkservd restart



Restart Cpanel:

/etc/init.d/cpanel restart



Restart Bind:

service named start



Run anything in /scripts:

./scriptname

to ssh servers by nothingless on Jan 21, 2006
How To Locate Files

lsof | grep searchterm

to ssh servers by nothingless on Jan 21, 2006
How To Fix Bandwidth Updating
If bandwidth stats aren't updating:

/scripts/runweblogs username
/scripts/runlogsnow

to ssh servers by nothingless on Jan 21, 2006
How To Fix 403 Errors for public_html
If all the public_html folders got their permissions wrong:

chmod 755 /home/*/public_html

to ssh servers by nothingless on Jan 21, 2006
How To Empty /var
If /var is too full:

cd /var
du -sh *



If the log directory is the problem:

rm -f /var/log/*.1
rm -f /var/log/*.2
rm -f /var/log/*.3
rm -f /var/log/*.4



(The /var/log directory contains archived files that always end with a number: exim_mainlog.1. Any file ending with a number can be safely deleted.)

If the problem is with the exim_mainlog being too large, try rotating the logs:

/usr/sbin/logrotate -vf /etc/logrotate.conf



If you get an error about a duplicate log entry:

cd /etc/logrotate.d
rm -rf httpd.rpmorig.log



And try the rotate again.

If the problem is in spool:
System Admin - Level 2

How to find the Kernal version from sheel
# uname -r

How to find the apache version from sheel
[root@srv1 ~]# httpd -v
Server version: Apache/2.0.52
Server built: Nov 30 2004 11:22:20


How to search for certain terms in your Domlogs, using SSH.

for files in /usr/local/apache/domlogs/*; do grep "wget" $files; done;



-OR-

cd /usr/local/apache/domlogs
grep wget *
grep lynx *
grep curl *



Replace wget with other file names/terms you might want to search for.

If that takes too long, try doing it one by one:

grep wget a*
grep wget b*
grep wget c*
grep wget d*
grep wget e*
grep wget f*
grep wget g*
grep wget h*
grep wget i*
grep wget j*
grep wget k*
grep wget l*
grep wget m*
grep wget n*
grep wget o*
grep wget p*
grep wget q*
grep wget r*
grep wget s*
grep wget t*
grep wget v*
grep wget w*
grep wget x*
grep wget y*
grep wget z*



Alternatively, if you get an error like "Argument list too long":

for i in `ls /usr/local/apache/domlogs|grep -v 'bytes_log'`; do echo "checking on $i" && grep wget /usr/local/apache/domlogs/$i && grep lynx /usr/local/apache/domlogs/$i && grep curl /usr/local/apache/domlogs/$i; done > /root/grep-domlogs-results.txt


Then simply take a look at this file /root/grep-domlogs-results.txt
to ssh servers by nothingless on Jan 21, 2006
Looking up recent dictionary attacks
Use the code below to look up what words were used in recent dictionary attacks using SSH.

grep "dictionary attack" /var/log/exim_mainlog

to ssh servers by nothingless on Jan 21, 2006
Looking into DOS and DDOS Attacks
A good guide to what to do when your server is attacked.

top -d2
netstat -nap | grep SYN | wc -l
netstat -nap | less



If there are many httpd processes showing up after step 1, you might be under attack. If you get high numbers for the second one, you are almost definitely under attack. Use the third one to see the IP addresses, and then ban them from the server:

iptables -A INPUT -s ip.address -j DROP



Also try the following for fixing stuff:

cd /dev/shm
ls



And delete anything that's not supposed to be there.

locate bindz
locate botnet.txt
locate dc
locate ex0.pl
locate kaiten
locate r0nin
locate udp.pl
locate ...
lsof | grep .,
locate mybot

to ssh servers by nothingless on Jan 21, 2006
Ban IPs from a server
Use the code below to permanently ban an IP address from accessing your server.

iptables -A INPUT -s ip.address -j DROP

to ssh servers by nothingless on Jan 21, 2006
How to tail logs

tail -200 /var/log/exim_mainlog
tail -200 /usr/local/apache/logs/error_log



To watch the log get updated in real time:

tail -f /var/log/messages

to ssh servers by nothingless on Jan 21, 2006
How To Manually Update Cpanel

/scripts/upcp

/scripts/upcp --force

to ssh servers by nothingless on Jan 21, 2006
How To Restart Services
Restart Apache:

service httpd restart



Restart Services:

service chkservd restart



Restart Cpanel:

/etc/init.d/cpanel restart



Restart Bind:

service named start



Run anything in /scripts:

./scriptname

to ssh servers by nothingless on Jan 21, 2006
How To Locate Files

lsof | grep searchterm

to ssh servers by nothingless on Jan 21, 2006
How To Fix Bandwidth Updating
If bandwidth stats aren't updating:

/scripts/runweblogs username
/scripts/runlogsnow

to ssh servers by nothingless on Jan 21, 2006
How To Fix 403 Errors for public_html
If all the public_html folders got their permissions wrong:

chmod 755 /home/*/public_html

to ssh servers by nothingless on Jan 21, 2006
How To Empty /var
If /var is too full:

cd /var
du -sh *



If the log directory is the problem:

rm -f /var/log/*.1
rm -f /var/log/*.2
rm -f /var/log/*.3
rm -f /var/log/*.4



(The /var/log directory contains archived files that always end with a number: exim_mainlog.1. Any file ending with a number can be safely deleted.)

If the problem is with the exim_mainlog being too large, try rotating the logs:

/usr/sbin/logrotate -vf /etc/logrotate.conf



If you get an error about a duplicate log entry:

cd /etc/logrotate.d
rm -rf httpd.rpmorig.log



And try the rotate again.

If the problem is in spool:

cd /var/spool/exim/msglog
rm -rf *

to ssh servers by nothingless on Jan 21, 2006
How To Empty /usr
How To Empty /usr

cd /usr/local/apache/domlogs/
rm -rf *.*
/scripts/restartsrv httpd

to ssh servers by nothingless on Jan 21, 2006
How To Empty /backup
If /backup/ is too full:

cd /backup/cpbackup/monthly/
rm -f *.gz
/scripts/restartsrv httpd



Check the space after this, and it should be fine.
to ssh servers by nothingless on Jan 21, 2006
How To Fix Incorrect Disk Space
How To Fix Incorrect Disk Space

/scripts/fixquotas
/scripts/updatemysqlquota

to ssh servers by nothingless on Jan 21, 2006
How To Turn Off/On Stats For One Account
http://forums.cpanel.net/showthread.php?t=15967&highlight=urchin

You can edit the:

pico /var/cpane/users/accountname



file and add settings for the stats packages.

skipanalog=1
skipawstats=1
skipwebalizer=1



That will turn them off and override the server setttings.

To update the stats now:

/scripts/runweblogs username
/scripts/runlogsnow

to ssh servers by nothingless on Jan 21, 2006
How To Fix MySQL Error 28
MySQL: 1030: got error 28 from server handler

cd /tmp
df -i /tmp
df -h /tmp



Delete anything that’s not supposed to be there.
Stop all databases

/etc/rc.d/init.d/chkservd stop
/etc/rc.d/init.d/mysql stop



Then fix tables:

cd /var/lib/mysql



Check each letter for errors:

myisamchk -cs a*/*.MYI



Repair where necessary:

myisamchk -r a*/*.MYI
myisamchk -r b*/*.MYI
myisamchk -r c*/*.MYI
myisamchk -r d*/*.MYI
myisamchk -r e*/*.MYI
myisamchk -r f*/*.MYI
myisamchk -r g*/*.MYI
myisamchk -r h*/*.MYI
myisamchk -r i*/*.MYI
myisamchk -r j*/*.MYI
myisamchk -r k*/*.MYI
myisamchk -r l*/*.MYI
myisamchk -r m*/*.MYI
myisamchk -r n*/*.MYI
myisamchk -r o*/*.MYI
myisamchk -r p*/*.MYI
myisamchk -r q*/*.MYI
myisamchk -r r*/*.MYI
myisamchk -r s*/*.MYI
myisamchk -r t*/*.MYI
myisamchk -r u*/*.MYI
myisamchk -r v*/*.MYI
myisamchk -r w*/*.MYI
myisamchk -r x*/*.MYI
myisamchk -r y*/*.MYI
myisamchk -r z*/*.MYI



Turn everything back on:

/etc/rc.d/init.d/chkservd start
/etc/rc.d/init.d/mysql start


cd /var/spool/exim/msglog
rm -rf *

to ssh servers by nothingless on Jan 21, 2006
How To Empty /usr
How To Empty /usr

cd /usr/local/apache/domlogs/
rm -rf *.*
/scripts/restartsrv httpd

to ssh servers by nothingless on Jan 21, 2006
How To Empty /backup
If /backup/ is too full:

cd /backup/cpbackup/monthly/
rm -f *.gz
/scripts/restartsrv httpd



Check the space after this, and it should be fine.
to ssh servers by nothingless on Jan 21, 2006
How To Fix Incorrect Disk Space
How To Fix Incorrect Disk Space

/scripts/fixquotas
/scripts/updatemysqlquota

to ssh servers by nothingless on Jan 21, 2006
How To Turn Off/On Stats For One Account
http://forums.cpanel.net/showthread.php?t=15967&highlight=urchin

You can edit the:

pico /var/cpane/users/accountname



file and add settings for the stats packages.

skipanalog=1
skipawstats=1
skipwebalizer=1



That will turn them off and override the server setttings.

To update the stats now:

/scripts/runweblogs username
/scripts/runlogsnow

to ssh servers by nothingless on Jan 21, 2006
How To Fix MySQL Error 28
MySQL: 1030: got error 28 from server handler

cd /tmp
df -i /tmp
df -h /tmp



Delete anything that’s not supposed to be there.
Stop all databases

/etc/rc.d/init.d/chkservd stop
/etc/rc.d/init.d/mysql stop



Then fix tables:

cd /var/lib/mysql



Check each letter for errors:

myisamchk -cs a*/*.MYI



Repair where necessary:

myisamchk -r a*/*.MYI
myisamchk -r b*/*.MYI
myisamchk -r c*/*.MYI
myisamchk -r d*/*.MYI
myisamchk -r e*/*.MYI
myisamchk -r f*/*.MYI
myisamchk -r g*/*.MYI
myisamchk -r h*/*.MYI
myisamchk -r i*/*.MYI
myisamchk -r j*/*.MYI
myisamchk -r k*/*.MYI
myisamchk -r l*/*.MYI
myisamchk -r m*/*.MYI
myisamchk -r n*/*.MYI
myisamchk -r o*/*.MYI
myisamchk -r p*/*.MYI
myisamchk -r q*/*.MYI
myisamchk -r r*/*.MYI
myisamchk -r s*/*.MYI
myisamchk -r t*/*.MYI
myisamchk -r u*/*.MYI
myisamchk -r v*/*.MYI
myisamchk -r w*/*.MYI
myisamchk -r x*/*.MYI
myisamchk -r y*/*.MYI
myisamchk -r z*/*.MYI



Turn everything back on:

/etc/rc.d/init.d/chkservd start
/etc/rc.d/init.d/mysql start