Ubuntu
This page is now depreciated, and is no longer being updated. |
---|
The page was becoming too large - all content from this page, and newer updates, can be found via the Category page link below.
This page and its contents will not be deleted. |
See Ubuntu |
Initial Setup
Much of this section is borrowed from http://www.howtoforge.com/perfect-server-ubuntu8.04-lts and http://www.howtoforge.com/how-to-install-ubuntu8.04-with-software-raid1, they are well worth a read!
This section will create a Ubuntu VM installed on one partition, software RAID'ed across two VMDK's. To explain, my ESX's storage originally wasn't resilient, hence the software RAID across VMDK's on separate physical disks, if you've got resilient storage you should probably wouldn't use software RAID.
However, once I'd bought a nice (SOHO) NAS, I moved one disk and VM config across to NAS, thinking I'd eventually ditch the software RAID. Luckily I didn't get round to it, so when I managed to destroy my NAS (partly my fault), I could easily recover my VM's from where they left off by creating new ones and re-using the surviving VMDK file. Therefore, unless you're running a truly enterprise class NAS, that's cost you £1k's to buy, and £1k's in yearly support I'd still recommend you software RAID your critical VM's (eg mail server) across two separate devices. The whole reason you have a home set-up is to play, which inevitably means break!
Prepare Virtual Machine
- Create a virtual machine with the following options (use Custom)
- Guest OS: Linux > Ubuntu 32bit
- CPU: 1
- Memory: 756 MB
- Disk: 36GB
- Then add a second 36GB disk on a separate physical datastore (if you intend to use software RAID)
- Attach Ubuntu install ISO to the CD-ROM
OS Installation
Follow the default or sensible choices for your locale, however, use the following notes as well...
- Configure the network
- Enter the server's hostname (not a FQDN, just the hostname)
- Partition Disks
- If setting up software RAID follow the steps below, otherwise just select Guided - use entire disk and set up LVM
- Select "Manual
- Then create a partition...
- Select the first disk (
sda
) and on the next screen, Yes, to Create new empty partition table on this device? - Select the FREE SPACE, then Create a new Partition, and use all but the last 2GB of space,
- And then select type of Primary, and create at Beginning
- Change Use as to physical volume for RAID, and change the Bootable flag to Yes, the select Done setting up this partition
- Select the first disk (
- Repeat the above on the remaining FREE SPACE on
sda
, to create another primary physical volume for RAID, but 'not bootable - Select the second disk,
sdb
, and repeat the steps taken forsda
to create two identical partitions - On the same screen, select the Configure Software RAID option (at the top), and then confirm through the next screen
- Create a RAID pack/multidisk...
- Select Create MD device, then select RAID1 (ie a mirror), then confirm 2 Active devices, and 0 Spare devices
- Select both
/dev/sda1
and/dev/sdb1
partitions, and then select Finish
- Repeat the above to create a RAID volume using
/dev/sda2
and/dev/sdb2
partitions - Now select the RAID device #0 partition (select the #1 just under RAID1 device line), and change the Use as and select Ext3...
- Change the Mount point to /, then select Done configuring this partition
- Now select the RAID device #1 partition (select the #1 just under RAID1 device line), and change the Use as and select Swap area
- Then select Done configuring this partition then finally Finish partitioning and write changes to disk, and confirm to Write the changes to disks
- Accept the "The kernel was unable to re-read...system will need to restart" complaints for each RAID multidisk, after which the install will continue (note there's a little more to do post install to ensure you can boot using the second disk should the first fail).
- If setting up software RAID follow the steps below, otherwise just select Guided - use entire disk and set up LVM
- Software Selection
- DNS Server - Only required in order to configure split DNS, which is required for an exchange server install
- OpenSSH Server - Required (allows you to Putty/SSH to the server)
Post OS Install Config
- Enable Root
- Use the command
sudo passwd root
- Enter user password, and then a strong password for the root account
- Use the command
- Finish Software RAID config - only if configured during install
- Start-up grub (by entering
grub
and enter the following commands (seems to work better via SSH than direct console)...device (hd1) /dev/sdb
root (hd1,0)
setup (hd1)
quit
- Then edit the
/boot/grub/menu.lst
config file. Go to the end of the file where the boot options are, and create a copy of the first option and edit the following linestitle
Add "Primary disk fail" or something similar to endroot
Changehd0
tohd1
- To check the RAID setup of your drives use
mdadm --misc -D /dev/md0
mdadm --misc -D /dev/md1
- Start-up grub (by entering
Change IP Address
- Edit the
/etc/network/interfaces
file in the following fashion
# The primary network interface auto eth0 iface eth0 inet static address 192.168.1.150 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1
- Then check the local hosts file
/etc/hosts
, so that the IP v4 part looks like...
127.0.0.1 localhost 192.168.1.150 mail.home.int mail
- Check that DNS resolution is setup correctly (add DNS nameservers as required, as found in
/etc/resolv.conf
in order of pref...
nameserver 127.0.0.1
- Then restart networking
sudo /etc/init.d/networking restart
Install VM Tools
The pre-built modules that come with the VMTools installer aren't compatible, therefore the script needs to be able to compile them, however the required library files aren't available by default, so the procedure is a little laboured.
Ubuntu 8.04.4 LTS
- Install the build library files...
apt-get install build-essential
apt-get install linux-headers-2.6.24-26-server
- Use
uname -r
to get the right headers version number
- Use
- Select "Install VM Tools" from the VI Client
- Mount the VM Tools CD-ROM
mount /media/cdrom0/
- Copy to home directory
cp /media/cdrom/VMwareTools-4.0.0-219382.tar.gz /home/user/
- Uncompress and then move into the
vmware-tools-distrib
directorytar xf VMwareTools-4.0.0-219382.tar.gz
cd vmware-tools-distrib
- Run the install script
./vmware-install.pl
- Restart
shutdown -r now
Ubuntu 10.04.1 LTS
VM Tools can be installed via two methods, neither of which is ideal...
- Using the normal VM Tools CD - requires additional library install and sometimes mounting the CDROM doesn't work too well.
- Using APT package manager - doesn't work quite as well as it could (upgrading VM Tools isn't supported), and support for this method is rumoured to be dropped in future releases
VM Tools CD
- Install the build library files (not required for ESX v4.0 update 2 and later)...
apt-get install build-essential
- Select "Install VM Tools" from the VI Client
- Mount the VM Tools CD-ROM
mount /dev/cdrom /media/cdrom/
- If
/media/cdrom/
doesn't exist, create withmkdir /media/cdrom
- If
- Copy to tmp directory (version number below will vary)
cp /media/cdrom/VMwareTools-4.0.0-236512.tar.gz /tmp/
- Unmount the CD-ROM, and move into tmp directory
umount /media/cdrom/
cd /tmp/
- Uncompress and then move into the
vmware-tools-distrib
directorytar xzvf VMware*.gz
cd vmware-tools-distrib
/
- Run the install script, and accept defaults
./vmware-install.pl
- Restart
shutdown -r now
APT Package Manager
- Install VM Tools using apt package manager
- Open VMware Packaging Public GPG Key at http://packages.vmware.com/tools/VMWARE-PACKAGING-GPG-KEY.pub
- On the server open a new file called
VMWARE-PACKAGING-GPG-KEY.pub
with the/tmp
directory - Copy and paste the contents of the webpage into the file and save
- Import the key using the following command
apt-key add /tmp/VMWARE-PACKAGING-GPG-KEY.pub
- You should get
OK
returned
- If you need to add a proxy see http://communities.vmware.com/servlet/JiveServlet/download/1554533-39836/Vmware%20Tools%20Guide%20Linux%20osp_install_guide.pdf
- Open a new vi in VI called
/etc/apt/sources.list.d/vmware-tools.list
- Add the following line
deb http://packages.vmware.com/tools/esx/<esx-version>/ubuntu lucid main restricted
where <esx-version> is the appropriate esx version found at http://packages.vmware.com/tools/esx/index.html
- Update the repository cacahe
apt-get update
- Install VM Tools
apt-get install vmware-tools
NTP
Not required if your server doesn't really need bang on accurate time
Out of the box your server will sync every time its restarted and drift a bit in-between. There is an additional resource demand in running the NTP daemon so unless you need to, there's no need to install the full blown NTP daemon.
I tend to have one or two servers updating from remote (public) servers, and then all others updating from those.
- Install the service
apt-get install ntp
- Update the NTP config file,
/etc/ntp.conf
(Example below is for a server updating from public European servers - see http://www.pool.ntp.org/)server 0.europe.pool.ntp.org
server 1.europe.pool.ntp.org
server 2.europe.pool.ntp.org
server 3.europe.pool.ntp.org
- Restart the NTP service
service ntp restart
- Verify using the following commands
ntpq -np
date
Update the OS
- Run the following command to update the apt package database
apt-get update
- To install any updates
apt-get upgrade
Random Settings
Locale
To change the local time-zone use...
dpkg-reconfigure tzdata
To change the keyboard layout in use...
dpkg-reconfigure console-data
...if console-data
isn't installed, use...
apt-get install console-data
...and reboot to apply
\tmp
Boot Time Clean-up
The files in /tmp
get deleted if their last modification time is more than TMPTIME
days ago.
- Edit
/etc/default/rcS
- Change
TMPTI80aM80E
value to specify no of days- Use
0
so that files are removed regardless of age. - Use
-1
so that no files are removed.
- Use
Proxy Server
Proxy settings need to be added as environment variables, which can be added to to your profile file so as to be always be applied
- Edit
/etc/profile
- Append to the bottom (edit as required)
export http_proxy=http://username:pass@proxyserver:port/
export ftp_proxy=http://username:pass@proxyserver:port/
Note that some applications will ignore the environment variables, and will need to be set specifically for those apps.
Hostname Change
Procedure below guides you through the files etc that need updating in order to change a machine's hostname. Note that if you get probs SSH'ing to the server afterwards see Server Hostname Change
- Update the following files
/etc/hosts
/etc/hostname
- Set the hostname (not FQDN)
hostname <servername>
- Reboot
Allow Remote SSH Login Without Password Prompt
In order to be able to access a remote server via an SSH session without needing to suppy a password, the remote server needs to trust the user on the local server. In order to do this, the public key for the user needs to be imported to the remote server. This is particularly useful when trying to script using ssh, scp, rsync, etc where you need to interract with a remote server.
You need to be clear on which user will access the remote the server, if your script is run as root, then its the root user that needs to have its public key exported.
Similarly, on the remote server you need to ensure that that the user that has the public key key imported into, has the rights to perform whatever it is that you want to achieve. This shouldn't be the root user (to do so you'd need to allow PermitRootLogin
in the remote server's SSH config, which is a security no-no).
- On the local server, create a public/private rsa key pair while logged in as the user that will access the remote server
ssh-keygen -t rsa
(leave passphrase blank)- This creates a public key in
~/.ssh/id_rsa.pub
- This creates a public key in
- Copy the public key to the user on the remote server
ssh-copy-id -i user@remote-svr
- The
user
is the user account on the remote server that the local server will be trusted by and run as.
- The
- Test the login as suggested by
ssh-copy-id
ssh user@remote-svr
Packages
Commands
Command | Purpose |
---|---|
dpkg --get-selections |
Show installed packages |
dpkg -L php5-gd |
Show file locations of php5-gd package
|
apt-get update |
Update the package database |
apt-get install <package> |
Install the <package> package
|
apt-get upgrade |
Upgrade installed system and packages with latest levels in package database |
tasksel install <task> |
Installs a collection of packages as a single task, eg lamp-server |
tasksel --list-task |
Show list of available tasks |
Troubleshooting
- Error 400 Bad Request
- Somewhat misleadingly, the problem is normal caused by being unable to contact the update server. Consider adding proxy server config to your machine
- The following packages have been kept back
- Package manager can hold back updates because they will cause conflicts, or sometimes because they're major kernel updates. Running
aptitude upgrade
normally seems to force kernel updates through.
- Package manager can hold back updates because they will cause conflicts, or sometimes because they're major kernel updates. Running
Firewall
Ubuntu comes with UFW (Uncomplicated Firewall), which is a config tool used to modify the standard inbuilt Netfilter. If preferred, iptables can still be used.
Changes are applied immediately. Once you've added your first rule there's an implied deny all.
Command | Purpose |
---|---|
ufw enable |
Enables the firewall |
ufw status |
Shows the firewall status and existing filters |
ufw status numbered |
Shows the firewall status and numbered existing filters (easier to delete) |
ufw allow from 192.168.1.10 |
Allow all traffic from 192.168.1.10 |
ufw allow http |
Allow http from any IP |
ufw allow proto tcp from 192.168.1.10 to any port 22 |
Allow TCP 22 (SSH) from 192.168.1.10 |
ufw delete 2 |
Delete rule 2 |
So, for example, to create a couple of rules and enable...
ufw allow proto tcp from 192.168.10.0/24 to any port 22 ufw allow proto tcp to any port 443 ufw enable
SNMP
Setup (Pre v10)
- Run the following command to update the package database
apt-get update
- Run the following command to install SNMP
apt-get install snmpd
- Create config file with contents as shown below
vi /etc/snmp/snmpd.conf
- Edit SNMPD config to allow remote polls
vi /etc/default/snmpd
- Remove
127.0.0.1
from line below#snmpd options (use syslog, close stdin/out/err).
SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/snmpd.pid 127.0.0.1'
- Restart SNMP
/etc/init.d/snmpd restart
- Test with the following, replacing <hostname> with server's hostname
snmpwalk -v 1 -c public -O e <hostname>
rocommunity public syslocation "CR DC" syscontact info@sandfordit.com
Setup (v10)
- Run the following command to update the package database
apt-get update
- Run the following command to install SNMP
apt-get install snmpd
- Create config file with contents as shown below the procedure
vi /etc/snmp/snmpd.conf
- Edit SNMPD config to allow remote polls
vi /etc/default/snmpd
- Remove
127.0.0.1
from line below#snmpd options (use syslog, close stdin/out/err).
SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/snmpd.pid 127.0.0.1'
- Restart SNMP
/etc/init.d/snmpd restart
- Test with the following, replacing <hostname> with server's hostname (must be run from a machine with snmp installed, not just snmpd)
snmpwalk -v 1 -c public <hostname> system <hostname>
#### # First, map the community name (COMMUNITY) into a security name # (local and mynetwork, depending on where the request is coming # from): # sec.name source community #com2sec paranoid default public <- Comment com2sec readonly default public <- Uncomment
... then later ...
syslocation "CR DC" syscontact info@sandfordit.com
MySQL
Install
- Run the following command to update the package database
apt-get update
- Run the following command to install MySQL
apt-get install mysql-server
To allow access from remote hosts...
- Open MySQL service TCP/IP port by editing the
/etc/mysql/my.cnf
config file and restarting- Change bind IP to server's IP, EG
bind-address = 192.168.1.123
- Restart service
/etc/init.d/mysql restart
- Change bind IP to server's IP, EG
- Allow remote access to a user account
- EG
GRANT ALL PRIVILEGES ON *.* TO 'user'@'%' IDENTIFIED BY 'pass' WITH GRANT OPTION;
- EG
Backup
Based on http://www.cyberciti.biz/faq/ubuntu-linux-mysql-nas-ftp-backup-script/
- Create the required folders using...
mkdir backup
mkdir backup/mysql
- Create the file below (editing as required) as
/backup/mysql.sh
- Make the file executable
chmod +x /backup/mysql.sh
- Perform a test run of the backup
- Schedule the script to run with crontab
crontab -e
30 1 * * * /bin/bash /backup/mysql.sh
#!/bin/bash
### MySQL Server Login and local backup info ###
MUSER="root"
MPASS="password"
MHOST="localhost"
MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"
BAK="/backup/mysql"
LOG="/backup/mysql.log"
GZIP="$(which gzip)"
NOW=$(date -u +%Y%m%d)
## FTP info
FTPDIR="/Backup/db"
FTPUSER="backup"
FTPPASS="backup"
FTPSERVER="ftphost"
## Functions
Logger()
{
echo `date "+%a %d/%m/%y %H:%M:%S"`: $1 >> $LOG
}
## Main Script
Logger "Started backup script..."
[ ! -d $BAK ] && mkdir -p $BAK
[ ! -d $BAK/tmp ] && mkdir -p $BAK/tmp
mv $BAK/* $BAK/tmp
DBS="$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse 'show databases')"
for db in $DBS
do
FILE=$BAK/$db.$NOW.gz
Logger "Backing up $db to $FILE"
$MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS $db | $GZIP -9 > $FILE
done
Logger "Completed local backup"
## FTP to remote server
ftp -in <<EOF
open $FTPSERVER
user $FTPUSER $FTPPASS
bin
cd $FTPDIR
lcd $BAK
mput *
close
bye
EOF
if [ "$?" == "0" ]; then
Logger "FTP upload completed successfully"
/bin/rm -f $BAK/tmp*
Logger "Previous local backup files removed"
else
Logger "FTP upload failed !!!"
fi
In some versions of MySQL you will receive an error similar to...
mysqldump: Got error: 1044: Access denied for user 'root'@'localhost' to database 'information_schema' when using LOCK TABLES
It appears to be a bug, which seems to keep cropping up. As a workaround change the $MYSQLDUMP
line to
$MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS --skip-lock-tables $db | $GZIP -9 > $FILE
Note that you won't backup the information_schema
table if you need to implement this workaround
Perl
Install Module
Installing a perl module isn't tricky, but there is a certain nack to it, see below...
- Get the module's package name (eg for Net::XWhois)
sudo apt-cache search perl net::xwhois
- Then install the package
sudo apt-get install libnet-xwhois-perl
Check Module(s) Installed
To check for a specific module use (checking for Net::XWhois
)
perl -MNet::XWhois -e "print \"Module installed.\\n\";"
To list all installed modules
perl -MFile::Find=find -MFile::Spec::Functions -Tlwe \ 'find { wanted => sub { print canonpath $_ if /\.pm\z/ }, no_chdir => 1 }, @INC'
Python
Python v2 comes pre-installed, however if you want to run newer Python 3 scripts, this will need to be installed alongside.
- Install the package
apt-get install python3
- Note that more than one version of Python 3 may be available, cancel the install are retry with specific version if required, eg
apt-get install python3.1
- Note that more than one version of Python 3 may be available, cancel the install are retry with specific version if required, eg
To enter the Python 3 interpreter, run phython3
, to make sure you get the right environment for a script use the following shebang
#! /usr/bin/env python3
See Python for further info
AWStats
Initial Setup
- Install the package
apt-get install awstats
- Edit the the generic template config file if required
/etc/awstats/awstats.conf
- Create apache config file for site with contents show below
- eg
/etc/apache2/sites-enabled/awstats
- eg
- Restart apache
service apache2 restart
- Site should now be available via URL similar to
Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /awstats/ /usr/lib/cgi-bin/ <Directory /usr/lib/cgi-bin/> Options ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> <Directory /usr/share/awstats/> Order allow,deny Allow from all </Directory>
Add a Site
- Create a specific config file for the site to monitor
cp /etc/awstats/awstats.conf /etc/awstats/awstats.mysite.com.conf
- Edit the config file for the site, specifically (see below for further options)
LogFile=”/path/to/your/domain/access.log”
LogFormat=1
(this will give you more detailed stats)SiteDomain=”mysite.com”
HostAliases=”www.mysite.com localhost 127.0.0.1"
(example for a local site)
- Perform an initial stats gather for the site
/usr/lib/cgi-bin/awstats.pl -config=mysite.com -update
- Test that you can see some stats, using URL similar to
- Add a scheduled job to crontab to update automatically
crontab -e
- EG every 30 mins
*/30 * * * * /bin/perl /usr/lib/cgi-bin/awstats.pl -config=mysite.com -update >/dev/null
Further options
- Wiki sites (and other sites where an URL parameter can specify a specific page
URLWithQuery=1
- useful for Wiki's etc where query param indicates a different pageURLWithQueryWithOnlyFollowingParameters="title"
- only treats variances in param title as distinct pagesURLReferrerWithQuery=1
follows on from two above
Other
To perform a one-off update from a specific log file...
/usr/lib/cgi-bin/awstats.pl -config=server -LogFile=access.log
- Updates can only be added in chronological order, therefore you may need to delete the data file for a particular month, and rebuild it entirely.
Scheduled updates are configured in /etc/cron.d/awstats
Syslog to MySQL Database
This procedure achieves three things...
- Allows remote hosts to use the local server as a syslog destination
- Directs syslogs to MySQL database on the server
- Allows viewing of syslogged events through LogAnalyser web front end
...it is assumed that you already have a local MySQL and Apache server running!
- Set-up your server to send syslog messages to a MySQL database
apt-get install rsyslog-mysql
- Enter the root password to your MySQL instance when prompted
- Update the
rsyslog
config (/etc/rsyslog.conf
) to receive syslog data, and to route messages through a queue- Uncoment the following..
$ModLoad ommysql # load the output driver (use ompgsql for PostgreSQL)
$ModLoad imudp # network reception
$UDPServerRun 514 # start a udp server at port 514
- Add the following...
$WorkDirectory /rsyslog/work # default location for work (spool) files
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName dbq # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
- Restart the service
service rsyslog restart
- Uncoment the following..
- Install LogAnalyser
- Download latest build from http://loganalyzer.adiscon.com/downloads
- Uncompress
- EG
tar xf loganalyzer-3.5.0.tar.gz
- EG
- Move the contents or
/src
to webserver- EG
mkdir /var/www/syslog
- EG
mv /src/* /var/www/syslog/
- EG
- Move utility scripts to same folder
- EG
mv /contrib/* /var/www/syslog/
- EG
- Make them both executable,
- EG
chmod +x /var/www/syslog/*.sh
- EG
- Run the config script in the directory
- EG
/var/www/syslog# ./configure.sh
- EG
- Browse to webpage
- Ignore the error, and follow the link to install (configure)
- Accept defaults until step 7, where you change the following
- Name of the Source - your name for the local syslog db
- Source Type - MySQL Native
- Database Name - Syslog
- Database Tablename - SystemEvents
- Database User - rsyslog
- Database Password - rsyslog
- Config completed!
Troubleshooting
Network
No NIC
Especially after hardware changes, its possible the networking config no longer refers to the right interface.
- Use
ifconfig
to confirm the current network config - Use
dmesg | grep -i eth
to ascertain what's been detected at boot time - Assuming it states that say
eth0
has been changed toeth1
then just update the/etc/network/interfaces
file
Software RAID
Replacing a RAID 1 Disk
This procedure was written from the following starting point...
- A machine originally with two disks in RAID1 has failed, one disk has been replaced, and machine started again
...and adapted from this post http://www.howtoforge.com/replacing_hard_disks_in_a_raid1_array
- Backup whatever you can before proceeding, one mistake or system error could destroy your machine
- Confirm which disk is new, and which is old (if the new disk is blank this is easy as there will be no partition info!)
fdisk -l
- Partition the new disk the same as the original
sfdisk -d /dev/sda | sfdisk /dev/sdb
- Confirm that the layout of both disks is now that same
fdisk -l
- Add the newly created partitions to the RAID disks
mdadm --manage /dev/md0 --add /dev/sdb1
- You may have more
sd
partitions thanmd
partitions, the array size return throughmdadm -D /dev/md*
should roughly match the number of blocks found fromfdisk -l
- The arrays should now be being sync'ed, check progress by monitoring
/proc/mdstat
more /proc/mdstat
SSH
Server Hostname Change
If the hostname (or IP) of the server you are SSH'ing to changes, the old entry needs to be removed from your SSH key known hosts file
ssh-keygen -R <name or IP>
Reboot Required?
If a package update/installation requires a reboot to complete the following file will exist...
/var/run/reboot-required
To see which packages caused this to be set, inspect the contents of...
/var/run/reboot-required.pkgs