最近一直在思考运维该怎么做,该做成什么样,怎么才能把一向苦逼的运维变的不再苦逼,怎么解决运维和开发之间的矛盾,怎么改变运维弱势的地位,依然未有成型的思考,知道前方路漫漫,一点一点来吧。
于是想总结下了,把自己知道的写下来,写的过程中也能继续思考,给未来一点思路。那么,这个专题- 构建机房运维基础架构,就开始吧。
原理介绍
pxe 是一种网卡支持的引导方式,我们只是用它来网络引导,实现远程装机的目的。
首先要配置网卡支持PXE启动模式,DELL R420 – R720 机器 开机 CTRL + S 进入配置界面,设置成如下:

下面看一下典型网络安装系统的流程:

这里用了三个服务:
dhcp –用来供网卡(内网卡)获取动态IP
tftp –用来获取基础os环境和安装ks文件
nfs –提供iso镜像文件
这里我们用 httpd 来代替 nfs,nfs在大规模装机的时候可能会有问题。
搭建PXE服务器
那么,搭建一台PXE服务器其实就是 配置三个服务:
dhcp tftp httpd (nginx)
下面我们在centos6.3 上面来安装配置PXE。
安装nginx dhcp tftp
yum -y install nginx dhcp xinetd tftp-server
dhcp的配置文件路径:
/etc/dhcp/dhcpd.conf
修改这个文件,写入:
ddns-update-style interim;
ignore client-updates;
allow unknown-clients;
allow bootp;
allow booting;
ping-check true;
default-lease-time 900;
max-lease-time 1200;
next-server 10.0.11.12;
filename “/pxelinux.0”;
option domain-name-servers 10.0.12.234;
option domain-name “nosa.me“;
subnet 10.0.11.0 netmask 255.255.255.0 {
option routers 10.0.11.1;
option subnet-mask 255.255.255.0;
range dynamic-bootp 10.0.11.100 10.0.11.250;
}
subnet 10.0.21.0 netmask 255.255.255.0 {
option routers 10.0.21.1;
option subnet-mask 255.255.255.0;
range dynamic-bootp 10.0.21.100 10.0.21.250;
}
很简单,10.0.11.12 是 tftp 的IP,DHCP分配给client IP 之后会指导客户端去请求 10.0.11.12 tftp 服务的 /pxelinux.0 文件 .
由于指定了domain-name-serves 和 domain-name ,此处 10.0.11.12 写 域名应该也可以(最好测试一下,我暂时没时间测试。)
tftp 的路径:
/var/lib/tftpboot/
/etc/xinetd.d/tftp
1.先把/etc/xinetd.d/tftp 里面的 disable 配置改成 no,这样xinetd启动的时候会起来 tftp
2.看下 /var/lib/tftpboot/ 目录下需要什么文件:
pxelinux.0
boot.msg
splash.lss
centos_6.3_x64
centos_6.3_x64/vmlinuz
centos_6.3_x64/initrd.img
pxelinux.cfg
pxelinux.cfg/default
解释:
1). pxelinux.0 是bootloader 文件,dhcp会指导client来拿到这个文件.
2). pxelinux.cfg/default 这是 tftp 最主要的配置文件,client端会根据这个配置来最终拿到ks 文件,这里的ks写的是域名,因为我们在DHCP里面配置了domain-name-serves 和 domain-name,因此可以解析域名的,用域名还有个好处是可以做负载均衡。
典型的 default 文件内容如下:
default centos_6.3_x64_raw_clean
prompt 1
timeout 600
display boot.msg
label local
localboot 0
label centos_6.3_x64_raw_clean
kernel centos_6.3_x64/vmlinuz
append netmask=255.255.255.0 ksdevice=em2 initrd=centos_6.3_x64/initrd.img nofb text ks=http://pxe.hy01.nosa.me/ks/centos_6.3_x64_raw_clean.cfg
label centos_6.3_x64_kvm_host
kernel centos_6.3_x64/vmlinuz
append netmask=255.255.255.0 ksdevice=em2 initrd=centos_6.3_x64/initrd.img nofb text ks=http://pxe.hy01.nosa.me/ks/centos_6.3_x64_kvm_host.cfg
3). boot.msg 安装时候显示的文本界面,在default里面定义
boot.msg 如下:
splash.lss
– case: centos_6.3_x64_raw_clean ip=10.0.10.*
– case: centos_6.3_x64_kvm_host ip=10.0.10.*
4). splash.lss 是安装时候的显示图片,不重要。
5). centos_6.3_x64目录 内核相关文件,用于client端 建立内核环境,ISO镜像里面有这两个文件,直接拷过来即可;定义在default文件中。
装机端加载好内核环境之后,会从 ks 指定的链接获取 ks 配置文件,比如 http://pxe.hy01.nosa.me/ks/centos_6.3_x64_raw_clean.cfg ,ks 会指导怎么格式化硬盘、密码、语言、防火墙等等;此时需要一个http server ,我们用 nginx .
nginx 的配置很简单, 只要让上面的ks链接能正常获取就可以了,这里不介绍配置,来看一下ks 的配置文件的格式把。
# Kickstart for NC net install
# 2011-04-27 Cecil.Han Merge from Daniel version
##################################################
# All alternative parts I add [Optional] mark
##################################################
#Modified by NingningLi on 2012/09/10
# Do not try to probe the monitor
#monitor –noprobe
# This cd is for platform=x86, AMD64, or Intel EM64T
# System authorization information
auth –useshadow –enablemd5
# System bootloader configuration
bootloader –location=mbr
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart –all –initlabel
# Use text mode install
text
# Firewall configuration
#firewall –enabled –port=22:tcp
firewall –disabled
# System keyboard
keyboard us
# System language
lang en_US
# Installation logging level
logging –level=info
# Use Net Install installation
#cdrom
url –url http://pxe.hy01.nosa.me/iso/centos_6.3_x64
# Network information
#network –bootproto=static
# Root password
rootpw –iscrypted $1$rCxxxr.wrewrW6234234234K/
# SELinux configuration
selinux –disabled
# Do not configure the X Window System
skipx
# System timezone
timezone –utc Asia/Shanghai
# continue to install if unsupported hardware
unsupported_hardware
# Install OS instead of upgrade
install
#reboot after installation
reboot
# Disk partitioning information
%include /tmp/partconfig
%pre –interpreter /bin/sh
export PATH=$PATH:/sbin:/bin:/usr/sbin:/usr/bin
DRIVER_INSTALL=”`fdisk -l | grep -i ‘^Disk /dev/’ | awk ‘{print $2, $3}’ | sed ‘s/://g’ | sed ‘s#/dev/##g’ | awk ‘BEGIN{disk=””; size=0}{if(size == 0 || size > $2) {size = $2; disk = $1}}END{print disk}’`”
echo “””part /boot –fstype=”ext3” –size 512 –ondisk=$DRIVER_INSTALL
part swap –size 4096 –ondisk=$DRIVER_INSTALL
part pv.01 –size=1 –grow –ondisk=$DRIVER_INSTALL
volgroup domovg pv.01
logvol / –vgname=domovg –fstype=”ext4″ –size=18432 –name=root
logvol /home –vgname=domovg –fstype=”ext4″ –size=1024 –name=home””” > /tmp/partconfig
# Disk partitioning cfg backup
#part /boot –fstype=”ext3″ –size 512 –ondisk=sda
#part swap –size 4096 –ondisk=sda
#part pv.01 –size=1 –grow –ondisk=sda
##part pv.01 –size=20480 –ondisk=sda
#volgroup domovg pv.01
#logvol / –vgname=domovg –fstype=”ext4″ –size=1024 –name=root
#logvol /usr –vgname=domovg –fstype=”ext4″ –size=2048 –name=usr
#logvol /var –vgname=domovg –fstype=”ext4″ –size=2048 –name=var
#logvol /home –vgname=domovg –fstype=”ext4″ –size=1024 –name=home
#logvol /opt –vgname=domovg –fstype=”ext4″ –size=1024 –name=opt
#logvol /tmp –vgname=domovg –fstype=”ext4″ –size=3072 –name=tmp
# you can check the Description of each rpm packages in cdrom:/script/RPMINFO
%packages –nobase
# Kernel
# System
basesystem
centos-release
#centos-release-notes
filesystem
initscripts
setup
SysVinit
# Lib & Module
dhcp-common
glibc
libgcc
libstdc++
#libtermcap
lockdev
m2crypto
#nss_ldap
#pam_ccreds
pam_passwdqc
pam_pkcs11
readline
# Tools
authconfig
bc
bind-utils
bzip2
coreutils
cpio
crontabs
dmidecode
dmraid
dstat
e2fsprogs
eject
file
ftp
gpm
grub
hdparm
info
iproute
iptables
iputils
links
logrotate
logwatch
irqbalance
lsof
mailx
make
man
mcelog
mdadm
minicom
#mkinitrd
mlocate
nc
ntp
nscd
openldap-clients
openssh-clients
openssh-server
passwd
patch
patchutils
pciutils
procmail
procps
psacct
rpm
rsync
screen
sendmail
shadow-utils
smartmontools
strace
sudo
symlinks
sysstat
tcpdump
telnet
#termcap
compat-libtermcap
traceroute
unzip
util-linux
vim-common
vim-enhanced
vim-minimal
vixie-cron
wget
which
yum
# extra
ctags
glib2
glib2-devel
libdbi
libicu
libnfnetlink
lrzsz
pkgconfig
# Shell
bash
# unused packages
-postfix
-mysql-libs
-ecryptfs-utils
-cryptsetup-luks
-dhclient
-dhcpv6_client
-ed
-kudzu
-libhugetlbfs
-rootfiles
-pm-utils
-selinux-policy-targeted
-setools
-setserial
-sysfsutils
-system-config-network-tui
-pam_pkcs11
-*firmware*
-b43-openfwwf
# unused 32 bit distribution
# Need to test, I’m not sure whether we can use *
#-*.i386
#-*.i686
-audit-libs.i386
-cracklib.i386
-cyrus-sasl-lib.i386
-db4.i386
-device-mapper.i386
-e2fsprogs-libs.i386
-gpm.i386
-keyutils-libs.i386
-krb5-libs.i386
-libgcc.i386
-libselinux.i386
-libsepol.i386
-libstdc++.i386
-libtermcap.i386
-lockdev.i386
-mkinitrd.i386
-nss_ldap.i386
-openldap.i386
-pam.i386
-pam_ccreds.i386
-pam_passwdqc.i386
-readline.i386
-zlib.i386
# Post work
# copy the scripts from cd to disk before chroot to the new installed os
%post
/etc/init.d/sshd restart
script_url=”http://pxe.hy01.nosa.me/script“
wget ${script_url}/post_install.sh &>/dev/null
sh -x post_install.sh raw_clean &>/dev/null
%include /tmp/partconfig 部分是硬盘的格式化配置,可能需要注意下;url 部分是 iso的地址,这里也用域名,最好做负载均衡,因为各种RPM安装包都是从这里获取的;在post阶段执行一段初始化脚本。
其他参数意义不一一说明了,可以看下面的链接。
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/s1-kickstart2-options.html
用DDNS管理ILO IP
如果我们不在机房,怎么控制机器装机呢,可以使用控制卡,这时如果我们使用DDNS 可以省去手动配置控制卡 IP的麻烦,只要插上网线就可以自动获取,大大减少工作量;而且控制卡发出DHCP请求的时候会带上SN,so 根据SN就可以解析到控制卡 IP 。
安装:
yum -y install dhcp bind
dhcp的配置 /etc/dhcp/dhcpd.conf :
ddns-update-style interim;
#ddns-update-style none;
ignore client-updates;
default-lease-time 69120000;
max-lease-time 86400000;
option time-offset -18000;
option domain-name “ilo.nosa.me“;
option domain-name-servers 10.2.1.1;
subnet 10.2.0.0 netmask 255.255.0.0 {
range dynamic-bootp 10.2.1.11 10.2.255.254;
option broadcast-address 10.2.255.255;
option routers 10.2.1.1;
}
key ilo {
algorithm hmac-md5;
secret HqX9xaJ75tgQ1S8hHz9L7Q==;
}
zone ilo.nosa.me. {
primary 10.2.1.1;
key ilo;
}
zone 2.10.in-addr.arpa. {
primary 10.2.1.1;
key ilo;
}
bind 的配置文件 /etc/named.conf :
options {
directory “/var/named”;
dump-file “/var/named/data/cache_dump.db”;
statistics-file “/var/named/data/named_stats.txt”;
memstatistics-file “/var/named/data/named_mem_stats.txt”;
allow-transfer { none; };
recursion yes;
};
logging {
channel default_debug {
file “data/named.run” size 20M;
severity info;
print-time true;
};
};
zone “.” IN {
type hint;
file “named.ca“;
};
key ilo {
algorithm hmac-md5;
secret HqX9xaJ75tgQ1S8hHz9L7Q==;
};
zone “ilo.nosa.me.” IN {
type master;
file “named.ilo.nosa.me“;
allow-update {key ilo;};
};
zone “2.10.in-addr.arpa.” IN {
type master;
file “named.2.10.in-addr.arpa“;
allow-update {key ilo;};
};
#include “/etc/named.rfc1912.zones”;
仅供参考,具体的DDNS配置请查资料。