SolengTech‎ > ‎Open Source‎ > ‎

HowTo: Provision Centos for Configuration Management

The purpose of this is to show how to provision a production ready, secure, Configuration Management system that:
  1. Has been deployed in prfoduction at IT departments in large firms;
  2. Can be deployed to Cloud Computing environment;
  3. Is cost effective for SOHO operations and infrastructure;
  4. Has a CM system met Federal requirements Cyber Security;
  5. Saves several hundred hours Admin time setting up and stabilizing a complex server for the CM system.
Managing multiple projects with multiple contributors for multiple customers with multiple IP bases has become the normal environment for small organizations with large customers. In most cases the smaller firm just uses the larger firm's configuration management until the number of projects, restarting of old projects, developers, and inventory of IP forces a CM system to be put in place. While many small firms do deploy some level of CM, there is no real profit in the effort -- It may be an important check item to a customer, but you can't bill your customers for IT and CM infrastructure. The CM used by a small firm is rarely acceptable to an IT department and each customer has different requirements -- Therefore a small firm using the larger customer's CM system can be the right choice: It meets their CM requirements, they already pay for their CM system, and they are not going to pay for your CM system.

New business in Medical Devices, Renewable Energy, and in the US SBIR contracts can place Configuration Management and Security requirements down to the contractor or small business. Cyberattacks resulting in IP theft becoming more common to the point where smaller firms need to lock down their networks even more at a time when Telecommuting,  Web Services and other technology are standard and expected in a development environment.

An organization that addresses CM can have a competitive advantage in terms of meeting customer requirements. The cost is manageable IF a secure, approved by IT, approved by industry/government, cost effective solution is deployed. There are two solutions based on an early version of SourceForge that can meet the CM requirements. The Centos Linux distribution is similar to RedHat which is used by most U.S. IT departments. It is a free version of RedHat, but support can be purchased which is an IT requirement. A Virtual Machine is used for packaging and distribution. This avoids the time and effort of provisioning the server by packaging all those changes into an image that can run on any VMWare or VirtualBox environment. Virtual Machines are typically how cloud computing servers are packaged, and thus it can be deployed as a cloud solution.

Project Management, Software Configuration Management

While the CM packages are available for download, the server that they run on is not. GForgegroup took the initiative to package on VMWare images, however after download quite a significant amount of work was needed to provision a production ready server that is easy to admin. So the value add here are the steps and reasoning to set up a production ready GForge Community Edition Server. Instructions for FusionForge are upcoming, but for much of this they will be similar in nature.

SolengTech GForge AS Community Edition Download

The download image is based on GForge 5.7b2 (rc1 coming soon) community edition vmware image. It has been updated, and enhanced to
two distributions:
   1) the GForge AS 5.7 community edition;
   2) SolengTech customizations from years of GForge and SCM experience.

This test image contains the Community Edition of GForge AS, which allows unlimited users and project members, but has some of the advanced features of GForge AS removed.

To use this vmware image, first install VMWARE, whether it's vmware workstation, server or player and open the gforge-ce-56-SolengTech directory in vmware.

You should allocate 768MB of RAM to the vmware image for best performance.

The root password of the machine is testpass123 and you should change that immediately.

The gforge site administrator username is 'gforgeadmin' with a password of 'gforgeadmin' - again change this immediately. This is NOT a login account.

The mailman site password is also testpass123 and should also be changed immediately.

passwd root
passwd gforgeadmin
/usr/lib/mailman/bin/mmsitepass

NETWORK CONFIGURATION


Login with the root password as outlined above and
type 'system-config-network' to set up networking.
If you use dhcp networking to have an ip address
assigned, you can type 'ifconfig' to get the IP address.

Then type

cd /opt/gforge5
php change-hostname.php

and set the hostname to the IP address you just got.

You can then access your GForge using your browser by
entering the IP Address into the URL bar.

OPTIONAL HOSTNAME CONFIGURATION


The vm is configured by default with a hostname of
'gforgedemosolengtech.biz.tm'. To change this, first your
network administrator should create a new DNS entry
with the IP address of your vm. The linux OS
has several files that must be changed to accept
the new hostname, and they are detailed here:

* /etc/mail/local-host-names
* /etc/hosts
* /etc/sysconfig/network

* /etc/gforge/gforge.conf
* /etc/gforge/httpd.conf
* /etc/gforge/plugins/mailman/mailman.conf

When you are done making these changes, you
will need to restart sendmail and apache:

service sendmail restart
service httpd restart
service mailman reload

And rebuild the gforge configuration cache:

php /opt/gforge5/bin/create_config_cache.php

You are now ready to use your GForge AS installation.

SOLENGTECH GFORGE CHANGES

The following changes were applied to the base gforge release:gforge-ce-56-SolengTech.7z. They reflect many hours of work, and experiense using gforge as a production server.  There are also recommendations for free tools applicable to a home network. In other words, this package can be used in a corporate environment or a home network accessible to  the public internet.

These changes allow a gforgedemo machine to come up seamlessly on a DHCP network. These are for documentation, and the

1) Change hostname to gforgedemo.solengtech.biz.tm.

Packages like sendmail and mailman are easier to configure if the hostname is a fully qualified domain name (fqdn).

vi /etc/idmapd.conf
Domain = solengtech.biz.tm

vi /etc/sysconfig/network
HOSTNAME=gforgedemo.solengtech.biz.tm
DHCP_HOSTNAME=gforgedemo

2) Updated to current Centos. Result was updates to 240 packages, and installed 17.

yum -y update

3) Change hostname from gforgedemo.com to gforgedemo.

This allows the machine to work seamlessly in an envirnment where the domainname is supplied by the network.

hostname gforgedemo.solengtech.biz.tm
cd /opt/gforge5;php change-hostname.php (gforgedemo)
vi /etc/mail/local-host-names (gforgedemo)
vi /etc/hosts (replace centos with gforgedemo)

4) Remove park-www.trellian.com, relay.comanche.denmark.eu from mailman.

This appears to be a spammer exploit of mailman in the original distribution.The default config.pck in the Mailman archives contains this which is used as default for all new lists. A pretty clever change, as this is not the normal location for defaults. The references were removed from default configurations.

Without this fix, the web interface of all lists use park-trellian.com as the default domain, and you seamlessly move to thier site from the mailman web interface. Also, you will start getting mail bounces as it attempts to email to park-www.trellian.com.

service mailman stop
cd /var/lib/mailman
grep -lR trellian *
grep -lR trellian * | xargs -n 1 rm -f
grep -lR trellian *
service mailman start

5) Re-create mailman default list.
  •  Set site password to testpass123
  •  Set domain name (solengtech.biz.tm) to allow list to be created.
  •  Add mailman list hooks into /etc/aliases (displayed after list creation)

cd /usr/lib/mailman/bin
./newlist mailman
vi /etc/aliases (copy in mailman mailing list aliases from newlist before GFORGEBEGIN.)


6) Install gnome desktop environment, and vncserver.

This is a much more friendly way to monitor and configure the machine. The memory usage can be temporary if no one logs in at runlevel 5, little memory will be used. The console can remain runlevel 3.

yum -y groupinstall "Gnome Desktop Environment"

Results in 153 package installs.

6a) Install Development Tools

This is optional, but if you are managing software projects with the forge, it is convenient to have a GCC development environment around.

yum -y groupinstall "Development Tools"
# LTIB packages for rpm based systems.
yum install glibc glibc-devel sudo zlib rpm rpm-build ncurses ncurses-devel

7) Change mailman site configuration to gforgedemo.

(hostname as fqdn negates need for this, i.e. gforgedemo.solengtech.biz.tm)
(hostname gforgedemo.solengtech.biz.tm)

/usr/lib/mailman/Mailman/mm_cfg.py (fqdn to gforgedemo)
/usr/lib/mailman/bin/config_list (gforgedemo, all lists)

8) Install vnc server, and sendmail configuration package.

yum -y install tigervnc tigervnc-server sendmail-cf
make -C /etc/mail;service sendmail restart
cd ~;vncserver testpass123
cd .vnc/
# (replace twm with exec gnome-session)
vi xstartup
vi /etc/sysconfig/vncservers (Add root on boot)
chkconfig vncserver on

9) Reboot, and use new vnc interface.

10) Install Firefox, evolution, system configs. Configure GNOME desktop.

yum -y install firefox evolution
yum -y install system-config-network system-config-date system-config-users
Add launchers to panel: terminal, firefox, evolution, weather, system monitor.
system-config-date (set time zone)

11) Add webmin, and yum repos.

For instructions and references:
http://blogs.adobe.com/acroread/2008/02/adobe_reader_now_available_via.html
http://www.adobe.com/products/reader/dlm/firefox_steps.html

Install Adobe Repo
wget http://linuxdownload.adobe.com/linux/i386/adobe-release-i386-1.0-1.noarch.rpm
rpm -Uvh adobe-release-i386-1.0-1.noarch.rpm
yum -y install flash-plugin AdobeReader_enu

Install Webmin Repo
Create the /etc/yum.repos.d/webmin.repo  file containing :
           cat >/etc/yum.repos.d/webmin.repo <<EOF
[Webmin]
name=Webmin Distribution Neutral
baseurl=http://download.webmin.com/download/yum
enabled=1
EOF

You should also fetch and install my GPG key with which the packages are signed, with the command : rpm --import http://www.webmin.com/jcameron-key.asc You will now be able to install with the command : yum install webmin


## WARNING, do not use virtualmin on this configuration at this time,
## repo conflicts, mailer change etc...maybe later
rpm --import http://www.webmin.com/jcameron-key.asc
wget http://www.webmin.com/download/rpm/webmin-current.rpm
rpm -Uvh webmin-*.rpm
yum -y install yum-utils (package-cleanup):

http://wiki.centos.org/AdditionalResources/Repositories/RPMForge
yum install yum-priorities

Edit the .repo files in /etc/yum.repos.d/ and set up priorities by adding the line:

priority=N

[base], [addons], [updates], [extras] ... priority=1
[centosplus],[contrib] ... priority=2
Third Party Repos such as rpmforge ... priority=N  (where N is > 10)

Add the rpmforge repo(supported by Centos/RedHat)
wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.1-1.el5.rf.i386.rpm
rpm --import http://dag.wieers.com/rpm/packages/RPM-GPG-KEY.dag.txt
rpm -K rpmforge-release-0.5.1-1.el5.rf.*.rpm
rpm -i rpmforge-release-0.5.1-1.el5.rf.*.rpm
yum -y update

11a) Upgrade pear
Many of the Webmin packages require Pear, and they should be updated to the current release.
Newer releases drop Pear, and replace with setuptools (easy_install)

pear upgrade --force Archive_Tar
pear channel-update pear.php.net
pear update-channels
pear upgrade-all

11b) CPAN

Wait on this step until you need it... It will be part of the normal build soon.
# Need to install applypatch from CPAN authors.
wget http://www.perl.com/CPAN/authors/Johan_Vromans/makepatch-2.04.tar.gz
tar zxf makepatch-2.04.tar.gz
cd makepatch-2.04
perl Makefile.PL
make;make install

# Need to install ncftp on CentOS
# http://www.question-defense.com/2010/01/25/install-ncftp-ncftpget-ncftpput-using-yum-on-centos-linux-server

          
# Install the EPEL Repo for Redhat/CentOS 5
wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm #check the release to make sure it's the latest.
rpm -Uvh epel-release-*.rpm

# Install the packages that CPAN would like to have.
yum -y install ncftp pam-devel lynx postgresql-devel cpanspec perl-YAML
# (perform configuration, SAVE to DISK)
cpan
no
o conf auto_commit yes
o conf use_sqlite yes
o conf prerequisites_policy follow
o conf build_requires_install_policy yes
o conf make_install_arg UNINST=1
o conf mbuild_install_arg --uninst 1
quit

cpan
 # install YAML (If perl-YAML is not available.)
  install LWP::UserAgent
install Bundle::CPAN
install CPAN::SQLite
 reload cpan
 reload index
 force install warnings
 install Authen::Libwrap Authen::PAM DBD::Pg DBD::mysql
 install DBI IO::Pty Net::LDAP Net::SSLeay Sys::Syslog
 upgrade

12) Get new git plugin (already installed after GForge 5.7rc1)
yum -y install git
cd /opt/gforge5/plugins
mv scmgit scmgit-orig
svn co https://svn1.gforge.com/svn/scmgit
mkdir /var/lib/gforge/gitroot;chown apache:apache /var/lib/gforge/gitroot
cd /
ln -s /var/lib/gforge/gitroot
su - gforge
psql -U gforge gforge5 -c "INSERT INTO PLUGIN(plugin_name, plugin_desc, plugin_type, plugin_order, exclusive_type) VALUES('scmgit', 'Git repository', 'project', 11, 'scm')"
exit
vi /opt/gforge5/cron15
# Add '$(base_str)/plugins/scmgit/cronjobs/create_git.php'
vi /opt/gforge5/crondaily
# Add '$(base_str)/plugins/scmgit/cronjobs/parse_git_history.php'
# From web, create git template project
# unix/project name git_project
# description: "Basic template project for git"
# scm: git
# No option to change to template, must do manually.
su - gforge
psql gforge gforge5
gforge5=> update project set is_template=true,template_project_id=NULL where unix_name='git_project';
# Check results
gforge5=> select project_id,unix_name,template_project_id,is_template from project;
 project_id |   unix_name   | template_project_id | is_template
------------+---------------+---------------------+-------------
          2 | cvs_project   |                     | t
          3 | svn_project   |                     | t
          1 | empty_project |                     | t
          4 | support       |                   3 | f
          5 | testcvs       |                   3 | f
          6 | testsvn       |                   3 | f
          8 | git_project   |                     | t
(7 rows)
\q
exit
php /opt/gforge5/bin/create_config_cache.php

13) Daily Backup
Backups can be done with this script, or using Webmin.

# Optional method (backups not optional)
# you may do this with webmin as well.
cd /root
scp xxxx:backup.cron .
ln -s /root/backup.crom /etc/cron.daily/backup.cron
mkdir -p /mnt/backup/unix/last-full

14) Postgres Backup

mkdir /var/lib/pgsql/db_repository
chown postgres:postgres /var/lib/pgsql/db_repository
chmod 0700 /var/lib/pgsql/db_repository
(schedule daily backups to PostgreSQL Backup from Webmin to this directory)
https://gforgedemo:10000/postgresql

15) Webmin Backup

https://gforgedemo:10000/backup-config/
(
 Local file: /mnt/backup/webmin-backup.tar
 check: Webmin module configuration, Server configuration files
 Scheduled Backups-> Simple Schedule, Weekly
)


16) Set up host shares

Example cifs mounts in fstab.

vi /etc/fstab - add examples

17) Add essential updates to login screen.

cd /root && vi .bashrc

18) Add TrueType Fonts

# Created for GForge AS (/etc/gforge/gforge.conf TrueType)

vi /etc/gforge/gforge.conf
yum -y install cinelerra
# yum -y install fonts-*
mkdir -p /usr/share/fonts/TrueType
updatedb
cd /usr/share/fonts/TrueType
locate .ttf | xargs -n 1 ln -s
cd /opt/gforge5
php /opt/gforge5/bin/create_config_cache.php
php /opt/gforge5/bin/create_lang_cache.php

19) GForge Cron15

# Remove buggy CVS cron, and add GIT Plugin Cron
vi /opt/gforge5/cron15.php
`$base_str/plugins/scmgit/cronjobs/create_git.php`;

20) Update databases to current

vi /etc/gforge/gforge.conf
# $config['name']='SolengTechGForge'; (Personalize your GForge)

cd /opt/gforge5
php upgrade.php
php check-deps.php

21) Add a Theme

Here's how to add a theme...

psql gforge gforge5
insert into theme (theme_id,dirname,fullname)
  values(2,'solengtech5','SolengTech 5 default theme');

(already configured in VMWare image)

22) Enabling Root login for GNOME

   1. Login in as a regular user and open the terminal (command line) and edit the configuration text file

      su -c 'gedit /etc/pam.d/gdm'

   2. Locate the line that that read as follows:

      auth required pam_succeed_if.so user != root quiet

   3. Remove or comment out line by prefixing #.

      # auth required pam_succeed_if.so user != root quiet

   4. Save and close the editor.

On Fedora 11, you also need to edit /etc/pam.d/gdm-password, following the above steps.

23) Shrink VM's

http://www.howtoforge.com/how-to-shrink-vmware-virtual-disk-files-vmdk

2 Windows Host

Before we try to shrink the virtual disk files, we should try to remove any unneeded files from the virtual machine to free space. For example, on Debian-based VMs, you can run

    yum clean all
    pear clear-cache
    cpan clean

to clear out the local repository of retrieved package files.

Next, run

    cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill

to fill the unused space with zeros.

Then power down the VM and open the command window on the Windows host:
Navigate to the directory where the .vmdk files are located, e.g.:

cd C:\VirtualMachines\gforge-ce-56-SolengTech

Try to find out where the vmware-vdiskmanager.exe program is located on your Windows system (mine is C:\Programme\VMware\VMware Server\vmware-vdiskmanager.exe), and how your .vmdk file is named (e.g. Other Linux 2.6.x kernel.vmdk). You can then shrink the .vmdk file as follows:

"C:\Program Files\VMware\VMware Server\vmware-vdiskmanager.exe" -k "Red Hat Enterprise Linux 4-cl1-000001-cl8.vmdk"

That way I was able to shrink a .vmdk file from ~1.6GB to 1.3GB, and compressed (.zip) from ~430MB to 240MB.

del *.vmem

Getting rid of vmem files...
http://vikashkumarroy.blogspot.com/2008/12/getting-rid-of-vmem-file-with-vmware.html

Comments