How to operate cpacman

Contents:

Intro (read this first)
How to add a server
How to add a system class
How to install package
How to update server with lates erratas
How to maintain server metadata

Intro
Throughout this document we'll use following assumptions/defenitions
- cpacman is installed in /opt/cpacman
- $CPACMAN_HOME is set up to point to cpacman home directory (/opt/cpacman)
- lines started with `$' are shell commands
- symbols f, l, and d aside file names mean, respectively, file, link and directory
- subsections under the hood describe low level behaviour to help you understand what's going on
How to add a server
- in /opt/cpacman/servers create directory with hostname for your server (assuming your server hostname is my.site.com):
```
    $ cd $CPACMAN_HOME/servers
    $ mkdir my.site.com
    
```
- create sysinfo.cfg file inside server directory which will describe various characteristics of your server (os class - rhelas3 = "RedHat Enterprise Linux AS3")
```
    $ cd $CPACMAN_HOME/servers/my.site.com
    $ cat > sysinfo.cfg
    [os]
    id=rhelas3
    ^D
    
```
  this data will help cpacman to define which rules apply to this server
  
  in this example we identified server as server which runs rhelas3 so [rhelas3] section from cpacman.conf would apply
  
  next in case of redhat type server you have to copy contents of /var/lib/rpm into my.site.com/rpmdb so that you have local copy of server package database thus reducing trafic and being able to reproduce environment even having dead server (somewhat similar to tripwire)
How to add a system class
In previous section we used sytem class rhelas3 to describe our system, but for it to be useful we have to define couple of things:
- entry in /etc/cpacman.conf to set up some reasonable defaults for this class
- registered server ruleset to perform operations on that server
First one is quite simple: for example rhelas3 class of server needs variable rpm_db_path to lookup the packages for this operating system, other options are quite generic and it's sufficient to put them in DEFAULT section of cpacman.conf (all options from this section will be included in other sections by default):

template_db
path to configuration templates directory, usually $CPACMAN_HOME/templates

server_db
path to directory containing server entries (d) toolbox - path to directory containing cpacman binaries/scripts

For your installation you could also provide set of common defaults there as well, like this one:
rpm_base=/var/spool/rpm
to use it later on in expressions like %(rpm_base)s/rhl72 or similar.

Ok, so we've got to the point where we can describe something usefull for our rhelas3 class. Let's define lookup path for relevant packages as
rpm_db_path=%(rpm_base)s/rhelas3:%(rpm_base)s/common

what it does - it tells cpacman to look in directory /var/spool/rpm/rhelas3 for needed packages and if it's not there look in /var/spool/rpm/common.

To add more flexibility cpacman now can "inherit" some properties. Example below shows how to do it:
```
rpm_db_path=%(rpm_base)s/rhelas3.mine:{rhelas3}:%(rpm_base)s/common.as3
```
In this case cpacman will read and render rpm_db_path from base class rhelas3 and substitute it in place of {rhelas3}

Please note also that DEFAULT section is not being rendered so no inheritance there ;)

To register server ruleset (in case existing ones don't work for you) involves more programming. First of all you'd have to create your own class derived from Server.ServerRules which would "explain" how to transfer files to server, how to install them on server and how to confirm package installation. Also after you generate class MyRules you have to register it in Server.registered_rules doing:

Server.registered_rules['my_rules_name']=MyRules

this will inform cpacman to associate os id my_rules_name with MyRules class.

Now the trick is - you'd have to do it inside Server.py since there's no code to look for available plugins :(

How to install package

This involves three steps:
- find package
- "plan" installation
- run installation
- confirm installation
First of all to install package you have to find package (or at least know where it is) and use correct version. You can do so using find_package tool:
```
  $ $CPACMAN_HOME/cpacman/find_package.py -h my.site.com httpd
    Looking up relevant packages for my.site.com
       package httpd found
       (usually latest is the most recent):
          /var/spool/rpms/rhel3as/httpd-2.0.46-25.ent.i386.rpm
          /var/spool/up2date/httpd-2.0.46-26.ent.i386.rpm
  
```
what this does - it pick up server specific data like rpm_db_path from server directory and looks in all possible directories for specified package (in our case apache) reporting all instances (all versions too) and it's up to you which one you'll decide to use. As you see from the output package from /var/spool/up2date is more recent so we'll try to install it:
```
  $ $CPACMAN_HOME/cpacman/install_packages.py -h my.site.com -m "Installing \
  web server" httpd-2.0.46-26.ent.i386.rpm
  
```
In this example we are installing package httpd-2.0.46-26.ent.i386.rpm on host my.site.com and as a log message we put "Installing web server" so in the future it would be easy to determine reasons or any additional data on why this package was installed on the server.

Note that script is trying to perform fake install on local copy of server package database and it'll complain if some dependencies are missing without creating entries in 2install and 2config
under the hood

what's happening under the hood at this moment:
- package filename, location and description are being added to file 2install inside my.site.com directory
- files matching pattern .*.sh are being copied to 2config directory inside my.site.com Those files are scripts which would run after package installs - usually those scripts do initial setup or adjustment in configuration so you don't have to go after every httpd install and activate manually httpd service on reboot (that's just an example)
after this you could "fix" things a bit (in case you know what you're doing) going straight to 2install or 2config
Now as we created an installation plan let's get to install:
```
 
  $ $CPACMAN_HOME/cpacman/installer.py -h my.site.com -u root
  
```
this would pick all the files mentioned in 2install and all scripts from 2config relative to packages being installed copy them to server and the would try to install packages followed by running all relevan scripts in alphabetical order.

In simpliest case above mentioned command would ask two times for password - first time to transfer files, second time to run remote command to install them etc. After it's done it'll remove files from the server.

The only thing left after successfull install is to "confirm" (i.e. record transaction in local database):
```
  $ $CPACMAN_HOME/cpacman/confirm.py -h my.site.com 
  
```
under the hood

above operation would do next:
- move all entries from 2install to comment_rpm along with the timestamp of operation
- move all scripts from 2config to log/ so actions performed on server would be recorded
- register packages in local copy of remote package database

How to update server with latest erratas

Two new tools joined cpacman suite: up2dater.py and update.py. Both server different purposes.

up2dater.py

This tool fetches redhat errata updates from RHN.redhat.com and stores them in /var/spool/up2date. It doesn't install anything - it's only purpose is to fetch files. This tool was designed to be used from crontab or manually to keep updated local package database.

update.py

contrary to the up2dater.py this tool is actually schedules updated packages for installation. Briefly: it checks local copy of server's RPM database against local repositories that are applicable to that server for any packages that could be updated.

The procedure

Honestly - there's not much to it:
```
 $ $CPACMAN/toolbox/up2dater.py -h my.server.org
 $ $CPACMAN/toolbox/update.py -h my.server.org
```
After those two commands you'll have all relevant packages scheduled for install on server. From this point proceed just as if you did it manually through install_packages.py:
```
 $ $CPACMAN/toolbox/installer.py -h my.server.org -u root
```
Pitfalls

PGP

Most common problems are either fact that package originates from the source with unknown PGP key, which will produce message like follows:
```
warning: rpmts_HdrFromFdno: V3 DSA signature: NOKEY, key ID c431416d
can't load jpackage-utils-1.5.34-1jpp.noarch.rpm file falling back to defaults
```
To fix it - download PGP public key from vendor and install it using:
```
 $ sudo rpm --dbpath ${CPACMAN_HOME}/servers/server.name/rpmdb --import VENDOR-PGP-PUBLIC-KEY
 
```
Note that we're installing it into host system's RPM database and not local copy of server's database. For some reason I couldn't get around that problem so I can import those keys into local copy and have RPM check local copy instead of host database.

Wrong updates

With the diversity of platforms it could happen that classes defined for you system can include packages that overlap with base system and you should pay close attention to that. For example in our current implementation I had to define couple of classes for same OS level (RHEL AS3): rhelas3, rhelas3.backports4as, rhelas3.testing, rhelas3.masked; all covering same OS level but different flavor. For example only one of our machines has packages backported from AS4 installed, another couple of machines are testing boxes so they accept "beta" packages we produce. So it's always a good idea to check if updates are comming from /var/spool/up2date folder doing:
```
  $ $CPACMAN/toolbox/update.py -h my.server.org -n
  ...
```
and checking the produced list (note that no packages will be scheduled if you used "-n" switch

Failing updates

Sometimes for different reasons set of packages picked by update.py can't be installed cleanly. Most noticable indication of that - messages about failed dependancies when packages are tested. To get around it cpacman allows you to do automatic skip - just add -s to it's options:
```
  $ $CPACMAN/toolbox/update.py -h my.server.org -s
  ...
```
In this case cpacman will do it's best to install all packages but it will start getting rid of packages that generate conflicts until everything is clear. update.py will also indicate which packages were skipped and it will put some tech info about the reasons. Something like:
```
Had to skip packages
tomcat5-servlet-2.4-api-5.0.30-8jpp.noarch.rpm  /var/spool/auconda/rpms/jpp
        ((tomcat5-jasper, 5.0.30, 3au), (tomcat5-servlet-2.4-api, 0:5.0.30-3au), 8, None, 0)
```
where last line is of format: ((N,V,R), (reqN, reqV), needsFlags, suggestedPackage, sense)

How to update server with latest erratas manualy
For simplicity sake we'll assume bash is the shell of choice here
First of all we need list of packages to be installed:
```
   $ up2date --dry-run -u --dbpath=/path/to/cpacman/servers/server.name/rpmdb/
  
```
It'll generate list of packages to be updated. Now it's up to you to generate filelist out of it

Copy output of previous command (only lines with package names and versions) into some file, say "updates".

Now let's get filelist out of it
```
  $ awk /\w/{print $1 "-" $2 "-" $3 "." $4 ".rpm"} updates > update_packages
 
```
an now it's time to push it to server:
```
   $ /path/to/cpacman/toolbox/install_packages.py -h server.name -m "errata updates" `cat update_packages`
 
```

How to install batches of packages

Simply put it's partially manual process but with some minor scripting it's quite automated :). I'll use our setup as example.

In $CPACMAN_HOME create directory batches. Whenever you're doing install of bunch of files which are in fact requirements so that you can install some package X (say tomcat) before you do "confirm" but after you made sure the whole batch installs OK and produces desirable result - you can copy file 2install into $CPACMAN_HOME as install.X (in my case - install.tomcat. Last step would be to remove comments from that file so next time you can add your own comment for installation.

Now, that you have your batch saved - whenever you need to install same batch you just do something like:
```
 $ awk {print $0 "\t" "# X base install with dependancies" } \
   ../batches/install.tomcat > my.server.net/2install 
 $ $CPACMAN_HOME/toolbox/install_packages.py -h my.server.net
```
You've probably noticed empty call to install_packages.py - that's to ensure RPM can swallow transaction so that no new packages are added but cpacman will retry to apply packages listed in 2install file.

Note: abovementioned method is not the only way to work with batches it's just the way I was doing them in our setup. You can develop your own strategy with batches and it could probably be as effective or even more effective than mine depending on many factors.

How to operate cpacman

Intro

How to add a server

How to add a system class

How to install package

under the hood

under the hood

How to update server with latest erratas

up2dater.py

update.py

The procedure

Pitfalls

PGP

Wrong updates

Failing updates

How to update server with latest erratas manualy

How to install batches of packages