System and Network Administration

lecture in color

So far, we know:

How to build a system
How to compile and install software
How to set up a user environment.
But we don't know anything yet.

Systems Maturity Model

'How well' the process of managing systems works.
Level 1: ad-hoc: randomly follow scripts.
Level 2: document changes.
Level 3: document 'best practices' for others to follow.
Level 4: assure enterprise-wide compliance with 'best practices'

The goal: "System Integrity"

System does what you told it to.
It doesn't stop doing it.
No matter what.
Even with you around...

Maintaining integrity:

Revision control: how do you remember changes made to a system?
Change control: how do you assure that changes take place in an orderly manner, with the ability to undo them?
Cookie cutters: replicating successful approaches.
Change detection: how do assure that others do not compromise your systems?

Revision control:

you're changing a lot of files.
you can't necessarily remember the changes.

Types of revision control:

by file
by directory hierarchy
by computer
by network

Revisions of files

Old: Source Code Control System (SCCS).
Current: Revision Control System (RCS): Walter F. Tichy, Purdue.
keeps a `revision history' of changes made to a file.
maintains that history in minimal space.

RCS Commands:

ci
- check in, save a copy of a file.
co
- check out, get a saved file.
rcs
- revision control system, changes the state of a file.
rlog
- revision log, gives the history of revisions made to a file.
rcsdiff
- revision diff, allows you to see the differences between revisions.

How to use RCS

Make an RCS directory
```
 % mkdir RCS   
```
This is a directory to hold revisions. This should be at the level of the file you are working on, say "foo".
When starting up perform this command:
```
 % ci foo       
```
"Check in" the current contents of "foo" as a revision. You will be asked to comment on the revision. Revisions of "foo" are stored in the file "RCS/foo,v". foo IS DELETED by this command. Don't feel bad, just check it out as noted below.
To get a read-only copy, type:
```
 % co foo      
```
"Check out" the most current copy of "foo" from "RCS/foo,v". This is a read-only version that you can compile but can't edit.
To edit changes into the file, use
```
 % co -l foo   
```
This command `locks' the file so that only you can check it back in. You should edit it and then check it in so that others may edit it as well. To check it back in use the ci command.
If you want to continue editing after posting changes, use
```
 % ci -l foo   
```
This checks in the file and keeps it locked to you.
To break a lock set by another person:
```
 % rcs -u foo  
```
This unlocks the file, sending mail to the offender who locked it.
```
 % co -l foo   
```
This checks the file out so you can edit it.
To revert a file to a previous revision:
```
 % rlog foo    
```
Make a listing of all revisions in "RCS/foo,v". This includes all comments given during updating.
```
 % co -r2.3 foo  "Check out" revision 2.3 of "foo" from "RCS/foo,v".
```
Revision numbers are given in the output of rlog.
```
 % rcs -l foo  
```
Lock the CURRENT revision of foo.
```
 % chmod u+w foo 
```
Make foo editable.
```
 % ci foo      
```
Make the former revision the current one.
To see what's different about the current file and revision 1.2,
```
 % rcsdiff -r1.2 foo | more
```

RCS comments

saving revisions is VERY EFFICIENT: typically one can store 40-50 revisions in the space of two copies of a file.
revisions are stored in REVERSE DIFFERENTIAL FORM.
- start with current file
- save changes needed to make previous file
- save changes needed to make file before that.
- ...
utilizes command 'diff' to compute differences.

Diff

command reports differences in files.
several options useful for software maintenance.

-c : create a "context diff" showing how changes fit in context.

then one can use "patch" to update a file like the first to a file like the second!

Example, if file1 is

 This is a test.
 This is another test. 
 This is a third test.  
 This is a fourth test.

and file2 is

 This is a test.
 This is another test. 
 This is a dumb test.  
 This is a fourth test. 
 This is a fifth test.

then

 diff -c file1 file2

produces

 *** file1       Tue Feb 29 14:28:06 2000
 --- file2       Tue Feb 29 14:28:19 2000
 ***************
 *** 1,4 ****
   This is a test.
   This is another test. 
 ! This is a third test.  
   This is a fourth test. 
 --- 1,5 ----
   This is a test.
   This is another test. 
 ! This is a dumb test.  
   This is a fourth test. 
 + This is a fifth test.

-e : create a script of ed commands that'll change one file into another!
- then one can use "sed" to change one file+ the diffs into the other!
- used by RCS to compress revision histories!
- Example: for the same two files above,
```
 diff -e file1 file2 
```
  produces:
```
 4a
 This is a fifth test. 
 .
 3c
 This is a dumb test.  
 .
```

Using RCS in System Administration

back up copies of system files as you change them
- /etc/hosts
- /etc/inetd.conf
- /etc/passwd
back up original versions of software that you modify

Patch

Problem: have to reload whole software tree even to incorporate minor changes.
Approach: send only the changes.
Tool "patch": applies changes sent in an email message to a large hierarchy of files.
Written by Larry Wall (Perl inventor)

Using Patch

implementor makes two trees of files, 'old' and 'new'
you have 'old'.
'fixed' version is 'new'
implementor executes
```
 diff -r -c old new >patch01
```
implementor mails patch01 to you (or places it on the Internet)
you execute:
```
 cd old
 patch -p 1 <patch01
```
and all files change from the old to new versions!

Problems with single file version control

'version skew': trying to use incompatible versions of several files together.
Files in a hierarchy often must CHANGE TOGETHER.
Need a tool that keeps changes 'in sync' for a whole tree of files.

Tree Revision control: CVS

Concurrent Version System
ON TOP OF RCS
Stores revisions for a hierarchy of files.

Using CVS

Many commands starting with 'cvs'
```
 cvs checkout <module>
```
makes a private copy of the module for you to edit. cd there and then:
```
 cvs update
```
gets new changes from others.
```
 cvs add file
```
adds a new file to the module in your private working tree.
```
 cvs remove file
```
removes a file from the module in your private working tree.
```
 cvs commit file
```
copies a file from your private working tree to the master tree.
Involved startup to create modules.
Once started, many people can cooperate on creating modules.
Network version: "cvs checkout" now works over the Internet.

Limitations of RCS and CVS

very inefficient on binary files (RCS algorithm depends upon text file structure)
only works when files are located together in the filesystem.
Reality: UNIX files are distributed throughout the filesystem.
Dream: RCS/CVS for SYSTEMS, not just software.
Reality: must settle for discipline, automated checking, and automated synchronization rather than revision control.

The big compromise:

If you can't beat "them", control yourself, detect them, and change stuff back.
discipline: control what goes into your system
automated checking: find out what changed.
automated synchronization: change things back if they change inadvertently.

Integrity maintenance:

of a machine
of a network

Approaches:

policies that control what admins do. (root can work around any tool's limits)
tools that enforce state (such as rdist, distr, cfengine)
tools that check integrity (such as tripwire and aide)

Example: integrity of a machine (by policy)

Thou shalt not install files unnecessarily in / or /usr.
Thou shalt keep records of files actually installed therein.
Thou shalt maintain /local/bin as list of commands to be put in users' paths.
Thou shalt never install any file in /local/bin, always use symlinks to disjoint packages.
Thou shalt keep records of the links thou installest in /local, in /loc/slink/slink.conf.
Thou shalt compile and install packages as yourself, in disjoint filesystem subtrees (of /loc).
Thou shalt not upgrade or delete existing packages without checking with users.

SLINK: my way of maintaining links (there are several others)

link target source: makes source appear within target.
copy target source: copies files.
Examine allegro:/local/bin, /loc/slink/slink.conf for details.

Tools and policy:

root can do anything.
policy must control what root may do.
one approach: make tools sensitive to policies:
Examine allegro:/loc/slink/slink.mod for the modification policies for EECS.

My contribution

(what I learned through years of writing tools to maintain software repositories):
Tools should not ENFORCE policies (which leads to disgruntled admins fighting tools) but should REINFORCE them (by making it easier to comply than to dissent).

Case history of slink approaches:

no slink: put everything in /usr/local
- ran for 5 years.
- massive corruption in /usr/local
- had to start over.
Slink1.0 (also CMU depot) the fascist approach:
- you have to name your package subdirs in a particular way.
- all subdirs with specific names get linked into /local
- result: CHAOS. Took Schmolze a year to figure out how to install Emacs under the scheme.
- Software installation of all but the simplest packages came to grinding abrupt halt. (while corruption continued when we had to install things)
Slink2.0: the anarchist's approach:
- you get to name software the way you want.
- you have to tell slink how.
- result: works, but happily corrupts things you don't intend if you mistype. e.g., no cd command, then
```
 link /loc/fool/bin bin
```
  links stuff into /bin in the root partition!
Slink5.0: incorporating policy decisions
- you can tell slink your POLICY on modifications (2000 lines of perl to figure this out. Sigh)
- it won't do things against your wishes.
- no possibility of corruption even as root (unless you intend it)

EECS Policy advantages and disadvantages:

can more easily rebuild systems and upgrade system software.
must fight with vendor software over where it gets installed. (sometimes we lose. see /interleaf on every host)
must keep detailed lists of vendor filesystem modifications.
more short-term work to modify Makefiles to install in local hierarchies.
less long-term work in deleting packages that aren't useful.
can create multiple user environments from the same packages.

Multiple environments:

/local/bin: normal user commands
/local/old/bin: old (deprecated) commands
/local/new/bin: new (untested) commands

Software revision control:

what software is available to users?
are all files associated with software the same version?
revision skew: some files in some version, some files in another.

EECS software revision policy

put new stuff into /local/new/bin,
when tested, put into /local/bin,
while moving the old version to /local/old/bin,
and removing links to any version in /local/old/bin.
if no one complains, move unlinked source files to Trash (hidden directory on largo).
if no one complains about this, delete the Trash.

try unlinking the version in /local/old/bin. if no one complains, delete it.

                         initial
 /loc/adm/foo1.0/bin/foo -------> /local/new/bin 
                         tested
                         -------> /local/bin 
                         replaced
                         -------> /local/old/bin

lecture in color

/comp/150NET/notes/rcs-old.php
downloaded on Nov-23-2009 03:22:42 PM,
was last modified on Feb-17-2004 10:49:21 PM.
All lecture note content is copyright 2004 by
Alva L. Couch, Computer Science, Tufts University