Limiting and Logging

• limiting: making sure users don't use more than their fair share of
  • cpu time (limit command)
  • disk space (quotas)
  • printing
  • other resources

  proactive strategy: prevent resource misuse.

• Kinds of limits
  • soft: user can override, 'suggestion'
  • hard: user can't override, 'absolute'.
• logging: what did users do, and what will it cost them?
  • cpu time.
  • disk.
  • printing.
  • other resource uses.

  reactive strategy: charge for resource abuses/kick people off after they transgress.

• Kinds of logging:
  • normal activity: useful for debugging
  • exceptional activity: useful for tracing security breaches.

Basic policy decision

• are we retroactive or proactive?
• what risk is there in letting people do what they want?
• how hard is it to clean up afterward?

ECE/CS policy: mostly retroactive

• easy to trace who did what
• better to let everyone do everything, and then kick people off when they abuse privilege.
• acid test: is what you're doing keeping other people from doing what they need to do. If so, you lose.

Business policy: mostly proactive

• prevent abuses
• prevent misuse.
• much more likely that abuses will affect operations.

Logging:

• to know what happened.
• to investigate security breaches.
• to charge people for their use.

Caveats:

• if you log something then you have to clean up logs periodically or they fill up disk.
• logs are useless if no one audits them for problems.
• it takes computer time to log things.
• there may be legal restrictions on what logs you should keep or not keep.
• "If you aren't monitoring it, you aren't managing it."
• "If you aren't auditing the logs, you aren't monitoring it."

Important logs kept for all UNIX's:

• utmp (command: who)
• wtmp/lastlog (command: last)
• pacct (command: lastcomm)

Active logins: /var/adm/utmp

• /var/adm/utmpx on suns, /var/log/utmp on linux
• current users logged in now
• always enabled; can't turn it off
• commands to read this: w and who

Past logins: /var/adm/wtmp

• (/var/adm/wtmpx on suns, /var/log/lastlog on linux, /var/adm/lastlog elsewhere)
• login history
• always enabled; can't turn it off
• commands to read it: last <user>
• can't be deleted in multiuser mode.
  • kernel opens.
  • if you rm an open file, it persists.
  • kernel puts the file back on reboot.
  • to delete, turn off logging, cat /dev/null >/var/adm/wtmp, turn on logging again.
• should be saved as archive in case of legal problems. Can tell when specific users were present on the system.

Command history: /var/adm/pacct

• complete list of all commands typed by everyone.
• most common reason for running out of space on /var
• sun command to start accounting:

   /usr/sbin/acct/accton /var/adm/pacct
  

• sun command to stop accounting:

   /usr/sbin/acct/accton
  

• produce human-readable display of accounting info:

   /usr/sbin/acct/acctprc1 </var/adm/pacct | more
  

• produce summary of accounting over time:

   /usr/sbin/acct/acctprc1 </var/adm/pacct | /usr/sbin/acct/acctprc2 >tacct
  

• commands to print results:

   /usr/sbin/acct/prtacct tacct
  

• command to merge data of different types: acctmerge.

Unified model of accounting:

• a hack
• developed over time by too many people
• developed at a time when UNIX was funded by chargeback.

pacct (BIG)

• easiest way to bring down your system.
• transform,summarize-> daily, weekly summaries (smaller)
• acctmerge-> unified summaries (even smaller)
• charge for services-> billing reports (per-user fees)

Other accounting:

• command to print disk usage: diskusg /dev/rrz6c > dacct makes an ascii report on disk usage on your machine.
• command to initiate chargebacks for disk usage: /usr/sbin/acct/dodisk
• /usr/sbin/acct/acctcon1 </var/adm/wtmp: connect time accounting

The problem with logging

• Most logging is essentially printing out status messages.
• But daemons don't have a controlling terminal.
• "In space, no one can hear you scream!"
• One approach: syslog

syslog

• System log facility.
• managed by a daemon syslogd.
• syslog system call writes to daemon.

  contents of logging/syslog.cc...
  #include <syslog.h>
  // void openlog(const char *ident, int logopt, int facility);
  // void syslog(int priority, const char *message); 
  //  void closelog(void);
  
  main() { 
      openlog("foo", 0, LOG_USER); 
      syslog(LOG_INFO, "hi there, I'm me"); 
      closelog(); 
  } 
  ...end of logging/syslog.cc
  

  This produces the following log entry:

   Apr 28 14:37:28 sunfire06 foo: [ID 301932 user.info] hi there, I'm me
                                                        ^^^^^^^^^^^^^^^^ message
                                                  ^^^^ priority
                                             ^^^^ facility
                                   ^^^^^^^^^ process identifier
                             ^^^ identifier in logopen call
                   ^^^^^^^^^ machine
   ^^^^^^^^^^^^^^^ date
  

• Facility: the subsystem to which the logfile applies: (copied from man syslog)
  • LOG_KERN: Messages generated by the kernel. These cannot be gen- erated by any user processes.
  • LOG_USER: Messages generated by random user processes. This is the default facility identifier if none is specified.
  • LOG_MAIL: The mail system.
  • LOG_DAEMON: System daemons, such as in.ftpd(1M).
  • LOG_AUTH: The authorization system: login(1), su(1M), getty(1M).
  • LOG_LPR: The line printer spooling system: lpr(1B), lpc(1B).
  • LOG_NEWS: Reserved for the USENET network news system.
  • LOG_UUCP: Reserved for the UUCP system; it never used syslog.
  • LOG_CRON: The cron/at facility; crontab(1), at(1), cron(1M).
  • LOG_LOCAL0: Reserved for local use.
• Priority: how important this message is: (copied from man syslog)
  • LOG_EMERG: A panic condition. This is normally broadcast to all users.
  • LOG_ALERT: A condition that should be corrected immediately, such as a corrupted system database.
  • LOG_CRIT: Critical conditions, such as hard device errors.
  • LOG_ERR: Errors.
  • LOG_WARN: Warning messages.
  • LOG_NOTICE: Conditions that are not error conditions, but that may require special handling.
  • LOG_INFO: Informational messages.
  • LOG_DEBUG: Messages that contain information normally of use only when debugging a program.
• shell writing to daemon: logger

   logger -p daemon.notice "9 o'clock and all is well"
  

  • daemon: the facility to which the log entry applies.
  • notice: how important is this message?

Log management

• Logs get really, really big!
• redirection: where do logs get stored.
• rotation: keeping logfiles small.
• monitoring: detecting specific events.

Log redirection

• You don't have to store logs on the local machine.
• Syslogd will accept requests from a network.
• LOGHOST: a machine that you've designated to collect all log information.
• Note: there is no security; a hacker can fill your loghost with trash!

Syslogd configuration

• configuration file: syslogd.conf.
• goal: identify log messages that
  1. get stored
  2. get printed to peoples' sessions.
  3. spawn an external process of some kind, e.g., page an operator.

      #ident        "@(#)syslog.conf        1.5     98/12/14 SMI"   /* SunOS 5.0 */
      #
      # Copyright (c) 1991-1998 by Sun Microsystems, Inc.
      # All rights reserved.
      #
      # syslog configuration file.
      #
      # This file is processed by m4 so be careful to quote (`') names
      # that match m4 reserved words.  Also, within ifdef's, arguments
      # containing commas must be quoted.
      #
      *.err;kern.debug;daemon.notice;mail.crit      /var/adm/messages
      
      *.alert;kern.err;daemon.err                   operator
      *.alert                                               root
      
      *.emerg                                               *
      
      # if a non-loghost machine chooses to have authentication messages
      # sent to the loghost machine, un-comment out the following line:
      #auth.notice                  ifdef(`LOGHOST', /var/log/authlog, @loghost)
      
      mail.debug                    ifdef(`LOGHOST', /var/log/syslog, @loghost)
      
      #
      # non-loghost machines will use the following lines to cause "user"
      # log messages to be logged locally.
      #
      ifdef(`LOGHOST', ,
      user.err                                      /dev/sysmsg
      user.err                                      /var/adm/messages
      user.alert                                    `root, operator'
      user.emerg                                    *
      )
      user.info             /var/opt/SUNWut/log/messages
      local1.info           /var/opt/SUNWut/log/admin_log
     

• /var/adm/messages (/var/log/messages): main system log.
• /var/opt/SUNWut/log/messages: store user data here.

Log monitoring

• reading files is really, really slow.
• several log monitoring tools are useful.
• simple monitor: swatch
• my (silly) monitor: peep swatch
• watches logs on the loghost.
• matches regular expressions in the log.
• emails people, restarts machines, etc.

swatch configuration

• matches
  • watchfor /{pattern}/ - watch for a particular pattern
  • ignore /{pattern}/ - ignore this pattern if it comes up.
• actions: (apply to most recent action)
  • mail addresses={user-list};subject={subject} - mail users if this happens.
  • pipe {program-name} - print matching lines on input of given program.
  • exec {program-name} - execute given program
  • threshold {events}:{seconds} - do one action per {events} matches in {seconds} seconds.
  • several more options.
• example configuration:

   watchfor /ssh.*denied/
     threshold 100:60
     mail addresses:couch@cs.tufts.edu;subject='dictionary attack in progress'
  

  • /ssh.*denied/: a line with both ssh and denied, separated by arbitrary other characters (.*) *Perl regular expressions!*.
  • if this happens more than 100 times in one minute, there's a hacker trying to break in; stop him!

Log rotation

• Logs get really, really large!
• Most common cause of filling up filesystems: logs overflowing.
• "log rotation": simple rotation strategy: (once a day, e.g.)

   /var/log/messages => 
   /var/log/messages.1 => 
   /var/log/messages.2 => 
   /var/log/messages.3 => 
   /var/log/messages.4 => discarded!
  

• Called by "cron":

   10 3 * * * /usr/sbin/logadm
  

  Every day, at 3:10 am, run /usr/sbin/logadm. This rotates the logs, as above.

Limiting

• csh limit command (for cpu and other resources)
• disk use by disk quota
• service access: /etc/hosts.allow

csh limit command: (man csh)

• limit resource my-limit
• limit coredumpsize 1M
  • limits the size of core dumps to 1 megabyte (comp11)
• Controllable resources currently include
  • addresspace (the maximum address space in bytes for a process),
  • coredumpsize (the size of the largest core dump that is created),
  • cputime (the maximum number of CPU seconds to be used by each process),
  • datasize (the maximum growth of the data region allowed beyond the end of the program text),
  • descriptors (the maximum number of open files for each process),
  • filesize (the largest single file that can be created),
  • memoryuse (the maximum size to which a process's resident set size can grow), and
  • stacksize (the maximum size of the automatically extended stack region)." (man csh)
• Can be done in /etc/csh.cshrc (soft limits)
• Can also be controlled by modifying behavior of /bin/login (hard limits) using the ulimit system call (man 3 ulimit) In this case, user can't override limits.

Disk quotas

• soft limit: you can exceed this for awhile.
• hard limit: you can't exceed this.
• per device (not directory, you can't hide things)

Quota handling:

• mount option for ufs, nfs. 'userquota' or 'groupquota' ('groupquota' is dec-specific and not found elsewhere) (quotas IGNORED unless files mounted)
• quotas in filesystem: top level file 'user.quotas' lists quotas BINARY file, edited through 'edquota'

   # more /etc/fstab
   /dev/rz0a        /              ufs rw 1 1
   /proc            /proc          procfs rw 0 0
   /dev/rz1c        /usr           ufs rw 1 2
   /dev/rz2c        /var           ufs rw 1 2
   /dev/rz0b        swap1          ufs sw 0 2
   /dev/rz5c        /export/0      ufs rw 1 3
   /dev/rz6c        /export/1      ufs rw,userquota 1 4
   # quotacheck /dev/rz6c # figures out who's been naughty or nice
   # quota -v couch
   Disk quotas for user couch (uid 30): 
        Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
                 /       0       0       0               0       0       0        
              /usr       0       0       0               0       0       0        
              /var       0       0       0               0       0       0        
             swap1       0       0       0               0       0       0        
         /export/0       0       0       0               0       0       0        
         /export/1     646   50000  100000              97   50000  100000        
   # edquota couch 
   # quota -v couch
       Quotas for user couch:
       /export/1: blocks in use: 646, limits (soft = 50000, hard = 100000)
             inodes in use: 97, limits (soft = 50000, hard = 100000)
   
   # edquota -p couch comp150 # sets quotas for couch to those for comp150
  

Caveats:

• necessary to design quotas for appropriate behavior
  • should you 'overbook'?
    • yes: there's a chance that the system will go down with people behaving themselves
    • no: might never use your disk space.