The Administrative Environment

• Filesystems
• Databases
• Services

Filesystem definition

• Disk drive: the physical device that contains files and directories.
• Partition: a section of a physical disk drive, perhaps the whole disk.
• Device: the operating system's name for a partition. Example: /dev/dsk/c0t2d0s2
• Filesystem: a structure of data within a device that allows the operating system to treat it as a directory containing files.
• Mounting: the act of mapping a filesystem into the directory structure of a computer.

Making a filesystem

• Partition the disk appropriately.
• Make a filesystem on the disk.
• Mount the filesystem within your directory structure.
• Copy files into it if necessary.

Raw Disk Geometry

• Cylinder: one stripe of data circumnavigating the whole physical disk (utilizing multiple Tracks/Heads)
• Track/Head: one media reading implement on a disk.
• Sector: a part of a track, usually 512 bytes of data.
• Cylinders * Tracks/Cylinder * Sectors/Track * Bytes/Sector = Raw Bytes.
• The triple (Cylinders, Heads(or Tracks), Sectors) is called the geometry of a disk.
• For modern disks, this is fiction. Geometry is variable from outside to inside cylinders (more path length means higher density).

Partitioning

• Under Sun Solaris, use the format command:

  contents of admin-old/partition.txt...
  # format
  Searching for disks...done
  
  
  AVAILABLE DISK SELECTIONS:
         0. c0t2d0 <MICROP-1598-15MD1052405-AS24 cyl 1911 alt 2 hd 15 sec 71>
            /iommu@0,10000000/sbus@0,10001000/espdma@5,8400000/esp@5,8800000/sd@2,0
         1. c0t3d0 <SUN1.05 cyl 2036 alt 2 hd 14 sec 72>
            /iommu@0,10000000/sbus@0,10001000/espdma@5,8400000/esp@5,8800000/sd@3,0
  Specify disk (enter its number): 0
  selecting c0t2d0
  [disk formatted]
  
  
  FORMAT MENU:
          disk       - select a disk
          type       - select (define) a disk type
          partition  - select (define) a partition table
          current    - describe the current disk
          format     - format and analyze the disk
          repair     - repair a defective sector
          label      - write label to the disk
          analyze    - surface analysis
          defect     - defect list management
          backup     - search for backup labels
          verify     - read and display labels
          save       - save new disk/partition definitions
          inquiry    - show vendor, product and revision
          volname    - set 8-character volume name
          !<cmd>     - execute <cmd>, then return
          quit
  format> part
  
  
  PARTITION MENU:
          0      - change `0' partition
          1      - change `1' partition
          2      - change `2' partition
          3      - change `3' partition
          4      - change `4' partition
          5      - change `5' partition
          6      - change `6' partition
          7      - change `7' partition
          select - select a predefined table
          modify - modify a predefined partition table
          name   - name the current table
          print  - display the current table
          label  - write partition map and label to the disk
          !<cmd> - execute <cmd>, then return
          quit
  partition> print
  Current partition table (original):
  Total disk cylinders available: 1911 + 2 (reserved cylinders)
  
  Part      Tag    Flag     Cylinders        Size            Blocks
    0       root    wm       0 -   61       32.24MB    (62/0/0)     66030
    1       swap    wu      62 -  185       64.48MB    (124/0/0)   132060
    2     backup    wu       0 - 1910      993.76MB    (1911/0/0) 2035215
    3 unassigned    wm       0               0         (0/0/0)          0
    4 unassigned    wm       0               0         (0/0/0)          0
    5 unassigned    wm       0               0         (0/0/0)          0
    6        usr    wm     186 - 1910      897.03MB    (1725/0/0) 1837125
    7 unassigned    wm       0               0         (0/0/0)          0
  
  partition> 3
  Part      Tag    Flag     Cylinders        Size            Blocks
    3 unassigned    wm       0               0         (0/0/0)          0
  
  Enter partition id tag[unassigned]: 
  Enter partition permission flags[wm]: 
  Enter new starting cyl[0]: 186
  Enter partition size[0b, 0c, 0.00mb, 0.00gb]: 200c
  partition> print
  Current partition table (unnamed):
  Total disk cylinders available: 1911 + 2 (reserved cylinders)
  
  Part      Tag    Flag     Cylinders        Size            Blocks
    0       root    wm       0 -   61       32.24MB    (62/0/0)     66030
    1       swap    wu      62 -  185       64.48MB    (124/0/0)   132060
    2     backup    wu       0 - 1910      993.76MB    (1911/0/0) 2035215
    3 unassigned    wm     186 -  385      104.00MB    (200/0/0)   213000
    4 unassigned    wm       0               0         (0/0/0)          0
    5 unassigned    wm       0               0         (0/0/0)          0
    6        usr    wm     186 - 1910      897.03MB    (1725/0/0) 1837125
    7 unassigned    wm       0               0         (0/0/0)          0
  
  partition> 4
  Part      Tag    Flag     Cylinders        Size            Blocks
    4 unassigned    wm       0               0         (0/0/0)          0
  
  Enter partition id tag[unassigned]: 
  Enter partition permission flags[wm]: 
  Enter new starting cyl[0]: 386
  Enter partition size[0b, 0c, 0.00mb, 0.00gb]: 1525c
  partition> print
  Current partition table (unnamed):
  Total disk cylinders available: 1911 + 2 (reserved cylinders)
  
  Part      Tag    Flag     Cylinders        Size            Blocks
    0       root    wm       0 -   61       32.24MB    (62/0/0)     66030
    1       swap    wu      62 -  185       64.48MB    (124/0/0)   132060
    2     backup    wu       0 - 1910      993.76MB    (1911/0/0) 2035215
    3 unassigned    wm     186 -  385      104.00MB    (200/0/0)   213000
    4 unassigned    wm     386 - 1910      793.03MB    (1525/0/0) 1624125
    5 unassigned    wm       0               0         (0/0/0)          0
    6        usr    wm     186 - 1910      897.03MB    (1725/0/0) 1837125
    7 unassigned    wm       0               0         (0/0/0)          0
  
  partition> label
  Ready to label disk, continue? y
  
  partition> quit
  
  
  FORMAT MENU:
          disk       - select a disk
          type       - select (define) a disk type
          partition  - select (define) a partition table
          current    - describe the current disk
          format     - format and analyze the disk
          repair     - repair a defective sector
          label      - write label to the disk
          analyze    - surface analysis
          defect     - defect list management
          backup     - search for backup labels
          verify     - read and display labels
          save       - save new disk/partition definitions
          inquiry    - show vendor, product and revision
          volname    - set 8-character volume name
          !<cmd>     - execute <cmd>, then return
          quit
  format> quit
  # 
  
  ...end of admin-old/partition.txt
  

• What's happening:
  • Select disk: there are two in the system.
  • Go to partition submenu.
  • Give starting points and sizes of partitions in cylinders.
  • label disk to make partitions permanent.

Partitioning caveats

• Be prepared to lose everything on the disk.
• Partitions can overlap only if one is unused.
• Overlapping partitions in use creates great havoc!
• Creating unused holes does nothing except waste space.

Creating a filesystem

• Use the newfs command:

  contents of admin-old/newfs.txt...
  # newfs /dev/rdsk/c0t2d0s3
  newfs: construct a new file system /dev/rdsk/c0t2d0s3: (y/n)? y
  /dev/rdsk/c0t2d0s3:     213000 sectors in 200 cylinders of 15 tracks, 71 sectors
          104.0MB in 13 cyl groups (16 c/g, 8.32MB/g, 4032 i/g)
  super-block backups (for fsck -F ufs -o b=#) at:
   32, 17152, 34272, 51392, 68512, 85632, 102752, 119872, 136992, 154112, 171232,
   188352, 205472,
  ...end of admin-old/newfs.txt
  

• What's happening:
  • Structure disk.
  • Add super-blocks of information about the whole disk.

Filesystem creation caveats

• Erases everything.
• Super-blocks allow recovery from partial disk crash.
• Lots of extra options we'll understand later.

Mounting a filesystem

• Use the mount command:

  contents of admin-old/mount.txt...
  # df -k
  Filesystem            kbytes    used   avail capacity  Mounted on
  /proc                      0       0       0     0%    /proc
  /dev/dsk/c0t3d0s0      83319   27346   47642    37%    /
  /dev/dsk/c0t3d0s6     746982  475574  211650    70%    /usr
  fd                         0       0       0     0%    /dev/fd
  /dev/dsk/c0t3d0s1      41407   19979   17288    54%    /var
  swap                   99900     284   99616     1%    /tmp
  # mount /dev/dsk/c0t2d0s3 /mnt
  # df -k
  Filesystem            kbytes    used   avail capacity  Mounted on
  /proc                      0       0       0     0%    /proc
  /dev/dsk/c0t3d0s0      83319   27346   47642    37%    /
  /dev/dsk/c0t3d0s6     746982  475574  211650    70%    /usr
  fd                         0       0       0     0%    /dev/fd
  /dev/dsk/c0t3d0s1      41407   19979   17288    54%    /var
  swap                   99896     284   99612     1%    /tmp
  /dev/dsk/c0t2d0s3      99723       9   89742     1%    /mnt
  # 
  ...end of admin-old/mount.txt
  

• What's happening:
  • /dev/dsk/c0t2d0s3: the partition to mount.
  • /mnt: the mount point at which to make it visible within the filesystem.
  • df -k: lists mounted filesystems.
  • Now let's put something in the filesystem and unmount it:

    contents of admin-old/putin.txt...
    # cd /mnt
    # ls
    lost+found
    # touch foo    
    # ls
    foo         lost+found
    # cd ..
    # umount /mnt
    ls umount: /mnt busy
    (I cd'd OUT of this in another shell)
    # umount /mnt
    # ls /mnt
    # 
    ...end of admin-old/putin.txt
    

Mounting caveats

• cannot mount a device twice.
• device must contain a filesystem.
• cannot unmount a device into which a process has cd'd. Must cd out and try again.

Maintaining disks

• Arranging for mounting at boot time.
• Making disks available to other computers.
• Mounting disks from remote computers.
• Checking for disk problems.
• Dump and restore.

Mounting tables

• /etc/vfstab: list of disk partitions to use:

   
  #device           device             mount  FS    fsck     mount    mount
  #to mount         to fsck            point  type  pass    at boot  options
  /dev/dsk/c0t3d0s3 -                  -      swap   -        no        -
  /dev/dsk/c0t3d0s0 /dev/rdsk/c0t3d0s0 /      ufs    1        no        -
  /dev/dsk/c0t3d0s6 /dev/rdsk/c0t3d0s6 /usr   ufs    1        no        -
  /dev/dsk/c0t3d0s1 /dev/rdsk/c0t3d0s1 /var   ufs    1        no        -
  ^                 ^                  ^      ^     ^^^ when to check it. 
  ^                 ^                  ^      ^^^ type of filesystem. 
  ^                 ^                  ^^^^ where to make this appear. 
  ^                 ^^^^^^^^^^^^^^^^^^ corresponding 'raw' (character) device
  ^^^^^^^^^^^^^^^^^ block device
  

• device to mount: the block device (/dev/dsk)
• device to check: the raw (character) device (/dev/rdsk)
• mount point: where to make it appear in the filesystem.

NFS

• Network File System
• Allows mounting of directories from remote hosts.
• Sun design.
• Monstrous hack.
• Frightfully insecure.

Exporting filesystems

• Making a filesystem available on the network is called exporting the filesystem.
• Sharing information is contained in /etc/dfs/dfstab:

  contents of admin-old/dfstab.txt...
  
  #	Place share(1M) commands here for automatic execution
  #	on entering init state 3.
  #
  #	Issue the command '/etc/init.d/nfs.server start' to run the NFS
  #	daemon processes and the share commands, after adding the very
  #	first entry to this file.
  #
  #	share [-F fstype] [ -o options] [-d "<text>"] <pathname> [resource]
  #	.e.g,
  #	share  -F nfs  -o rw=engineering  -d "home dirs"  /export/home2
  share -F nfs -o rw=largo /export/1
  
  ...end of admin-old/dfstab.txt
  

Importing filesystems

• Mount the remote filesystem:

  # mount blackhole:/export/1 /mnt
  # ls /mnt
  foo         lost+found
  # df -k /mnt
  Filesystem            kbytes    used   avail capacity  Mounted on
  blackhole:/export/1    99720       8   89736     1%    /mnt
  # 
  

• Remote directory must be shared.
• blackhole: NFS server.
• largo: NFS client.
• /export/1: remote directory.
• /mnt: local mount point.

Maintaining disks

• sync: make disk agree with memory.
• fsck: check a disk for errors in structure.

   
  # fsck /dev/rdsk/c0t2d0s3
  ** /dev/rdsk/c0t2d0s3
  ** Last Mounted on /export/1
  ** Phase 1 - Check Blocks and Sizes
  ** Phase 2 - Check Pathnames
  ** Phase 3 - Check Connectivity
  ** Phase 4 - Check Reference Counts
  ** Phase 5 - Check Cyl groups
  3 files, 9 used, 99714 free (18 frags, 12462 blocks,  0.0% fragmentation)
  

• Only local disks.
• Cannot repair all problems.
• Asks before deleting anything.

Looking deeper

• So far, you've seen what a technical school trainee sees.
• We really have no idea what's going on inside the filesystems.
• To really understand filesystems, have to know something about the nature of filesystem and process space in UNIX.

Suprising filesystem facts

• At any time, a particular file's contents can be distributed over memory and disk.
• When a process writes into a file, the new content is buffered in memory until it's convenient to update the disk.
• If another process opens the file, content is read from memory if already in memory, instead of from the real disk.
• Updating of the actual disk is done by a separate daemon update.
• sync means to make the disk image agree with memory. It tells update to 'flush its buffers' to disk.

Why is fsck needed?

• Any file (or directory) can be stored partially in memory at any time.
• If the system crashes, parts of files can be lost.
• fsck cleans up after such partial losses, checking that every block on the disk is in an appropriate state.

Naive Model of disk structure

• To develop a basic understanding of a UNIX filesystem, please tolerate this oversimplified model of filesystem content.
• A disk contains two large arrays: inodes and blocks.
  • An inode is a pointer to file or directory content.
  • A block is a physical block of storage, 512 bytes.
  • These are kept separate on the disk and have fixed sizes.
• Each element of each array can be free or used.
  • it's free if no one is using it.
  • it's used if it's part of a file.
• super-block contains pointers to five things:
  • a list of used inodes
  • a list of free inodes
  • a list of used blocks
  • a list of free blocks
  • a root directory from which to start adding content.
• newfs sets up a super-block and several backups, so that
  • all inodes are in the free block list.
  • all blocks are in the free block list.
• An inode is simply a pointer to the head of a list of blocks comprising a file or directory. It also contains ownership and mode information.
• A directory is a particular kind of file that contains names and other inode numbers represented by the names.
• To create a file,
  • Allocate an inode for it.
  • Allocate free blocks to the inode.
  • Copy content into the sequence of blocks.
  • Map the inode to a name in some directory.
• same thing for directories, they're just files that contain name-to-inode mappings.
• To read a file:
  • use your current working directory and its name to determine a directory entry where the inode of the file resides.
  • Read the sequence of blocks listed within the inode.
• To write a file:
  • Find its inode.
  • start writing into the sequence of blocks listed for the inode.
  • If you run out of space, get new blocks from the free block list.
  • If you stop writing early, return extra blocks to the free list.

Special considerations

• A symbolic link is just a file that contains the path name of another file or directory.
  • To make:

     
    ln -s target_path linkname
    

  • Example:

     
    % ln -s /g/150/motd motd
    % ls -l motd
    lrwxrwxrwx   1 couch    faculty       11 Feb  3 14:16 motd -> /g/150/motd
    % ln -s ../../var/adm admin
    % ls -l admin
    lrwxrwxrwx   1 couch    faculty       13 Feb  3 14:16 admin -> ../../var/adm
    

  • Can span filesystems (e.g., jump from one to another).
  • Slow to read.
• A hard link is a reference from a directory to a file that also may live in another directory.
  • To make:

     
    ln target_path linkname
    

  • Example:

     
    % ls -l junk.c
    -rw-r--r--   1 couch    faculty      228 Jan 31 16:00 junk.c
    % ln junk.c junk.cc
    % ls -l junk.c junk.cc
    -rw-r--r--   2 couch    faculty      228 Jan 31 16:00 junk.c
    -rw-r--r--   2 couch    faculty      228 Jan 31 16:00 junk.cc
    

  • Link count was 1, now 2.
  • Indistinguishable from other file.
  • Cannot span filesystems.
  • Implementation: two pointers from different directories to same inode.
• Hard links to directories are mighty dangerous (and only root can access them).