The directories and files noted here are a small subset of those specified by the FHS document. Refer to the latest FHS document for the most complete information.

The complete standard is available online at http://www.pathname.com/fhs/.

/dev/hda - the master device on primary IDE channel./dev/hdb - the slave device on primary IDE channel./dev/tty0 - first virtual console./dev/tty1 - second virtual console./dev/sda - first device on primary SCSI or SATA channel./dev/lp0 - first parallel port.

 /etc |- X11/ |- skel/

arp, clock, halt, init, fsck.*, grub, ifconfig, mingetty, mkfs.*, mkswap, reboot, route, shutdown, swapoff, swapon

 /usr |- bin/ |- etc/ |- games/ |- include/ |- kerberos/ |- lib/ |- libexec/ |- local/ |- sbin/ |- share/ |- src/ |- tmp -> ../var/tmp/

 /usr/local |- bin/ |- etc/ |- games/ |- include/ |- lib/ |- libexec/ |- sbin/ |- share/ |- src/

 /var |- account/ |- arpwatch/ |- cache/ |- crash/ |- db/ |- empty/ |- ftp/ |- gdm/ |- kerberos/ |- lib/ |- local/ |- lock/ |- log/ |- mail -> spool/mail/ |- mailman/ |- named/ |- nis/ |- opt/ |- preserve/ |- run/ +- spool/ |- at/ |- clientmqueue/ |- cron/ |- cups/ |- exim/ |- lpd/ |- mail/ |- mailman/ |- mqueue/ |- news/ |- postfix/ |- repackage/ |- rwho/ |- samba/ |- squid/ |- squirrelmail/ |- up2date/ |- uucp |- uucppublic/ |- vbox/ |- tmp/ |- tux/ |- www/ |- yp/

By using the cat, more, or less commands on files within the /proc/ directory, users can immediately access enormous amounts of information about the system. For example, to display the type of CPU a computer has, type cat /proc/cpuinfo to receive output similar to the following:

processor : 0 vendor_id : AuthenticAMD cpu family : 5 model : 9 model name : AMD-K6(tm) 3D+ Processor stepping : 1 cpu MHz : 400.919 cache size : 256 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr mce cx8 pge mmx syscall 3dnow k6_mtrr bogomips : 799.53

When viewing different virtual files in the /proc/ file system, some of the information is easily understandable while some is not human-readable. This is in part why utilities exist to pull data from virtual files and display it in a useful way. Examples of these utilities include lspci, apm, free, and top.

As a general rule, most virtual files within the /proc/ directory are read-only. However, some can be used to adjust settings in the kernel. This is especially true for files in the /proc/sys/ subdirectory.

To change the value of a virtual file, use the echo command and a greater than symbol (>) to redirect the new value to the file. For example, to change the hostname on the fly, type:

echo www.example.com > /proc/sys/kernel/hostname 

Other files act as binary or Boolean switches. Typing cat /proc/sys/net/ipv4/ip_forward returns either a 0 or a 1. A 0 indicates that the kernel is not forwarding network packets. Using the echo command to change the value of the ip_forward file to 1 immediately turns packet forwarding on.

For a listing of some of the kernel configuration files available in the /proc/sys/ subdirectory, refer to Section 3.3.9, “/proc/sys/”.

This file provides information about the state of the Advanced Power Management (APM) system and is used by the apm command. If a system with no battery is connected to an AC power source, this virtual file would look similar to the following:

1.16 1.2 0x07 0x01 0xff 0x80 -1% -1 ?

Running the apm -v command on such a system results in output similar to the following:

APM BIOS 1.2 (kernel driver 1.16ac) AC on-line, no system battery

For systems which do not use a battery as a power source, apm is able do little more than put the machine in standby mode. The apm command is much more useful on laptops. For example, the following output is from the command cat /proc/apm on a laptop while plugged into a power outlet:

1.16 1.2 0x03 0x01 0x03 0x09 100% -1 ?

When the same laptop is unplugged from its power source for a few minutes, the content of the apm file changes to something like the following:

1.16 1.2 0x03 0x00 0x00 0x01 99% 1792 min

The apm -v command now yields more useful data, such as the following:

APM BIOS 1.2 (kernel driver 1.16) AC off-line, battery status high: 99% (1 day, 5:52)

This file is used primarily for diagnosing memory fragmentation issues. Using the buddy algorithm, each column represents the number of pages of a certain order (a certain size) that are available at any given time. For example, for zone DMA (direct memory access), there are 90 of 2^(0*PAGE_SIZE) chunks of memory. Similarly, there are 6 of 2^(1*PAGE_SIZE) chunks, and 2 of 2^(2*PAGE_SIZE) chunks of memory available.

The DMA row references the first 16 MB on a system, the HighMem row references all memory greater than 4 GB on a system, and the Normal row references all memory in between.

The following is an example of the output typical of /proc/buddyinfo:

Node 0, zone      DMA     90      6      2      1      1      ... 
Node 0, zone   Normal   1650    310      5      0      0      ... 
Node 0, zone  HighMem      2      0      0      1      1      ...

This file shows the parameters passed to the kernel at the time it is started. A sample /proc/cmdline file looks like the following:

ro root=/dev/VolGroup00/LogVol00 rhgb quiet 3

This tells us that the kernel is mounted read-only (signified by (ro)), located on the first logical volume (LogVol00) of the first volume group (/dev/VolGroup00). LogVol00 is the equivalent of a disk partition in a non-LVM system (Logical Volume Management), just as /dev/VolGroup00 is similar in concept to /dev/hda1, but much more extensible.

For more information on LVM used in Red Hat Enterprise Linux, refer to http://www.tldp.org/HOWTO/LVM-HOWTO/index.html.

Next, rhgb signals that the rhgb package has been installed, and graphical booting is supported, assuming /etc/inittab shows a default runlevel set to id:5:initdefault:.

Finally, quiet indicates all verbose kernel messages are suppressed at boot time.

This virtual file identifies the type of processor used by your system. The following is an example of the output typical of /proc/cpuinfo:

processor	: 0 
vendor_id	: GenuineIntel 
cpu family	: 15 
model		: 2 
model name	: Intel(R) Xeon(TM) CPU 2.40GHz 
stepping	: 7 cpu 
MHz		: 2392.371 
cache size	: 512 KB 
physical id	: 0 
siblings	: 2 
runqueue	: 0 
fdiv_bug	: no 
hlt_bug		: no 
f00f_bug	: no 
coma_bug	: no 
fpu		: yes 
fpu_exception	: yes 
cpuid level	: 2 
wp		: yes 
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca  cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm 
bogomips	: 4771.02

This file lists all installed cryptographic ciphers used by the Linux kernel, including additional details for each. A sample /proc/crypto file looks like the following:

name         : sha1 
module       : kernel 
type         : digest 
blocksize    : 64 
digestsize   : 20   
name         : md5 
module       : md5 
type         : digest 
blocksize    : 64 
digestsize   : 16

This file displays the various character and block devices currently configured (not including devices whose modules are not loaded). Below is a sample output from this file:

Character devices:  
  1 mem   
  4 /dev/vc/0   
  4 tty   
  4 ttyS   
  5 /dev/tty   
  5 /dev/console   
  5 /dev/ptmx   
  7 vcs  
  10 misc  
  13 input  
  29 fb  
  36 netlink 
  128 ptm 
  136 pts 
  180 usb   
  
Block devices:   
  1 ramdisk   
  3 ide0   
  9 md  
  22 ide1 
  253 device-mapper 
  254 mdp

The output from /proc/devices includes the major number and name of the device, and is broken into two major sections: Character devices and Block devices.

Character devices are similar to block devices, except for two basic differences:

For more information about devices refer to the following installed documentation:

/usr/share/doc/kernel-doc-<version>/Documentation/devices.txt

This file contains a list of the registered ISA DMA channels in use. A sample /proc/dma files looks like the following:

4: cascade

This file lists the execution domains currently supported by the Linux kernel, along with the range of personalities they support.

0-0   Linux           [kernel]

Think of execution domains as the "personality" for an operating system. Because other binary formats, such as Solaris, UnixWare, and FreeBSD, can be used with Linux, programmers can change the way the operating system treats system calls from these binaries by changing the personality of the task. Except for the PER_LINUX execution domain, different personalities can be implemented as dynamically loadable modules.

This file contains a list of frame buffer devices, with the frame buffer device number and the driver that controls it. Typical output of /proc/fb for systems which contain frame buffer devices looks similar to the following:

0 VESA VGA

This file displays a list of the file system types currently supported by the kernel. Sample output from a generic /proc/filesystems file looks similar to the following:

nodev   sysfs 
nodev   rootfs 
nodev   bdev 
nodev   proc 
nodev   sockfs 
nodev   binfmt_misc 
nodev   usbfs 
nodev   usbdevfs 
nodev   futexfs 
nodev   tmpfs 
nodev   pipefs 
nodev   eventpollfs 
nodev   devpts         
	ext2 
nodev   ramfs 
nodev   hugetlbfs         
	iso9660 
nodev   mqueue         
	ext3 
nodev   rpc_pipefs 
nodev   autofs

The first column signifies whether the file system is mounted on a block device. Those beginning with nodev are not mounted on a device. The second column lists the names of the file systems supported.

The mount command cycles through the file systems listed here when one is not specified as an argument.

This file records the number of interrupts per IRQ on the x86 architecture. A standard /proc/interrupts looks similar to the following:

  CPU0          
  0:   80448940          XT-PIC  timer   
  1:     174412          XT-PIC  keyboard   
  2:          0          XT-PIC  cascade   
  8:          1          XT-PIC  rtc  
 10:     410964          XT-PIC  eth0  
 12:      60330          XT-PIC  PS/2 Mouse  
 14:    1314121          XT-PIC  ide0  
 15:    5195422          XT-PIC  ide1 
NMI:          0  
ERR:          0

For a multi-processor machine, this file may look slightly different:

	   CPU0       CPU1          
  0: 1366814704          0          XT-PIC  timer   
  1:        128        340    IO-APIC-edge  keyboard   
  2:          0          0          XT-PIC  cascade   
  8:          0          1    IO-APIC-edge  rtc  
 12:       5323       5793    IO-APIC-edge  PS/2 Mouse  
 13:          1          0          XT-PIC  fpu  
 16:   11184294   15940594   IO-APIC-level  Intel EtherExpress Pro 10/100 Ethernet  
 20:    8450043   11120093   IO-APIC-level  megaraid  
 30:      10432      10722   IO-APIC-level  aic7xxx  
 31:         23         22   IO-APIC-level  aic7xxx 
NMI:          0 
ERR:          0

The first column refers to the IRQ number. Each CPU in the system has its own column and its own number of interrupts per IRQ. The next column reports the type of interrupt, and the last column contains the name of the device that is located at that IRQ.

Each of the types of interrupts seen in this file, which are architecture-specific, mean something different. For x86 machines, the following values are common:

This file shows you the current map of the system's memory for each physical device:

00000000-0009fbff : System RAM
0009fc00-0009ffff : reserved 
000a0000-000bffff : Video RAM area
000c0000-000c7fff : Video ROM 
000f0000-000fffff : System ROM
00100000-07ffffff : System RAM   
00100000-00291ba8 : Kernel code
00291ba9-002e09cb : Kernel data 
e0000000-e3ffffff : VIA Technologies, Inc. VT82C597 [Apollo VP3] e4000000-e7ffffff : PCI Bus #01   
e4000000-e4003fff : Matrox Graphics, Inc. MGA G200 AGP   
e5000000-e57fffff : Matrox Graphics, Inc. MGA G200 AGP 
e8000000-e8ffffff : PCI Bus #01   
e8000000-e8ffffff : Matrox Graphics, Inc. MGA G200 AGP 
ea000000-ea00007f : Digital Equipment Corporation DECchip 21140 [FasterNet]
ea000000-ea00007f : tulip ffff0000-ffffffff : reserved

The first column displays the memory registers used by each of the different types of memory. The second column lists the kind of memory located within those registers and displays which memory registers are used by the kernel within the system RAM or, if the network interface card has multiple Ethernet ports, the memory registers assigned for each port.

The output of /proc/ioports provides a list of currently registered port regions used for input or output communication with a device. This file can be quite long. The following is a partial listing:

0000-001f : dma1 
0020-003f : pic1 
0040-005f : timer 
0060-006f : keyboard 
0070-007f : rtc 
0080-008f : dma page reg 
00a0-00bf : pic2 
00c0-00df : dma2 
00f0-00ff : fpu 
0170-0177 : ide1 
01f0-01f7 : ide0 
02f8-02ff : serial(auto) 
0376-0376 : ide1 
03c0-03df : vga+ 
03f6-03f6 : ide0 
03f8-03ff : serial(auto) 
0cf8-0cff : PCI conf1 
d000-dfff : PCI Bus #01 
e000-e00f : VIA Technologies, Inc. Bus Master IDE   
e000-e007 : ide0   
e008-e00f : ide1 
e800-e87f : Digital Equipment Corporation DECchip 21140 [FasterNet]   
e800-e87f : tulip

The first column gives the I/O port address range reserved for the device listed in the second column.

This file represents the physical memory of the system and is stored in the core file format. Unlike most /proc/ files, kcore displays a size. This value is given in bytes and is equal to the size of the physical memory (RAM) used plus 4 KB.

The contents of this file are designed to be examined by a debugger, such as gdb, and is not human readable.

This file is used to hold messages generated by the kernel. These messages are then picked up by other programs, such as /sbin/klogd or /bin/dmesg.

This file provides a look at the load average in regard to both the CPU and IO over time, as well as additional data used by uptime and other commands. A sample /proc/loadavg file looks similar to the following:

0.20 0.18 0.12 1/80 11206

The first three columns measure CPU and IO utilization of the last one, five, and 15 minute periods. The fourth column shows the number of currently running processes and the total number of processes. The last column displays the last process ID used.

In addition, load average also refers to the number of processes ready to run (i.e. in the run queue, waiting for a CPU share.

This file displays the files currently locked by the kernel. The contents of this file contain internal kernel debugging data and can vary tremendously, depending on the use of the system. A sample /proc/locks file for a lightly loaded system looks similar to the following:

1: POSIX  ADVISORY  WRITE 3568 fd:00:2531452 0 EOF 
2: FLOCK  ADVISORY  WRITE 3517 fd:00:2531448 0 EOF 
3: POSIX  ADVISORY  WRITE 3452 fd:00:2531442 0 EOF 
4: POSIX  ADVISORY  WRITE 3443 fd:00:2531440 0 EOF 
5: POSIX  ADVISORY  WRITE 3326 fd:00:2531430 0 EOF 
6: POSIX  ADVISORY  WRITE 3175 fd:00:2531425 0 EOF 
7: POSIX  ADVISORY  WRITE 3056 fd:00:2548663 0 EOF

Each lock has its own line which starts with a unique number. The second column refers to the class of lock used, with FLOCK signifying the older-style UNIX file locks from a flock system call and POSIX representing the newer POSIX locks from the lockf system call.

The third column can have two values: ADVISORY or MANDATORY. ADVISORY means that the lock does not prevent other people from accessing the data; it only prevents other attempts to lock it. MANDATORY means that no other access to the data is permitted while the lock is held. The fourth column reveals whether the lock is allowing the holder READ or WRITE access to the file. The fifth column shows the ID of the process holding the lock. The sixth column shows the ID of the file being locked, in the format of MAJOR-DEVICE:MINOR-DEVICE:INODE-NUMBER. The seventh and eighth column shows the start and end of the file's locked region.

This file contains the current information for multiple-disk, RAID configurations. If the system does not contain such a configuration, then /proc/mdstat looks similar to the following:

Personalities :  read_ahead not set unused devices: <none>

This file remains in the same state as seen above unless a software RAID or md device is present. In that case, view /proc/mdstat to find the current status of mdX RAID devices.

The /proc/mdstat file below shows a system with its md0 configured as a RAID 1 device, while it is currently re-syncing the disks:

Personalities : [linear] [raid1] read_ahead 1024 sectors 
md0: active raid1 sda2[1] sdb2[0] 9940 blocks [2/2] [UU] resync=1% finish=12.3min algorithm 2 [3/3] [UUU] 
unused devices: <none>

This is one of the more commonly used files in the /proc/ directory, as it reports a large amount of valuable information about the systems RAM usage.

The following sample /proc/meminfo virtual file is from a system with 256 MB of RAM and 512 MB of swap space:

MemTotal:       255908 kB 
MemFree:         69936 kB 
Buffers:         15812 kB 
Cached:         115124 kB 
SwapCached:          0 kB 
Active:          92700 kB 
Inactive:        63792 kB 
HighTotal:           0 kB 
HighFree:            0 kB 
LowTotal:       255908 kB 
LowFree:         69936 kB 
SwapTotal:      524280 kB 
SwapFree:       524280 kB 
Dirty:               4 kB 
Writeback:           0 kB 
Mapped:          42236 kB 
Slab:            25912 kB 
Committed_AS:   118680 kB 
PageTables:       1236 kB 
VmallocTotal:  3874808 kB 
VmallocUsed:      1416 kB 
VmallocChunk:  3872908 kB 
HugePages_Total:     0 
HugePages_Free:      0 
Hugepagesize:     4096 kB

Much of the information here is used by the free, top, and ps commands. In fact, the output of the free command is similar in appearance to the contents and structure of /proc/meminfo. But by looking directly at /proc/meminfo, more details are revealed:

This file lists miscellaneous drivers registered on the miscellaneous major device, which is device number 10:

63 device-mapper 175 agpgart 135 rtc 134 apm_bios

The first column is the minor number of each device, while the second column shows the driver in use.

This file displays a list of all modules loaded into the kernel. Its contents vary based on the configuration and use of your system, but it should be organized in a similar manner to this sample /proc/modules file output:

nfs      170109  0 -          Live 0x129b0000 
lockd    51593   1 nfs,       Live 0x128b0000 
nls_utf8 1729    0 -          Live 0x12830000 
vfat     12097   0 -          Live 0x12823000 
fat      38881   1 vfat,      Live 0x1287b000 
autofs4  20293   2 -          Live 0x1284f000 
sunrpc   140453  3 nfs,lockd, Live 0x12954000 
3c59x    33257   0 -          Live 0x12871000 
uhci_hcd 28377   0 -          Live 0x12869000 
md5      3777    1 -          Live 0x1282c000 
ipv6     211845 16 -          Live 0x128de000 
ext3     92585   2 -          Live 0x12886000 
jbd      65625   1 ext3,      Live 0x12857000 
dm_mod   46677   3 -          Live 0x12833000

The first column contains the name of the module.

The second column refers to the memory size of the module, in bytes.

The third column lists how many instances of the module are currently loaded. A value of zero represents an unloaded module.

The fourth column states if the module depends upon another module to be present in order to function, and lists those other modules.

The fifth column lists what load state the module is in: Live, Loading, or Unloading are the only possible values.

The sixth column lists the current kernel memory offset for the loaded module. This information can be useful for debugging purposes, or for profiling tools such as oprofile.

This file provides a list of all mounts in use by the system:

rootfs / rootfs rw 0 0 
/proc /proc proc rw,nodiratime 0 0 none 
/dev ramfs rw 0 0 
/dev/mapper/VolGroup00-LogVol00 / ext3 rw 0 0 
none /dev ramfs rw 0 0 
/proc /proc proc rw,nodiratime 0 0 
/sys /sys sysfs rw 0 0 
none /dev/pts devpts rw 0 0 
usbdevfs /proc/bus/usb usbdevfs rw 0 0 
/dev/hda1 /boot ext3 rw 0 0 
none /dev/shm tmpfs rw 0 0 
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0 
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0

The output found here is similar to the contents of /etc/mtab, except that /proc/mount is more up-to-date.

The first column specifies the device that is mounted, the second column reveals the mount point, and the third column tells the file system type, and the fourth column tells you if it is mounted read-only (ro) or read-write (rw). The fifth and sixth columns are dummy values designed to match the format used in /etc/mtab.

This file refers to the current Memory Type Range Registers (MTRRs) in use with the system. If the system architecture supports MTRRs, then the /proc/mtrr file may look similar to the following:

reg00: base=0x00000000 (   0MB), size= 256MB: write-back, count=1 
reg01: base=0xe8000000 (3712MB), size=  32MB: write-combining, count=1

MTRRs are used with the Intel P6 family of processors (Pentium II and higher) and control processor access to memory ranges. When using a video card on a PCI or AGP bus, a properly configured /proc/mtrr file can increase performance more than 150%.

Most of the time, this value is properly configured by default. More information on manually configuring this file can be found locally at the following location:

/usr/share/doc/kernel-doc-<version>/Documentation/mtrr.txt

This file contains partition block allocation information. A sampling of this file from a basic system looks similar to the following:

major minor  #blocks  name      
  3     0   19531250 hda    
  3     1     104391 hda1    
  3     2   19422585 hda2  
253     0   22708224 dm-0  
253     1     524288 dm-1

Most of the information here is of little importance to the user, except for the following columns:

This file contains a full listing of every PCI device on the system. Depending on the number of PCI devices, /proc/pci can be rather long. A sampling of this file from a basic system looks similar to the following:

Bus  0, device 0, function 0: Host bridge: Intel Corporation 440BX/ZX - 82443BX/ZX Host bridge (rev 3). Master Capable. Latency=64. Prefetchable 32 bit memory at 0xe4000000 [0xe7ffffff].   
Bus  0, device 1, function 0: PCI bridge: Intel Corporation 440BX/ZX - 82443BX/ZX AGP bridge (rev 3).   Master Capable. Latency=64. Min Gnt=128.   
Bus  0, device 4, function 0: ISA bridge: Intel Corporation 82371AB PIIX4 ISA (rev 2).   
Bus  0, device 4, function 1: IDE interface: Intel Corporation 82371AB PIIX4 IDE (rev 1). Master Capable. Latency=32. I/O at 0xd800 [0xd80f].   
Bus  0, device 4, function 2: USB Controller: Intel Corporation 82371AB PIIX4 USB (rev 1). IRQ 5. Master Capable. Latency=32. I/O at 0xd400 [0xd41f].   
Bus  0, device 4, function 3: Bridge: Intel Corporation 82371AB PIIX4 ACPI (rev 2). IRQ 9.  
Bus  0, device 9, function 0: Ethernet controller: Lite-On Communications Inc LNE100TX (rev 33). IRQ 5. Master Capable. Latency=32. I/O at 0xd000 [0xd0ff].   
Bus  0, device 12, function  0: VGA compatible controller: S3 Inc. ViRGE/DX or /GX (rev 1). IRQ 11. Master Capable. Latency=32. Min Gnt=4.Max Lat=255.

This output shows a list of all PCI devices, sorted in the order of bus, device, and function. Beyond providing the name and version of the device, this list also gives detailed IRQ information so an administrator can quickly look for conflicts.

/sbin/lspci -vb

This file gives full information about memory usage on the slab level. Linux kernels greater than version 2.2 use slab pools to manage memory above the page level. Commonly used objects have their own slab pools.

Instead of parsing the highly verbose /proc/slabinfo file manually, the /usr/bin/slabtop program displays kernel slab cache information in real time. This program allows for custom configurations, including column sorting and screen refreshing.

A sample screen shot of /usr/bin/slabtop usually looks like the following example:

Active / Total Objects (% used)    : 133629 / 147300 (90.7%)  
Active / Total Slabs (% used)      : 11492 / 11493 (100.0%)  
Active / Total Caches (% used)     : 77 / 121 (63.6%)  
Active / Total Size (% used)       : 41739.83K / 44081.89K (94.7%)  
Minimum / Average / Maximum Object : 0.01K / 0.30K / 128.00K
OBJS   ACTIVE USE      OBJ   SIZE     SLABS OBJ/SLAB CACHE SIZE NAME  
44814  43159  96%    0.62K   7469      6     29876K ext3_inode_cache
36900  34614  93%    0.05K    492     75      1968K buffer_head  
35213  33124  94%    0.16K   1531     23      6124K dentry_cache   
7364   6463  87%    0.27K    526      14      2104K radix_tree_node   
2585   1781  68%    0.08K     55      47       220K vm_area_struct   
2263   2116  93%    0.12K     73      31       292K size-128   
1904   1125  59%    0.03K     16      119        64K size-32   
1666    768  46%    0.03K     14      119        56K anon_vma   
1512   1482  98%    0.44K    168       9       672K inode_cache   
1464   1040  71%    0.06K     24      61        96K size-64   
1320    820  62%    0.19K     66      20       264K filp    
678    587  86%    0.02K      3      226        12K dm_io   
678    587  86%    0.02K      3      226        12K dm_tio    
576    574  99%    0.47K     72        8       288K proc_inode_cache    
528    514  97%    0.50K     66        8       264K size-512    
492    372  75%    0.09K     12       41        48K bio    
465    314  67%    0.25K     31       15       124K size-256    
452    331  73%    0.02K      2      226         8K biovec-1    
420    420 100%    0.19K     21       20        84K skbuff_head_cache    
305    256  83%    0.06K      5       61        20K biovec-4    
290      4   1%    0.01K      1      290         4K revoke_table    
264    264 100%    4.00K    264        1      1056K size-4096    
260    256  98%    0.19K     13       20        52K biovec-16    
260    256  98%    0.75K     52        5       208K biovec-64

Some of the more commonly used statistics in /proc/slabinfo that are included into /usr/bin/slabtop include:

For more information on the /usr/bin/slabtop program, refer to the slabtop man page.

This file keeps track of a variety of different statistics about the system since it was last restarted. The contents of /proc/stat, which can be quite long, usually begins like the following example:

cpu  259246 7001 60190 34250993 137517 772 0 
cpu0 259246 7001 60190 34250993 137517 772 0 
intr 354133732 347209999 2272 0 4 4 0 0 3 1 1249247 0 0 80143 0 422626 5169433 
ctxt 12547729 
btime 1093631447 
processes 130523 
procs_running 1 
procs_blocked 0 
preempt 5651840  
cpu  209841 1554 21720 118519346 72939 154 27168 
cpu0 42536 798 4841 14790880 14778 124 3117 
cpu1 24184 569 3875 14794524 30209 29 3130 
cpu2 28616 11 2182 14818198 4020 1 3493 
cpu3 35350 6 2942 14811519 3045 0 3659 
cpu4 18209 135 2263 14820076 12465 0 3373 
cpu5 20795 35 1866 14825701 4508 0 3615 
cpu6 21607 0 2201 14827053 2325 0 3334 
cpu7 18544 0 1550 14831395 1589 0 3447 
intr 15239682 14857833 6 0 6 6 0 5 0 1 0 0 0 29 0 2 0 0 0 0 0 0 0 94982 0 286812 
ctxt 4209609 
btime 1078711415 
processes 21905 
procs_running 1 
procs_blocked 0

Some of the more commonly used statistics include:

This file measures swap space and its utilization. For a system with only one swap partition, the output of /proc/swaps may look similar to the following:

Filename                          Type        Size     Used    Priority 
/dev/mapper/VolGroup00-LogVol01   partition   524280   0       -1

While some of this information can be found in other files in the /proc/ directory, /proc/swap provides a snapshot of every swap file name, the type of swap space, the total size, and the amount of space in use (in kilobytes). The priority column is useful when multiple swap files are in use. The lower the priority, the more likely the swap file is to be used.

Using the echo command to write to this file, a remote root user can execute most System Request Key commands remotely as if at the local terminal. To echo values to this file, the /proc/sys/kernel/sysrq must be set to a value other than 0. For more information about the System Request Key, refer to Section 3.3.9.3, “/proc/sys/kernel/”.

Although it is possible to write to this file, it cannot be read, even by the root user.

This file contains information detailing how long the system has been on since its last restart. The output of /proc/uptime is quite minimal:

350735.47 234388.90

The first number is the total number of seconds the system has been up. The second number is how much of that time the machine has spent idle, in seconds.

This file specifies the version of the Linux kernel and gcc in use, as well as the version of Red Hat Enterprise Linux installed on the system:

Linux version 2.6.8-1.523 (user@foo.redhat.com) (gcc version 3.4.1 20040714 \  (Red Hat Enterprise Linux 3.4.1-7)) #1 Mon Aug 16 13:27:03 EDT 2004

This information is used for a variety of purposes, including the version data presented when a user logs in.

Every /proc/ directory contains a number of directories with numerical names. A listing of them may be similar to the following:

dr-xr-xr-x    3 root     root            0 Feb 13 01:28 1 
dr-xr-xr-x    3 root     root            0 Feb 13 01:28 1010
dr-xr-xr-x    3 xfs      xfs             0 Feb 13 01:28 1087
dr-xr-xr-x    3 daemon   daemon          0 Feb 13 01:28 1123
dr-xr-xr-x    3 root     root            0 Feb 13 01:28 11307
dr-xr-xr-x    3 apache   apache          0 Feb 13 01:28 13660
dr-xr-xr-x    3 rpc      rpc             0 Feb 13 01:28 637
dr-xr-xr-x    3 rpcuser  rpcuser         0 Feb 13 01:28 666

These directories are called process directories, as they are named after a program's process ID and contain information specific to that process. The owner and group of each process directory is set to the user running the process. When the process is terminated, its /proc/ process directory vanishes.

Each process directory contains the following files:

This directory contains information specific to the various buses available on the system. For example, on a standard system containing PCI and USB buses, current data on each of these buses is available within a subdirectory within /proc/bus/ by the same name, such as /proc/bus/pci/.

The subdirectories and files available within /proc/bus/ vary depending on the devices connected to the system. However, each bus type has at least one directory. Within these bus directories are normally at least one subdirectory with a numerical name, such as 001, which contain binary files.

For example, the /proc/bus/usb/ subdirectory contains files that track the various devices on any USB buses, as well as the drivers required for them. The following is a sample listing of a /proc/bus/usb/ directory:

total 0 dr-xr-xr-x    1 root     root            0 May  3 16:25 001 
-r--r--r--    1 root     root            0 May  3 16:25 devices 
-r--r--r--    1 root     root            0 May  3 16:25 drivers

The /proc/bus/usb/001/ directory contains all devices on the first USB bus and the devices file identifies the USB root hub on the motherboard.

The following is a example of a /proc/bus/usb/devices file:

T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=12  MxCh= 2 
B:  Alloc=  0/900 us ( 0%), #Int=  0, #Iso=  0 
D:  Ver= 1.00 Cls=09(hub  ) Sub=00 Prot=00 MxPS= 8 #Cfgs=  1 
P:  Vendor=0000 ProdID=0000 Rev= 0.00 
S:  Product=USB UHCI Root Hub 
S:  SerialNumber=d400 
C:* #Ifs= 1 Cfg#= 1 Atr=40 MxPwr=  0mA 
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub 
E:  Ad=81(I) Atr=03(Int.) MxPS=   8 Ivl=255ms

This directory contains information for specific drivers in use by the kernel.

A common file found here is rtc which provides output from the driver for the system's Real Time Clock (RTC), the device that keeps the time while the system is switched off. Sample output from /proc/driver/rtc looks like the following:

rtc_time        : 16:21:00 
rtc_date        : 2004-08-31 
rtc_epoch       : 1900 
alarm           : 21:16:27 
DST_enable      : no 
BCD             : yes 
24hr            : yes 
square_wave     : no 
alarm_IRQ       : no 
update_IRQ      : no 
periodic_IRQ    : no 
periodic_freq   : 1024 
batt_status     : okay

For more information about the RTC, refer to the following installed documentation:

/usr/share/doc/kernel-doc-<version>/Documentation/rtc.txt.

This directory shows which file systems are exported. If running an NFS server, typing cat /proc/fs/nfsd/exports displays the file systems being shared and the permissions granted for those file systems. For more on file system sharing with NFS, refer to Chapter 19, Network File System (NFS).

This directory contains information about IDE devices on the system. Each IDE channel is represented as a separate directory, such as /proc/ide/ide0 and /proc/ide/ide1. In addition, a drivers file is available, providing the version number of the various drivers used on the IDE channels:

ide-floppy version 0.99.
newide ide-cdrom version 4.61 
ide-disk version 1.18

Many chipsets also provide a file in this directory with additional data concerning the drives connected through the channels. For example, a generic Intel PIIX4 Ultra 33 chipset produces the /proc/ide/piix file which reveals whether DMA or UDMA is enabled for the devices on the IDE channels:

Intel PIIX4 Ultra 33 Chipset. 
------------- Primary Channel ---------------- Secondary Channel -------------                  
		enabled                          enabled 
		
------------- drive0 --------- drive1 -------- drive0 ---------- drive1 ------ 
DMA enabled:    yes              no              yes               no  
UDMA enabled:   yes              no              no                no  
UDMA enabled:   2                X               X                 X 
UDMA DMA PIO

Navigating into the directory for an IDE channel, such as ide0, provides additional information. The channel file provides the channel number, while the model identifies the bus type for the channel (such as pci).

This directory is used to set IRQ to CPU affinity, which allows the system to connect a particular IRQ to only one CPU. Alternatively, it can exclude a CPU from handling any IRQs.

Each IRQ has its own directory, allowing for the individual configuration of each IRQ. The /proc/irq/prof_cpu_mask file is a bitmask that contains the default values for the smp_affinity file in the IRQ directory. The values in smp_affinity specify which CPUs handle that particular IRQ.

For more information about the /proc/irq/ directory, refer to the following installed documentation:

/usr/share/doc/kernel-doc-<version>/Documentation/filesystems/proc.txt

This directory provides a comprehensive look at various networking parameters and statistics. Each directory and virtual file within this directory describes aspects of the system's network configuration. Below is a partial list of the /proc/net/ directory:

This directory is analogous to the /proc/ide/ directory, but it is for connected SCSI devices.

The primary file in this directory is /proc/scsi/scsi, which contains a list of every recognized SCSI device. From this listing, the type of device, as well as the model name, vendor, SCSI channel and ID data is available.

For example, if a system contains a SCSI CD-ROM, a tape drive, a hard drive, and a RAID controller, this file looks similar to the following:

Attached devices:  
Host: scsi1 
Channel: 00 
Id: 05 
Lun: 00   
Vendor: NEC      
Model: CD-ROM DRIVE:466 
Rev: 1.06   
Type:   CD-ROM                          
ANSI SCSI revision: 02 
Host: scsi1 
Channel: 00 
Id: 06 
Lun: 00   
Vendor: ARCHIVE  
Model: Python 04106-XXX 
Rev: 7350   
Type:   Sequential-Access                
ANSI SCSI revision: 02 
Host: scsi2 
Channel: 00 
Id: 06 
Lun: 00   
Vendor: DELL     
Model: 1x6 U2W SCSI BP  
Rev: 5.35   
Type:   Processor                        
ANSI SCSI revision: 02 
Host: scsi2 
Channel: 02 
Id: 00 
Lun: 00   
Vendor: MegaRAID 
Model: LD0 RAID5 34556R 
Rev: 1.01   
Type:   Direct-Access                    
ANSI SCSI revision: 02

Each SCSI driver used by the system has its own directory within /proc/scsi/, which contains files specific to each SCSI controller using that driver. From the previous example, aic7xxx/ and megaraid/ directories are present, since two drivers are in use. The files in each of the directories typically contain an I/O address range, IRQ information, and statistics for the SCSI controller using that driver. Each controller can report a different type and amount of information. The Adaptec AIC-7880 Ultra SCSI host adapter's file in this example system produces the following output:

Adaptec AIC7xxx driver version: 5.1.20/3.2.4 
Compile Options:   
TCQ Enabled By Default : Disabled   
AIC7XXX_PROC_STATS     : Enabled   
AIC7XXX_RESET_DELAY    : 5  
Adapter Configuration:            
SCSI Adapter: Adaptec AIC-7880 Ultra SCSI host adapter                            
Ultra Narrow Controller     PCI MMAPed 
I/O Base: 0xfcffe000  
Adapter SEEPROM Config: SEEPROM found and used.       
Adaptec SCSI BIOS: Enabled                     
IRQ: 30                    
SCBs: Active 0, Max Active 1, Allocated 15, HW 16, Page 255              
Interrupts: 33726       
BIOS Control Word: 0x18a6    
Adapter Control Word: 0x1c5f   
Extended Translation: Enabled 
Disconnect Enable Flags: 0x00ff      
Ultra Enable Flags: 0x0020  
Tag Queue Enable Flags: 0x0000 
Ordered Queue Tag Flags: 0x0000 
Default Tag Queue Depth: 8     
Tagged Queue By Device array for aic7xxx 
host instance 1:       {255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255}     
Actual queue depth per device for aic7xxx host instance 1:       {1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}
Statistics:   

(scsi1:0:5:0) Device using Narrow/Sync transfers at 20.0 MByte/sec, offset 15   
Transinfo settings: current(12/15/0/0), goal(12/15/0/0), user(12/15/0/0)   
Total transfers 0 (0 reads and 0 writes)              
		< 2K      2K+     4K+     8K+    16K+    32K+    64K+   128K+    
Reads:        0       0       0       0       0       0       0       0   
Writes:       0       0       0       0       0       0       0       0   

(scsi1:0:6:0) Device using Narrow/Sync transfers at 10.0 MByte/sec, offset 15   
Transinfo settings: current(25/15/0/0), goal(12/15/0/0), user(12/15/0/0)   
Total transfers 132 (0 reads and 132 writes)              
		< 2K      2K+     4K+     8K+    16K+    32K+    64K+   128K+    
Reads:        0       0       0       0       0       0       0       0   
Writes:       0       0       0       1     131       0       0       0

This output reveals the transfer speed to the SCSI devices connected to the controller based on channel ID, as well as detailed statistics concerning the amount and sizes of files read or written by that device. For example, this controller is communicating with the CD-ROM at 20 megabytes per second, while the tape drive is only communicating at 10 megabytes per second.

The /proc/sys/ directory is different from others in /proc/ because it not only provides information about the system but also allows the system administrator to immediately enable and disable kernel features.

A good way to determine if a particular file can be configured, or if it is only designed to provide information, is to list it with the -l option at the shell prompt. If the file is writable, it may be used to configure the kernel. For example, a partial listing of /proc/sys/fs looks like the following:

-r--r--r--    1 root     root            0 May 10 16:14 dentry-state
-rw-r--r--    1 root     root            0 May 10 16:14 dir-notify-enable 
-r--r--r--    1 root     root            0 May 10 16:14 dquot-nr 
-rw-r--r--    1 root     root            0 May 10 16:14 file-max 
-r--r--r--    1 root     root            0 May 10 16:14 file-nr

In this listing, the files dir-notify-enable and file-max can be written to and, therefore, can be used to configure the kernel. The other files only provide feedback on current settings.

Changing a value within a /proc/sys/ file is done by echoing the new value into the file. For example, to enable the System Request Key on a running kernel, type the command:

echo 1 > /proc/sys/kernel/sysrq

This changes the value for sysrq from 0 (off) to 1 (on).

A few /proc/sys/ configuration files contain more than one value. To correctly send new values to them, place a space character between each value passed with the echo command, such as is done in this example:

echo 4 2 45 > /proc/sys/kernel/acct

The /proc/sys/ directory contains several subdirectories controlling different aspects of a running kernel.

CD-ROM information, Id: cdrom.c 3.20 2003/12/17   
drive name:             hdc 
drive speed:            48 
drive # of slots:       1 
Can close tray:         1 
Can open tray:          1 
Can lock tray:          1 
Can change speed:       1 
Can select disk:        0 
Can read multisession:  1 
Can read MCN:           1 
Reports media changed:  1 
Can play audio:         1 
Can write CD-R:         0 
Can write CD-RW:        0 
Can read DVD:           0 
Can write DVD-R:        0 
Can write DVD-RAM:      0 
Can read MRW:           0 
Can write MRW:          0 
Can write RAM:          0

/usr/share/doc/kernel-doc-<version>/Documentation/networking/ ip-sysctl.txt

/usr/share/doc/kernel-doc-<version>/Documentation/filesystems/proc.txt

This directory contains information about System V IPC resources. The files in this directory relate to System V IPC calls for messages (msg), semaphores (sem), and shared memory (shm).

This directory contains information about the available and currently used tty devices on the system. Originally called teletype devices, any character-based data terminals are called tty devices.

In Linux, there are three different kinds of tty devices. Serial devices are used with serial connections, such as over a modem or using a serial cable. Virtual terminals create the common console connection, such as the virtual consoles available when pressing Alt-<F-key> at the system console. Pseudo terminals create a two-way communication that is used by some higher level applications, such as XFree86. The drivers file is a list of the current tty devices in use, as in the following example:

serial               /dev/cua        5  64-127 serial:callout 
serial               /dev/ttyS       4  64-127 serial 
pty_slave            /dev/pts      136   0-255 pty:slave 
pty_master           /dev/ptm      128   0-255 pty:master 
pty_slave            /dev/ttyp       3   0-255 pty:slave 
pty_master           /dev/pty        2   0-255 pty:master 
/dev/vc/0            /dev/vc/0       4       0 system:vtmaster 
/dev/ptmx            /dev/ptmx       5       2 system 
/dev/console         /dev/console    5       1 system:console 
/dev/tty             /dev/tty        5       0 system:/dev/tty 
unknown              /dev/vc/%d      4    1-63 console

The /proc/tty/driver/serial file lists the usage statistics and status of each of the serial tty lines.

In order for tty devices to be used as network devices, the Linux kernel enforces line discipline on the device. This allows the driver to place a specific type of header with every block of data transmitted over the device, making it possible for the remote end of the connection to a block of data as just one in a stream of data blocks. SLIP and PPP are common line disciplines, and each are commonly used to connect systems to one other over a serial link.

Registered line disciplines are stored in the ldiscs file, and more detailed information is available within the ldisc/ directory.

Some of the best documentation about the proc file system is installed on the system by default.

The hardware-based array manages the RAID subsystem independently from the host. It presents a single disk per RAID array to the host.

A Hardware RAID device connects to the SCSI controller and presents the RAID arrays as a single SCSI drive. An external RAID system moves all RAID handling "intelligence" into a controller located in the external disk subsystem. The whole subsystem is connected to the host via a normal SCSI controller and appears to the host as a single disk.

RAID controller cards function like a SCSI controller to the operating system, and handle all the actual drive communications. The user plugs the drives into the RAID controller (just like a normal SCSI controller) and then adds them to the RAID controllers configuration, and the operating system won't know the difference.

Software RAID implements the various RAID levels in the kernel disk (block device) code. It offers the cheapest possible solution, as expensive disk controller cards or hot-swap chassis ^[1] are not required. Software RAID also works with cheaper IDE disks as well as SCSI disks. With today's faster CPUs, Software RAID outperforms Hardware RAID.

The Linux kernel contains an MD driver that allows the RAID solution to be completely hardware independent. The performance of a software-based array depends on the server CPU performance and load.

To learn more about Software RAID, here are the key features:

In a typical situation, the disk drives are new or are formatted. Both drives are shown as raw devices with no partition configuration in Figure 4.1, “Two Blank Drives, Ready For Configuration”.

Two Blank Drives, Ready For Configuration

Figure 4.1. Two Blank Drives, Ready For Configuration

Repeat these steps to create as many partitions as needed for your RAID setup. Notice that all the partitions do not have to be RAID partitions. For example, you can configure only the /boot/ partition as a software RAID device, leaving the root partition (/), /home/, and swap as regular file systems. Figure 4.4, “RAID 1 Partitions Ready, Pre-Device and Mount Point Creation” shows successfully allocated space for the RAID 1 configuration (for /boot/), which is now ready for RAID device and mount point creation:

RAID 1 Partitions Ready, Pre-Device and Mount Point Creation

Figure 4.4. RAID 1 Partitions Ready, Pre-Device and Mount Point Creation

Once you create all of your partitions as Software RAID partitions, you must create the RAID device and mount point.

After completing the entire configuration, the figure as shown in Figure 4.8, “Final Sample RAID Configuration” resembles the default configuration, except for the use of RAID.

Final Sample RAID Configuration

Figure 4.8. Final Sample RAID Configuration

The figure as shown in Figure 4.9, “Final Sample RAID With LVM Configuration” is an example of a RAID and LVM configuration.

Final Sample RAID With LVM Configuration

Figure 4.9. Final Sample RAID With LVM Configuration

You can continue with your installation process. Refer to the Red Hat Enterprise Linux Installation Guide for further instructions.

To extend an LVM2 swap logical volume (assuming /dev/VolGroup00/LogVol01 is the volume you want to extend):

To add a swap volume group (assuming /dev/VolGroup00/LogVol02 is the swap volume you want to add):

To add a swap file:

To reduce an LVM2 swap logical volume (assuming /dev/VolGroup00/LogVol01 is the volume you want to extend):

The swap logical volume cannot be in use (no system locks or processes on the volume). The easiest way to achieve this it to boot your system in rescue mode. Refer to for instructions on booting into rescue mode. When prompted to mount the file system, select Skip.

To remove a swap volume group (assuming /dev/VolGroup00/LogVol02 is the swap volume you want to remove):

To remove a swap file:

After starting parted, use the command print to view the partition table. A table similar to the following appears:

Model: ATA ST3160812AS (scsi)
Disk /dev/sda: 160GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End    Size    Type      File system  Flags
 1      32.3kB  107MB  107MB   primary   ext3         boot
 2      107MB   105GB  105GB   primary   ext3
 3      105GB   107GB  2147MB  primary   linux-swap
 4      107GB   160GB  52.9GB  extended		      root
 5      107GB   133GB  26.2GB  logical   ext3
 6      133GB   133GB  107MB   logical   ext3
 7      133GB   160GB  26.6GB  logical                lvm

The first line contains the disk type, manufacturer, model number and interface, and the second line displays the disk label type. The remaining output below the fourth line shows the partition table.

In the partition table, the Minor number is the partition number. For example, the partition with minor number 1 corresponds to /dev/sda1. The Start and End values are in megabytes. Valid Type are metadata, free, primary, extended, or logical. The Filesystem is the file system type, which can be any of the following:

If a Filesystem of a device shows no value, this means that its file system type is unknown.

The Flags column lists the flags set for the partition. Available flags are boot, root, swap, hidden, raid, lvm, or lba.

Before creating a partition, boot into rescue mode (or unmount any partitions on the device and turn off any swap space on the device).

Start parted, where /dev/sda is the device on which to create the partition:

parted /dev/sda

View the current partition table to determine if there is enough free space:

print

If there is not enough free space, you can resize an existing partition. Refer to Section 6.1.4, “Resizing a Partition” for details.

mkpart primary ext3 1024 2048

cat /proc/partitions

/sbin/mkfs -t ext3 /dev/sda6

e2label /dev/sda6 /work

mkdir /work

LABEL=/work /work ext3 defaults 1 2

mount /work

Before removing a partition, boot into rescue mode (or unmount any partitions on the device and turn off any swap space on the device).

Start parted, where /dev/sda is the device on which to remove the partition:

parted /dev/sda

View the current partition table to determine the minor number of the partition to remove:

print