|
Softpanorama
(slightly skeptical)
Open Source Software Educational Society |
May the
source be with you,
but remember the KISS principle ;-)
|
UFS The Unix File System
Science is facts; just as houses are made of stones, so is science
made
of facts; but a pile of stones is not a house and a collection of facts
is not necessarily science.
-- Henri Poincaire
Filesystems is a very interesting area, one of the few areas in Unix where new
algorithms still can make a huge difference in performance.
Often the historical view on filesystems is a bit too Unix-centric and states
that the Berkeley Fast File System is the ancestor of most modern file systems.
This view ignores competitive and earlier implementations from IBM(HPFS), DEC (VAX
VMS), Microsoft (NTFS) and others.
Still Unix filesystems became a classic and concepts introduced in ti dominate
all modern filesystems It also introduced many interesting features and algorithms
into the area. For example a very interesting concept of extended attributes introduced
in the 4.4 BSD filesystem have recently been added to Ext2fs:
Immutable files can only be read: nobody can write or delete them. This
can be used to protect sensitive configuration files.
Append-only files can be opened in write mode but data is always appended
at the end of the file. Like immutable files, they cannot be deleted or renamed.
This is especially useful for log files which can only grow. All-in all
following attributes are avialable at ext2f:
- A
(no
Access time):
if a file or directory has this attribute set, whenever it is accessed,
either for reading of for writing, its last access time will not be updated.
This can be useful, for example, on files or directories which are very
often accessed for reading, especially since this parameter is the only
one which changes on an inode when it's open read-only.
- a
(
append only):
if a file has this attribute set and is open for writing, the only operation
possible will be to append data to its previous contents. For a directory,
this means that you can only add files to it, but not rename or delete any
existing file. Only root
can set or clear this attribute.
- d
(no
dump):
dump (8)
is the standard UNIX
utility for backups. It dumps any filesystem for which the dump counter
is 1 in /etc/fstab
(see chapter
"Filesystems and Mount Points"). But if a file or directory has
this attribute set, unlike others, it will not be taken into account when
a dump is in progress. Note that for directories, this also includes all
subdirectories and files under it.
i
(
immutable):
a file or directory with this attribute set simply can not be modified at
all: it can not be renamed, no further link can be created to it
[1] and it cannot be removed.
Only root
can set or clear this attribute. Note that this also prevents changes to
access time, therefore you do not need to set the
A
attribute when i
is set.
s
(
secure deletion):
when such a file or directory with this attribute set is deleted, the blocks
it was occupying on disk are written back with zeroes.
S
(
Synchronous mode):
when a file or directory has this attribute set, all modifications on it
are synchronous and written back to disk immediately.
Unix filesystem is a classic, but classic has it's own problems: it's actually
an old and largely outdated filesystem that outlived its usefulness. Later
ideas implemented in HPFS, BFS and several other more modern filesystems are absent
in plain-vanilla implementation of Unix file systems. Balanced trees now serve the
base of most modern filesystems including ReiserFs (which started as NTFS clone
but aqured some unique features in the process of development):
The
Reiser Filesystems
by Hans Reiser [and Moscow University researchers], a very ambitious project
to not only improve performance and add journaling, but to redefine the
filesystem as a storage repository for arbitrarily complex objects.
reiserfs.
Reiserfs is faster than ext2/3 because it uses balanced trees for it's directory-structures.
It was used by Suse and Gentoo.
Unfortunately the novel feature introduced in HPFS called extended attributes
never got traction in other filesystems. Of course the fundamental
decision to make attributes indexable deserves closer examination, given the costs
of indexing, but still the fixed set of attributes (like in UFS) created too many
problems to ignore this issue. Still I think that extended attributes should be
present in a filesystem, and they can replace such kludges as #! notation in UNIX
for specifying default processor in executable files.
http://www.scit.wlv.ac.uk/~jphb/spos/notes/ufs.basics.html
These notes describe the basic Unix file system and the kernel structures that
support it. For further information the readers should consult The Design of
Unix Operating System by M.J.Bach (Prentice-Hall 1986 ISBN 0-13-201757-1) and
The Magic Garden Explained by B.Goodheart and J.Cox (Prentice-Hall 1994 0-13-098138-9).
The Bach book is probably easier to read but the Goodheart and Cox book is more
up-to-date.
Modern Unix systems use a Virtual File System (VFS), this allows the system
to use many different actual file systems in a seamless fashion. At a low level,
driver software is required for each actual file system. This allows Network File
Systems (NFS), High-Sierra File Systems (HSFS - found on CDROMs), MSDOS File Systems
(PCFS) amongst others to be included in the Unix view of an integrated hierarchy
of files and directories. Included among the various supported file systems are
the Unix File System (UFS) and the older System V File System (S5FS). These constitute
the traditional Unix file system and will be described in detail in these notes.
When the UFS filesystem was introduced to BSD in 1982, its use of 32 bit
offsets ... structure on the disk for use with systems that don't
understand GPT. ...
www.freebsd.org/projects/bigdisk/index.html - 14k -
Cached -
Similar pages
Early versions of Unix used filesystems referred to simply as FS. FS only
included the boot block, superblock, a clump of inodes, and the data blocks. This
worked well for the small disks early Unixes were designed for, but as technology
advanced and disks got larger, moving the head back and forth between the clump
of inodes and the data blocks they referred to caused
thrashing. BSD optimized this in FFS by inventing cylinder groups, breaking
the disk up into smaller chunks, each with its own inode clump and data blocks.
The intent of BSD FFS is to try to localize associated data blocks and metadata
in the same cylinder group, and ideally, all of the contents of a directory (both
data and metadata for all the files) in the same or nearby cylinder group, thus
reducing
fragmentation
caused by scattering a directory's contents over a whole disk.
Some of the performance parameters in the superblock included number of tracks
and sectors, disk rotation speed, head speed, and alignment of the sectors between
tracks. In a fully optimized system, the head could be moved between close tracks
to read scattered sectors from alternating tracks while waiting for the platter
to spin around.
As disks grew larger and larger, sector level optimization became obsolete (especially
with disks that used linear sector numbering and variable sectors per track). With
larger disks and larger files, fragmented reads became more of a problem. To combat
this, BSD originally increased the filesystem block size from one sector to 1k in
4.0BSD, and, in FFS, increased the filesystem block size from 1k to 8k. This has
several effects. The chances of a file's sectors being contiguous is much greater.
The amount of overhead to list the file's blocks is reduced. The number of blocks
representable in a fixed bit width block number is increased (allowing for larger
disks).
With larger block sizes, disks with many small files would waste a lot of space,
so BSD added block level fragmentation, where the last partial block of data from
several files may be stored in a single "fragment" block instead of multiple mostly
empty blocks.
UFS file system is made of:
- Boot block, the first block of every file system (block 0)
- Superblock
inodes:
UID and GID
rwe for ugo
New blocks allocated to a file are obtained from the free-block list to which blocks
are returned when a file is deleted.
The superblock is followed by blocks containing inodes and associated
inumber pairs. An inode describes an individual file with one inode
for each file in the file system. For each file system is allocated a maximum number
of inodes and therefore a maximum number of files. The maximum values depend on
the the file system size.
Inode 1 on each file system is unnamed and unused. Inode 2 must
correspond to the file system root directory that supports all other files in the
file system. Inodes after inode 2 are free and can be any file. Inodes and blocks
are not allocated in any particular order.
A directory entry, file or link, consists of the name and the
inumber representing the file. The link count indicates the number
of directory entries that refer to the same file. A file is deleted if the link
count is zero. When the file is deleted the associated inode is returned to the
free-inode list and its associated blocks are returned to the
free-block list.
Submitted by
Jeremy on August 7, 2007 - 9:26am.
In a recent lkml thread, Linus Torvalds was involved in
a discussion about mounting filesystems with the
noatime
option for better performance, "
'noatime,data=writeback'
will quite likely be *quite* noticeable (with different
effects for different loads), but almost nobody actually
runs that way." He noted that he set O_NOATIME when
writing git, "
and it was an absolutely huge time-saver
for the case of not having 'noatime' in the mount options.
Certainly more than your estimated 10% under some loads."
The discussion then looked at using the
relatime
mount option to improve the situation, "
relative atime
only updates the atime if the previous atime is older than
the mtime or ctime. Like noatime, but useful for applications
like mutt that need to know when a file has been read since
it was last modified." Ingo Molnar stressed the significance
of fixing this performance issue, "
I cannot over-emphasize
how much of a deal it is in practice. Atime updates are
by far the biggest IO performance deficiency that Linux
has today. Getting rid of atime updates would give us more
everyday Linux performance than all the pagecache speedups
of the past 10 years, _combined_." He submitted some
patches to improve
relatime, and noted about
atime:
"It's also perhaps the most stupid Unix design idea
of all times. Unix is really nice and well done, but
think about this a bit: 'For every file that is read
from the disk, lets do a ... write to the disk! And,
for every file that is already cached and which we read
from the cache ... do a write to the disk!'"
http://ufs-linux.sourceforge.net/docs/ufs.txt
As a file on a UFS filesystem,
the typical open routine called would be the ufs_open() ...
You will find the structure definition for si_user and associated
...
www.itworld.com/Comp/2377/swol-0309-insidesolaris/
- 57k -
Cached -
The ufs file system inode structure
contains the addresses of 12 direct blocks, one indirect block, and one
double indirect block. ...
uw713doc.sco.com/ODM_FSadmin/fssag-4.html - 41k
[May 29, 2007] Developing a file system for AIX by
Srikanth Srinivasan
Learn the intricacies of the AIX® file system framework. Every operating system
provides a native kernel framework that kernel developers have to understand
and adhere to when developing a piece of a kernel component for that operating
system. This article sheds some light on the AIX file system framework. You
need to understand the framework in order to develop a new file system, or to
port an existing file system to the AIX operating system.Introduction
AIX 5L™ is an award-winning operating system that delivers superior scalability,
reliability, and manageability. It is the default operating system that powers some
of the most powerful IBM UNIX® servers in the market.
Typically, a file system can be defined as a piece of software that helps
in storing, organizing, and retrieving data from a physical storage medium, be it
a hard disk drive, CD-ROM, or any other storage device. The code for such data organization,
by its very nature, should be portable. In the real world, though, every operating
system provides its own interfaces by which it requests a particular file system
operation, and it is expected that the underlying piece of software provides results
in the format that the operating system expects. The interfaces vary with different
flavors of operating systems, and need to be exported by the file system to be supported
on the particular operating system.
In this article, you'll learn about the AIX® operating system file system framework.
You'll also get an overview of the IO layer and an explanation of some important
concepts. Brief explanations are also included of the interfaces and methods when
developing a new file system, or when porting an existing file system to the AIX
operating system.
AIX, like many UNIX flavors, hosts
the file system as a kernel
extension. It is assumed that you have basic knowledge of UNIX programming
and file system concepts. It would also be helpful to know how to write kernel extensions
for AIX.
Understanding the logical file system and the virtual file system
The logical file system layer is the level of abstraction at which users can
request the various file operations, such as read, write, stat, and so on. The logical
file system interface supports UNIX-type file access semantics. The logical file
system layer acts as a superset of the virtual file system, which encapsulates disparate
file systems, that provides the kernel with a consistent view of the underlying
directory tree. The logical file system is also responsible for managing the kernel's
open file table and the per process file descriptor information.
The virtual file system is an abstraction of the underlying physical file system.
The virtual file system provides a standard set of interfaces that you should support
in order for your file system to be hosted over the AIX operating system. The virtual
file system bridges the underlying disparate physical file system to the logical
file system, providing a consistent directory tree hierarchy to the rest of the
operating system.
Each unique mount instance of a file system object is represented by a virtual
file system structure. A virtual file system can be a physical file system, a network
file system, or a logical file system (one that does not have a physical backing
store, such as ramfs). Figure 1 shows the AIX file system hierarchy.
In case of broken links
please try to use Google search. If you find the page please notify
us about new location
The Sun Solaris UFS implementation chapter of the
Solaris™ Internals:
Solaris 10 and OpenSolaris Kernel Architecture, Second Edition book
by Richard McDougall, Jim Mauro
ISBN 0-13-148209-2
The UNIX ufs
filesystem
DYNIX-ptx®
System Administration UFS Filesystem
UnixInsider(former SunWorld) paper
Managing filesystem types
From
Solaris
FAQ
Etc
http://home.lanet.lv/~sd70058/aboutos/os.html
by Mark BurgessA short introduction to operating systems
UNIX File System Structure UFS. A UFS file system is made of:.
Boot block, the first block of every file system (block 0); Superblock
Block 1 that contains: ...
www.bo.infn.it/alice/alice-doc/mll-doc/duix/admgde/node39.html
- 6k -
Cached -
Similar pages <
http://www.bo.infn.it/alice/alice-doc/mll-doc/duix/admgde/node122.html
Little UFS2 FAQ
Linux
userspace UFS2 tools.
Filesystems-HOWTO (part of
The Linux Documentation Project, link is not linux specific.)
UFS2 Tools: An open source tool for accessing UFS2 (BSD) slices from
within Windows
"Writing
AIX kernel extensions" (developerWorks, Aug 2006): This article explains
how to write a kernel extensions for AIX.
Open AFS: OpenAFS has an AIX port
for the AFS file system.
The Linux Documentation Project's
Filesystems HOWTO: FFS. Note that the distinction this draws between
FFS and UFS is wrong; both terms are used at present and have been used
in the past.
Little UFS2 FAQ: What is the difference between UFS and FFS? Note that
this gets the relationship between FFS and UFS backwards; see the "Local
Filesystems" chapter of The Design and Implementation of the 4.4BSD Operating
System, which refers to the upper layer as UFS, and the "Local Filestores"
chapter, which refers to the lower layer as FFS.
Kernel Extensions and Device Support Programming Concepts
,
SC23-4900-03, to learn about kernel programming and the kernel environment
for AIX.
Technical Reference: Kernel and Subsystems, Volume 1
, SC23-4917-03,
for detailed information about kernel services, device driver operations,
and file system operations for AIX.
Technical Reference: Kernel and Subsystems, Volume 2
, SC23-4918-03,
for detailed information about the configuration subsystem, communications
subsystem, LFT subsystem, printer subsystems, SCSI subsystem, Integrated
Device Electronics, SSA subsystem, and the serial DASD subsystem for
AIX.
http://www.uwsg.iu.edu/UAU/filesystem/
Copyright © 1996-2008 by Dr. Nikolai Bezroukov.
www.softpanorama.org was
created as a service to the UN Sustainable Development Networking Programme (SDNP)
in the author free time.
Submit
comments This document is an industrial compilation designed and created
exclusively for educational use and is placed under the copyright of the
Open Content License(OPL).
Original materials copyright belong to respective owners. Quotes are made
for educational purposes only in compliance with the fair use doctrine.
Standard disclaimer: The statements, views and opinions presented on
this web page are those of the author and are not endorsed by, nor do they necessarily
reflect, the opinions of the author present and former employers, SDNP or any other
organization the author may be associated with. We do not warrant the correctness
of the information provided or its fitness for any purpose.
Last modified:
June 02, 2008