Parallel Virtual File System Publisher's description
from Rob Ross
In recent years the disparity between I/O performance and processor performance has led to I/O bottlenecks in many applications, especially those using large data sets.
In recent years the disparity between I/O performance and processor performance has led to I/O bottlenecks in many applications, especially those using large data sets. A popular approach for alleviating this kind of bottleneck is the use of parallel file systems, which can be found on many commercial parallel computers.
The goal of the Parallel Virtual File System (PVFS) Project is to explore the design, implementation, and uses of parallel I/O. PVFS serves as both a platform for parallel I/O research as well as a production file system for the cluster computing community. PVFS is currently targeted at clusters of workstations, or Beowulfs.
The PVFS project is conducted jointly between The Parallel Architecture Research Laboratory at Clemson University and The Mathematics and Computer Science Division at Argonne National Laboratory. Additional funding for the PVFS project comes from NASA Goddard Space Flight Center Code 930 and The National Computational Science Alliance through the National Science Foundation's Partnerships for Advanced Computational Infrastructure.
PVFS provides the following features:
* Compatibility with existing binaries
* Ease of installation
* User-controlled striping of files across nodes
* Multiple interfaces, including a MPI-IO interface via ROMIO
* Utilizes commodity network and storage hardware
PVFS supports the UNIX I/O interface and allows existing UNIX I/O programs to use PVFS files without recompiling. The familiar UNIX file tools (ls, cp, rm, etc.) will all operate on PVFS files and directories as well. This is accomplished via a Linux kernel module which is provided as a separate package.
PVFS is easy to install. The Quick Start page describes how to set up a simple installation. Scripts and test applications are included to help with configuration, testing for correct operation, and performance evaluation.
PVFS stripes file data across multiple disks in different nodes in a cluster. By spreading out file data in this manner, larger files can be created, potential bandwidth is increased, and network bottlenecks are minimized. A 64-bit interface is implemented as well, allowing large (more than 2GB) files to be created and accessed.
Multiple user interfaces are available. This includes:
* MPI-IO support through ROMIO (see the ROMIO homepage for details)
* Traditional Linux file system access through the pvfs-kernel package
* The native PVFS library interface
What's New in This Release:Changes:
- fixes to build under redhat 2.4.20-20.9 kernel
- checks for missing headers on ancient 2.4 kernels
- removal of unused fields from metadata and control messages
- updated pvfsd rc file
- handle various combinations of redhat kernel patches
- fixes to kpvfsd that had previously been fixed in user space
- fix for sftp directory reading problems
- fix to avoid oopses on clients when bad errno values returned
- adjustments to deal wiht gcc 3.4.0 issues
- fixes to work with redhat 9
- fixes to compile kernel code for xx86_64
- updated logging intrastructure
- fixes to error reporting
- update of makefile in pvfs core
- fix to bug in pvfs_open.c where pcount < 0 caused errors
- removed duplicated code in mgr
- fix for problem with bvrecv() optimization
- more unused code removal
- adjustments to keep portland group compiler happy
- removed sd_path from metadata
- new iod.rc and mgr.rc files
- fixes to chmod
- fixes to get correct fs_ino back to clients
- multiple mtime fixes
- streamlined pvfs file detection
- shared library building
- fixed closing socket bug in iod
- reordered messaging in mgr to iods to improve concurrency
- rename fix
- improved metadata file checking
- do_access permission fixes
- fix for "iod out of space" client hang
- cleaner dead socket handling, including special handling of case where
mgr would close fds that clients were using
- noninteractive mkmgrconf
System Requirements:No special requirements.
Program Release Status: Minor Update
Program Install Support: Install and Uninstall