Storage

Storage server

Chimera has a storage server that is connected to the head node and compute nodes over QDR InfiniBand. This storage server is where home directories and group filesystems live (filesystems are mounted with NFS). This is where any long-term storage should be kept for Chimera. The server has been optimized for performance and reliability while making quite a bit of disk space available.

Filesystems

Each user has their own 100GB filesystem which is used for their home directory (/usa/USER). Additionally, each user will be in a group led by one of the faculty investigators. Each faculty investigator has a group filesystem as well that is readable and writable to members of their group. This group filesystem is available at /usa/FACULTY-grp (where FACULTY is the username for the faculty investigator) and will automatically be mounted when someone changes to that directory. The group filesystem is 1TB for free by default. Additional space can be purchased for this filesystem in TB increments. To see the size of a filesystem and how much is used and how much is available run the "df -h ." command in a directory for that filesystem.

Scratch Space

Additionally there is local storage on each compute node available at /scratch/local/. This space can be used while programs are running, but no long-term storage should be kept there. Files will be removed from there. Any data needed for long-term storage should be copied to the users' home (if a small amount) or to /usa/FACULTY-grp (if a large amount or something that should be shared between users in the group).

Snapshots

The storage server also uses ZFS and snapshots are taken at regular times. This is a good way to retrieve files that have been accidentally deleted or modified. For more info on this see http://www.eecis.udel.edu/wiki/ececis-docs/index.php/FAQ/FileStorage#toc5

Technical info on Storage server

The storage server is optimized for performance and reliability. The storage space is made up of mirrored pairs of disks. Additionally, there is a pair of SLC SSDs that are mirrored to speed up writes. There is an MLC SSD that is used for a read cache (for frequently read data that does not fit in memory). Also there is a hot spare disk drive for the mirrored storage disks. All of the data on the storage server (under /usa) is replicated to a backup server.

Storage Needs

The following table gives the storage requirements for each investigator: