Filesystems, directories and quotas
Objectives
You will learn:
- what the different filesystems are
- what they should be used for
- what the quotas are on the different filesystems
Filesystems
The table below indicates the range of default allocations available in /home, /nesi/project and /nesi/nearline filesystems (exceptions are possible, based on project requirements).
|
Filesystem |
/home |
/nesi/project |
/nesi/nobackup |
/nesi/nearline |
|
Default Quota |
few GB (max. 100,000 files) |
up to TB (max 20,000 files / TB) |
No storage Quota (1 Million file limit) |
No Size Limit, 500,000 files, each no smaller than 5 MB |
|
Usage |
private data |
project related data storage |
job input / output workspace |
archive |
|
Capacity |
175 TB |
1,590 TB |
4,400 TB |
>100 PB (media funded by each project) |
|
Access Speed |
Moderate |
Moderate |
Fast |
Slow. Only accessible via the Librarian Service. |
|
Expiration |
End of Project |
End of Project |
60 days (or earlier if space required) |
End of Project |
|
Data Backup |
Daily, last 10 versions retained for 90 days. |
Daily, last 10 versions retained for 90 days. |
None |
Replicated to offsite tape library |
|
Snapshots |
Daily (retention period 7 days) |
None |
None |
None |
Using the filesystems
/home
- Every user has a home directory
- Private data that you don’t need to share with anyone else (e.g. other project members and/or NeSI staff)
- Backed up
/nesi/project
- Every project has a directory on this filesystem: /nesi/project/<project-id>
- Critical data (time enduring)
- Source code, shared data
- Recommended for building software
- Backed up
/nesi/nobackup
- Every project has a directory on this filesystem: /nesi/nobackup/<project-id>
- This is where most data should be read/written and is the highest performance filesystem
- Actively managed to remove unused data and not backed up
- Advise to move valuable data to the /nesi/project filesystem as soon as the batch jobs are completed
/nesi/nearline
- Data cache for HSM
- Holds the records of all data written to tape via the librarian service
- Note: The librarian service is not completely setup yet.