MetaCenter data storage news

1) Introduction of quotas for the maximum number of files

Due to the growing amount of data in our arrays, some disk operations are already disproportionately long. Problems are mainly caused by mass manipulations with data (copying of entire user directories, searching, backup, etc.). Complications are mainly caused by a large number of files.

We would like to ask you to check the number of files in your home directories and reduce it, if possible (zip, rar,..). The current quota status can be checked like the following:

on the Metacentrum website in the table My account -> Quota http://metavo.metacentrum.cz/en/myaccount/kvoty
the table also appears after you login on a frontend (MOTD)

The quota will be set to 1 - 2 million files per user. We plan to introduce quotas gradually in the coming months. We have alrerady started with new storages.

If you have enough space on your storage directories, you can keep the packed data there. However we encourage users to archive the data that are of permanent value, large and not accessed frequently. If you really need to keep large numbers of files in your home directory, contact us at user support e-mail meta@cesnet.cz

To reduce the number of files, please use access directly via /storage frontends, as described on our wiki in the section Working with data: https://wiki.metacentrum.cz/wiki/Working_with_data

2) Data backup

Information about data backup or snapshoting is provided on the above-mentioned wiki page Working with data https://wiki.metacentrum.cz/wiki/Working_with_data , including recommendations how to handle different types of data.

Some large disk arrays have a backup policy of saving snapshots (once a day, the backup is done usually during the night) of user's data. The snapshots are kept at least 14 days backwards. This offers some protection in case user unintentionally deletes some of his/her files. Generally, data that existed the day before the accident can be recovered.
Selected disk arrays with limited capacity are backuped.
Some disk arrays are not backed up and it is advisable to keep this in mind when storing important data.

To check the backup mode of individual disk arrays can be found

in the MOTD table each time you log on a frontend
https://wiki.metacentrum.cz/wiki/Working_with_data#Disk_arrays

3) Restrictions on the possibility of writing to home directories by another users

To increase the security of our users, we have decided to remove the possibility of writing to the root home directories by another users (ACL group and other), which contain sensitive files such as .k5login, .profile, etc. (to avoid manipulation with it).

Please be informed, from 1. 7. we start to automatically check the rights in root home user direstories, writing of other users (except the owner) will not be allowed. The ability to write to other subdirectories, typically due to data sharing within the group, remains.

More information can be found on our wiki pages in the section Data sharing in the group: https://wiki.metacentrum.cz/wiki/Sharing_data_in_group

MetaCentrum

Ivana Křenková, Mon Jun 07 14:24:00 CEST 2021