“I’m out of space!” Reclaiming space quick reference.
Space Reclamation and Deduplication are two essential processes for the correct functioning of the purge process. They include the processes space_reclaimer and fileDedup
For more information on "Out of Space" conditions see: Error: No more space on device
- Ensure the system is not running 10.0 or 9.2 releases, which have specific retention defects that can cause improper free space usage and also improper reservation hold issues for stale reservations. Upgrading appliances to release 10.1 or higher will itself solve the vast majority of space issues. Suspending new backups or incoming replication for up to 24 hours may be necessary in extreme cases to ensure logically deleted backups process successfully without filling the unit again to resolve this state.
- Check your systems current D2D size under Configure > Storage > Select your storage device > Edit. Ensure the appropriate balance of Backups to instant Recovery space is allocated, and that the device size is consistent with your storage device capacity.
- Check your Capacity Report and data reduction reports and review the following KB for information on resolving capacity overage issues: Error: No more space on device
- If your system is a UB appliance, ensure it is not making backups of itself or other unitrends UBs in your environment.
- If none of the steps above work, you may still be protecting more data than your unit can support with use of deduplication active. Deduplication is intended to be used when multiple full backups can concurrently co-exist. As an appliance data set increases over time, it may reach a point where this is no longer possible and deduplication reservations may actually reduce, not improve, retention. In these cases, re-imaging an appliance and manually disabling deduplication may be an interim step to provide successful backups if you cannot upgrade to a larger capacity appliance or deploy an additional appliance to sustain the data set. Please contact Unitrends Sales for details on pricing.
The process for a backup to be removed is as follows:
If a system is consistently suffering from space concerns, consideration should be taken to determine if the root cause for the space concerns can be alleviated by adjusting the backup strategy. When determining if a strategy is appropriate, keep in mind the various backup types:
- Logical deletion of the backup. This is a database operation, where the backup record is assigned a deletion flag for purging.
- Devmonitor initiates the Space_reclaimer process to begin a purge operation.
- Space_reclaimer initiates a query to get a list of logically deleted backups.
- This list is passed to fileDedup to unDeduplicate the backups selected.
- On inline deduplicated CentOS 6 systems, blocks are directly removed from the SIS
- On older syastems and Cent 5 systems (including Cent 5 systems field upgraded to Cent 6), SIS blocks must be moved from one SIS file to another to be saved, resulting in temporary space requirements, that if not met can lead to data purging issues.
- Space_reclaimer takes the file listing for the unDeduplicated backup and removes it from the system.
- Devmonitor then records the freed up space and continues to manage the space on the DPU.
Unitrends recommends incremental forever strategies for all clients on all OS that support that model. Database backups require scheduled full backups, and these shoudl be done not more than weekly, and in many cases vendors recommend biweekly or monthly database fulls using combinations of differentials and transaction backups between. Scheduling fulls more frequently than weekly for databases will result in severe degredation in retention capabilities as database files generally have poor deduplication capability. Database dump operations (bacling up DB exports from 3rd party tools) is much worse and will acheive near zero deduplciation capability. Ensuring database fulls are as far apart as would be recommended by the Application vendor and DBA is the best practice.
- Full backup: a capture of all data being protected.
- Incremental backup: a capture of the data that has changed since last backup.
- Differential backup: a capture of the data that has changed since last Full backup.
The incremental forever strategy is a backup strategy that differs from the traditional backup routine. Unlike a typical backup strategy where a fresh master backup is taken of the client system at regular intervals, this strategy employs a single primary master backup that is taken initially and then every subsequent backup is an incremental backup only. Periodically, based on certain defaults and retention settings, the autosynth process will evaluate a need for a new master backup and create this backup from the previous master backup and the incremental backups collected up to the present day minus one day. This strategy has a number of different benefits however, on older systems that are not using inline deduplication, it shodl be noted that although a synthetic full eliminates the time expense and backup load on client systems, there's no space savings. On inline deduplicated clients (typically on "S" series units, though some other Generation 6 and 7 units support this feature), synthesis is done in-place fully deuplicated and will always save space vs scheduled full backups and is the default recomendation for all compatible systems.
Retention settings play a part of the autosynth process by acting as a trigger method for when to perform the new master backup. This can be determined by how often the incremental backups run and what the min/max day retention settings. The condition is as follows, If the maxmin-1 > 14 then every 14 days on a daily incremental backup schedule a new synthetic master is run. If the max-min-7 < 14 then the value of the max-min-1 is when the new master is run. Now this behavior can change depending on how regularly the incremental backup is taken. It is
possible with a 14 day either default or min/max setting to see a new master run sooner if the number of incrementals taken per day is increased beyond one incremental per day, in the cases of hourly incrementals.
Autosynth, incremental process:
- contact client
- Undedupe last Master
- Copy last Master
- Append or overwrite with Incremental changes.
- Recompress new Master
- Register new Master
- Remove all previous data
Q. How do we stop auto synth?
A. This process should not be disabled. Disabling it prevents incremental backups from working entirely. Additionally, synthesis has the same space requirements as scheduled fulls (or less), if synthesis if happening too frequently resulting in capacity issues, address retention settings noted above. If device capacity is preventing successful synthesis, remove clients from protection and delete their backups, or, upgrade to a larger capacity unitrends appliance. All unitrends appliances are sold with the expectation to maintain a minimum of 2 fulls of each client at all times. If your unique data set, change rates, compressability, or size of set prevents this from being possible on your particular deployment, then no amount of software adjusments alone will solve this issue.
What’s your priority?
This will be custom to the end user of the system. Each backup strategy has its benefits and draw backups. If a customer wants a faster recovery time. Then a master and differential backup strategy would, in most cases, be the optimum solution as this backup strategy only requires to backups to be restored for a full restoration of the data. Three in the cases of a full server restore from a disaster scenario.
Application based backups, i.e. Exchange, oracle, Sharepoint, and SQL backups, share the same behavior with regards to the backup process, but do not utilize synthesis. The application logs only get truncated on specific backup types, and this differs by database type. For example, Exchange truncates on fulls or incrementals (but not diffs). SQL truncates only when using the transaction lob backup type and does not truncate on fulls. Within the application based backup schedule it is important to make sure a regularly scheduled full/master backup of the application is taken at regular intervals. The log truncation portion of the backup operation occurs after the backup has successfully been captured. Accommodating the appropriate balance bewtween log retention, storage requirements, and backup frequency is a key question backup vendors are often asked to answer, but this answer can only come from anylizing server-side configuration and business requirements, and that question shoudl be answered by your database administrator or Microsoft support. Unitrends can configure any backup schedule necessary to meet business requirements, however, use of more frequent fulls or many backups per-day may reduce appliance retention accordingly to support those models.