Snapshot Policy

As of version 4.4, WEKA has introduced the Snapshot policy in the UI. Before that version an external tool was needed to create a Snapshot Policy called SnapTool.

Snapshot policies establish the rules and schedules for creating, managing, and retaining point-in-time copies of data, known as snapshots. These snapshots provide a reliable mechanism for data recovery or rollback in scenarios such as accidental deletion, corruption, or other data integrity issues.

By automating the creation of snapshots based on specified criteria, such as time intervals or frequency, snapshot policies enhance data protection, streamline disaster recovery, and ensure business continuity. They enable organizations to restore their data to a consistent state quickly without requiring full backups, optimizing storage usage while minimizing the risk of data loss.

Note

More detailed information can be found at WEKA Docs

Goals for this module

The following items are going to be discussed in this module:

Snapshot Policy Actions
- Using the UI
- Using the CLI

Snapshot Policy - Actions (UI)

To use a Snapshot Policy, it has to be created and then assigned to a Filesystem

Create

Switch back to the browser and navigate to Manage > Snapshot Policies. If you have been logged out, make sure you're logged in as test-admin
The cluster will have one Policy defined. This policy is there as an example. It can be used, but can not be modified.
As we want to create our own schedule, click + Create Policy at the far right of the screen.
Provide the following parameters for the Policy
- Policy Name: W-Tues-01AM
- Description: Create a snapshot every Tuesday at 01:00 AM and keep 4 snapshots before removing the oldest version
- For the below schedules, set the retention to 1, de-select the Upload to OBS and select the corresponding toggle switch
  - Hourly: Off
  - Daily: Off
  - Monthly: Off
  - Periodic: Off
- Weekly: On
  - Retention: 4
  - Upload to OBS: both unchecked!!!^*1
  - Every week at: 01:00AM
  - On: Tuesday
Warning

As the settings are very self explanatory, we will not discuss all the settings that can be set for the different time based snapshot schedules. Two items may be for now unknown, Upload to OBS; Local and Remote. This setting tells the cluster to upload the snapshot to a local S3 bucket, or a remote S3 bucket. Both can also be selected. The difference between these (Local and Remote) is that a type of Local OBS can be used for Tiering. the type Remote can JUST be used for snapshots. Not for tiering.
Click Save

Assign

Now that we have the Snapshot Policy, we have to assign it tho a Filesystem. Click in the far right corner, the chain icon ()
Select the fs_ui Filesystem and click Attach

Note

When you have selected the Filesystem, click somewhere next to the dropdown box to see the Attach button.
The Filesystem should appear at the right side Assigned Filesystems panel

Update

Now that we have a Policy, it is assigned to a Filesystem, we could wait till we hit the 01:00AM on a Tuesday marker, but we're gonna change the policy so it makes the snapshot TODAY in 15 minutes.

Select the created Policy and change the Weekly schedule to be fired today in 15 minutes. So as example, it is Thursday 10:13 AM, we want to make sure the snapshot is created every Thursday at 10:28 AM.
Make the changes needed and click Update.
Wait till the time has passed and then navigate to Manage > Snapshots. You will see the snapshot that has been created due to the Policy.

Note

The naming convention of the Snapshot will be Name of the Policy - Schedule triggered.time of snapshot

Dis-/enable

Sometimes, as example due to maintenance, it may be necessary to stop a Policy.

Navigate back to Manage > Snapshot Policies
Click Enabled text in the W-Tues-01AM Policy
In the message that pops-up, click Yes to confirm the Disabling of the Policy
Enabling the Policy again, just repeat the above steps, but then click the Disabled text and confirm the message by clicking Yes

Unassign

Another possibility it that the Policy isn't needed anymore, or at least not currently. It may be used in the very near future, so it doesn't have to be deleted.

To "unassign" a Policy from a Filesystem, hoover over the Fielsystem in the right hand side of the screen and click the unlink icon () that appeared.
In the screen that appears, toggle the switch to On and confirm the un-assignment by clicking Detach

Remark

Remove Waiting tasks, means that tasks that may be in the queue to be executed will be removed and not waited until they have ran
The fs_ui Filesystem will be removed from the right hand pane in the Assigned Filesystem column as shows No Filesystems Found

Delete

Deleting Policies is quiet simple and can be used in circumstances where the Policy is not needed anymore.

Hoover over the W-Tues-01AM Policy and a bin will appear
Click the bin icon () and confirm the deletion with clicking the Yes button

Snapshot Policy - Actions (CLI)

What we have been doing via the UI can also be done using the CLI. This makes it to automate the creation of the schedule based on more dynamic needs.

Create

As the command line for the schedule needs to have a JSON file as input, we need to have one. To do this we are going to EXPORT the sys-default schedule and then manipulate it to our needs.

Open the SSH session to the client using the Terminal Emulator

Warning

You may have been logged out off the client use the ssh and weka user login commands to get back in to the system
Type the following command to export the sys-default Policy

weka fs protection snapshot-policy export sys-default /tmp/sys-default.json
Copy the file to a new name using cp /tmp/sys-default.json /tmp/w-tues-01am.json
Use vi to manipulate the json file. Make sure you use sudo as otherwise you are not able to manipulate the json file.

vi /tmp/w-tues-01am.json

As we want to have the same Policy as the UI based, we will be manipulating the JSON file with the following content:

{
    "daily": {
        "days": "all",
        "enable": false,
        "retention": 1,
        "time": "22:05",
        "upload": "none"
    },
    "hourly": {
        "days": "monday, tuesday, wednesday, thursday, friday",
        "enable": false,
        "hours": "09, 10, 11, 12, 13, 14, 15, 16, 17, 18",
        "minuteOffset": 5,
        "retention": 1,
        "upload": "none"
    },
    "monthly": {
        "days": "07",
        "enable": false,
        "months": "all",
        "retention": 1,
        "time": "00:05",
        "upload": "none"
    },
    "periodic": {
        "days": "monday, tuesday, wednesday, thursday, friday",
        "enable": false,
        "end_time": "18:00",
        "interval": 30,
        "retention": 1,
        "start_time": "09:00",
        "upload": "none"
    },
    "weekly": {
        "days": "tuesday",
        "enable": true,
        "retention": 4,
        "time": "01:00",
        "upload": "none"
    }
}

Tip

The easiest way of exchanging the content of the file with the above, is to move ot the first line and type 50dd. This will remove 50 lines (dd) and provides a clean file. Then type i to go into the insert mode and then paste <CTRL>+<SHIFT>+V the content, and usse <ESC\>:qw! to write and closet the file

Now that we have the JSON file with the correct content we can create the Policy.

weka fs protection snapshot-policy create W-Tu-01AM-cl /tmp/w-tues-01am.json --description "Create a snapshot every Tuesday at 01:00 AM and keep 4 snapshots before removing the oldest version" --enabled true

The Policy should appear in the web browser in max. 2 seconds

Assign

The next step is to assign the Policy to a Filesystem. In our case the fs_cli Filesystem.

Type the following command to assign the Policy and the result of the command will be shown in the browser
```
weka fs protection snapshot-policy attach W-Tu-01AM-cl fs_cli
```
This way we now have the snapshot assigned to the Filesystem. As we don't want to wait till the schedule fires, we are going to update the Policy so it get triggered.

Update

As the CLI only allows for JSON files as parameters, we have to edit the JSON file to reflect the following:

Day: Current day
Time: Current time +15 minutes

Run the following command to modify the JSON file vi /tmp/w-tues-01am.json
Change the weekly section to correspond with the earlier mentioned day and time. Below screenshot is an example (Day and time of screenshot was Friday, 09:23)

Tip

TIme is 24hrs notation for the hours!
Then run the command to update the policy weka fs protection snapshot-policy update W-Tu-01AM-cl --path /tmp/w-tues-01am.json
After you ran the command the Policy will have been updated according to the Day and Time you provided in the JSON file (you may have to refresh the webpage to see the change)
To see the snapshots that are available in the cluster run weka fs snapshot --file-system fs_cli
The result is, until the time has passed for the schedule, empty.

Tip

Instead of running the command once every while, you can use the watch command that 'reruns' a given command every 2 Seconds (default). So the command would be watch weka fs snapshot --file-system fs_cli
Wait till the snapshot is shown before proceeding as the next step is disabling the Policy using the CLI.

Dis-/Enable

Disabling is quiet simple from the CLI, same as using the UI.

Run the following command to disable the Policy weka fs protection snapshot-policy update W-Tu-01AM-cl --path /tmp/w-tues-01am.json --enabled false
The UI will also show that the Policy has been disable

Unassign

For the same reason as in the UI part of this module, there may be circumstances where the Policy needs to be unassigned (detached), but not deleted.

Run the command weka fs protection snapshot-policy detach W-Tu-01AM-cl --remove-waiting-tasks fs_cli -f

Remark

As we now added the -f option, the Policy is forced detached from the Filesystem. No confirmation had to be given
The UI will not show any Filesystems in the right hand pane of Assigned Filesystems

Delete

To delete the Policy run the command weka fs protection snapshot-policy delete W-Tu-01AM-cl -f
After running the command the Policy has been removed and the UI will only show the sys-default Policy as expected.

Clean up of the snapshots

Now that we are done using snapshot, we will create new onces in another module, we want to clear out the snapshots we currently have. Run the following command to clear out all existing snapshots:

weka fs snapshot delete $(weka fs snapshot -o filesystem,name --no-header|tr -s " ") -f

Footnotes

^*1 We haven't touched OBS (Object Storage) yet, but this is using S3 buckets

This concludes this module in the lab