Configure IBM Spectrum Protect KM

Configuring Spectrum Protect Server Monitoring

IBM Spectrum Protect KM allows you to monitor several Spectrum Protect servers and storage agents with one PATROL agent. The configuration procedure is as follows:

1. Right-click the IBM Spectrum Protect main instance > Configuration > Spectrum Protect Servers…

   

2. Specify the Spectrum Protect server details:

   • Server ID: Enter an ID for the Spectrum Protect server.
   • (Required) Hostname/IP Address/FQDN: Enter the hostname, IP address, or FQDN of the node to be monitored.
   • (Required) System Type: Specify whether the Spectrum Protect node is a UNIX/Linux or Windows system.
   • (Required) Status: Select Enable to start the Spectrum Protect node monitoring; Disable to stop it.

3. Check the Create a separate device in TrueSight box if you want the Spectrum Protect server instance to appear as a separate device in TrueSight OM.

4. Click Update.

A Spectrum Protect Setup: <node-id> instance appears in the PATROL Console but the monitoring will only start after appropriate credentials are specified using the Configuration > Login and Configuration > Admin Interface Login KM Commands.

Configuring the Monitoring of Multiple Server Instances

IBM Spectrum Protect KM for PATROL supports the monitoring of multiple IBM Spectrum Protect Server and Storage Agent instances locally and remotely without the need to run multiple PATROL Agents. The KM can be installed on any PATROL Agent node to monitor Spectrum Protect servers remotely. Therefore, the IBM Spectrum Protect Administrative Client software needs to be installed on the PATROL Agent node. Follow the Administrative Client software installation procedure to install and configure the API before installing the KM.

Upon the initial discovery, the IBM Spectrum Protect KM will discover the IBM Spectrum Protect Admin Client installation on the PATROL Agent node and search for the dsmadmc binary file path under the default paths. If the file is not detected, a message will be displayed in the SOW (System Output Window) and in the TSM_<port>.log file in the PATROL log folder. This Admin Client path is shared between all Spectrum Protect server instances monitored from the PATROL Agent node. It is possible to manually configure the path, if the Admin Client is installed under a custom path.

Configuring the KM User Account

A user account with administrative privileges must be configured in BMC PATROL or BMC TrueSight Operations Management to read and execute IBM Spectrum Protect application programs and access file systems. Depending on the operating systems used, several options will be available. For more information, refer to Security Requirements.

To configure the KM user account:

1. In the Console, right-click the host instance > KM Commands > Configuration > Login… The Login Configuration dialog box is displayed:

   

2. To use:

   • the default PATROL Agent Account, check the Use Agent Default Account box and leave the Username, Password and Private Key File fields empty
   • a different user account, enter the login details in the Username, Password and optional Private Key File fields. For SSH authentication, you can specify the path to a private key file on the PATROL Agent corresponding to the specified username and password.

3. If a sudo user account is used:

   • check the Use Sudo box
   • click SUDO Binary File to verify the sudo binary file path

4. If you are monitoring a remote host, click Remote Connection Settings:

   

   • Connection Timeout (in Seconds): Enter the number of seconds after which the connection to the remote node will timeout.
   • Maximum Connections: Enter the maximum number of simultaneous connections allowed to the remote node.
   • Click Accept.

5. Click OK.

If the credentials are not valid, the TSMLoginStatus parameter will appear in “suspicious state” after the next data collection.

To configure the Admin Interface login

1. In the Console, right-click the server instance > Configuration > Admin Interface Login… The Admin Interface Login Configuration dialog is displayed:

   

2. Enter the login details for the KM user for DSMADMC (suggested name is patrol).

3. Select Create new account if you want the KM to create a new account if it has not already been set up. This operation requires a DSMADMC user account with administrative privileges. This new account will be created in the Spectrum Protect server instance defined in the DSM_CONFIG file selected in the KM Paths menu Configuration > Paths.

4. Click DSMADMC Binary File to verify or change the path on the PATROL Agent system.

   

5. Click OK.

If the credentials are not valid, the TSMLoginStatus parameter will appear in “suspicious state” after the next data collection.

Verifying the Application Discovery

To verify the application discovery:

1. Check that the Spectrum Protect Setup instance changes to display the server type. If not, check the message in the Spectrum Protect Setup instance and in the PATROL Console System Output Window (SOW) or in the log file <PATROL_HOME>/log/TSM_<port>.log. You can force a full discovery by right-clicking the Spectrum Protect Setup instance > KM Commands > Force Discovery.

2. Check that the application class instances are actually instantiated. The initial data collection may take some time depending on the complexity of your environment (up to one hour). If needed, force a data collection by right-clicking the server instance > KM Commands > Refresh Parameters.

If IBM Spectrum Protect server is configured in a clustered environment, you will have to enable multi-node mode monitoring.

Configuring the Instance Limits

All Spectrum Protect components are monitored, except for Clients, Domain Clients, Mount Requests and Manual Drives, which are disabled by default. Because there may be a very large number of instances to monitor that may represent an important workload to the PATROL Agents and console, it is recommended to only monitor the critical ones. You can customize the monitoring of any component to better suit your requirements.

To modify the default configuration:

1. In the Console, right-click the server instance > KM Commands > Configuration > Instance Limits…

   

2. Enter the number of instances to be displayed in the console for each monitored element

3. To disable the monitoring of a specific application class, enter 0 (zero) in the relevant field

4. Click OK.

Verifying the Initial Setup

To check the initial setup of the KM:

1. Check for any error on the PATROL Console System Output Window (SOW) or in the <PATROL_HOME>/log/TSM_<port>.log.

2. Check whether the discovered files and directory paths are correct, using the menu Configuration > Paths… from the server instance.

3. Refer to the label of the server instance to check whether the discovered node type is correct. The node type can be modified, using the menu Configuration > Node Type… from the server instance.

4. Check the Infoboxes and Parameters of all the IBM Spectrum Protect KM instances and verify that they are properly set.

5. Under each container instance, open all detail parameters (text parameters with names in the format TSM<Object>Details), and check for error messages.

6. Check the new value of the TSMLoginStatus and TSMNodeStatus parameters set after the completion of all data collection. If either is not in OK state, open the annotation point to determine the cause of the error. IBM Spectrum Protect KM will check the status of the node using query status through DSMADMC interface. Any errors in the query will set the parameter TSMNodeStatus to trigger a warning or alarm. This error filter can be changed through the menu Configuration > Node Status…. Refer to Configuring the Node Status for more details. If the query cannot be executed, the KM will check the existence of the following daemons to determine the status of the node. By default, these daemons are sought:

   On a Server:

   Daemon	Description
   dsmserv	Spectrum Protect Server Daemon

   On a Storage Agent:

   Daemon	Description
   dsmsta	Spectrum Protect Storage Agent Daemon

7. After the KM has been running for at least an hour:

   • Right-click the server instance > KM commands > KM Status… from the IBM Spectrum Protect instance. A report is displayed, detailing information on the managed system. Check through the report for any warning or alarm. If any error is found during any of the above steps (including any incorrect data in Infoboxes or any warning or alarm in the KM Status report), determine the cause(s) and take steps to resolve these errors. You may need to refer to the User Guide, or contact Technical Support, attaching the KM Status report.
   • If needed, click Save As Task to open the report in a task window. You could later save it as a text file or print it.

8. The KM installed on the server may monitor the same storage pools, automated libraries and drives which are already monitored through the storage agents. This may be unnecessary and can trigger duplicate alerts. You can disable these components on either server or storage agent by setting their instance limit to 0 (zero), using the menu Configuration > Instance Limits….

Restarting PATROL Agent and PATROL Console

Restarting the PATROL Agent and PATROL Console is optional but often a good practice to confirm that the IBM Spectrum Protect KM operates correctly.

In addition, the PATROL Agent restart will force all KM discoveries and data collections to start immediately.

Configuring Alerts

Configuring the Discovery Overrides

The solution automatically discovers the installation, binary, and library paths. This information can however be overridden.

To configure paths:

1. In the Console, right-click the server instance > KM commands > Configuration > Paths…

2. In the KM section, verify and possibly modify the Home, Temp, and Debug Directory Paths

3. In the Spectrum Protect section, verify and possibly modify the Installation, the DSM_DIR Directory, the DSM_CONFIG File, DSM_LOG Directory, and the Remote Temp Directory (remote nodes only) Paths

   Note: The DSM_CONFIG file will contain the .opt file that specifies the Spectrum Protect server instance ID and the required connection settings used by the Spectrum ProtectAdmin Client to connect to the Spectrum Protect server.

4. In the Binary Paths and Library Paths sections:

   • From the list, select the paths to be removed
   • In the Path field, enter the new path(s) to be included. If several paths are to be entered, separate them with a colon (:).

5. Click OK.

If you want to rediscover the paths, empty all fields, click OK, and restart the PATROL Agent.

Configuring Logs

By default, the following log files are monitored. These log file paths are relative to the Spectrum Protect installation directory (for example /opt/tivoli/tsm on UNIX and C:\Program Files\TSM on Windows), unless the path is fully pathed.

The <server-id> will be converted automatically with the Spectrum Protect server instance ID.

Configuring Databases

By default, the following database paths are monitored. These database paths are relative to the IBM Spectrum Protect installation directory (for example /opt/tivoli/tsm on UNIX and C:\Program Files\TSM on Windows).

Filtering Elements to Monitor

By default, the solution monitors all the discovered elements, which may represent an important workload to the agents and servers. Because the monitoring of some elements may be irrelevant for various reasons, you can apply filters to indicate which elements will be monitored or not.

Note that some instances have specific monitoring rules:

• Daemons: Only the Server Daemon or the Storage Agent Daemon is monitored
• Jobs: Future jobs are not monitored by the KM
• Automated Libraries and Manual Drives: On a Storage Agent, only the local automated libraries and local manual drives discovered will be monitored

To filter elements to monitor:

1. In the Console, right-click one of the following instances depending on the elements that you wish to monitor:

   • Automated Drives > KM Commands > Configuration > Automated Drives… to filter automated drives to be monitored
   • Automated Libraries > KM Commands > Configuration > Automated Libraries… to filter automated libraries to be monitored
   • Clients > KM Commands > Configuration > Clients… to filter clients to be monitored
   • Client Nodes > KM Commands Configuration > Client Nodes… to filter client nodes to be monitored
   • Daemons > KM Commands > Configuration > Daemons… to filter daemons to be monitored
   • Domains > KM Commands > Configuration > Domains… to filter domains to be monitored
   • Databases > KM Commands Configuration > Databases… to filter databases to be monitored
   • Logs > KM Commands Configuration > Logs… to filter logs to be monitored
   • Manual Drive > KM Commands Configuration > Manual Drive… to filter manual drives to be monitored
   • Policies > KM Commands Configuration > Policies… to filter policies to be monitored
   • Storage Pools > KM Commands Configuration > Storage Pools… to filter storage pools to be monitored
   Note: These menus are also available from the child instances and will apply to all objects created under the child instances.

   

   The [ALL] entry ensures that all the elements found during PATROL discovery are instantiated and monitored.

2. To enable elements for monitoring:

   • In the current configuration list, select the elements to enable
   • Select Enable
   • Click Update

   A log filter will have to be configured if you enable log files for monitoring.

3. To disable elements for monitoring:

   • In the current configuration list, select the elements to disable
   • Select Disable
   • Click Update

   All jobs associated to disabled policies are not monitored.

   Disabled instances will be instantiated in an OFFLINE state. Commands will not be executed, thus leaving parameters not set.

4. To delete elements for monitoring:

   • In the current configuration list, select the elements to delete
   • Select Delete
   • Click Update

5. To add a new element to monitor:

   • Type the name of the element to be monitored (as it appears in the PATROL console) or a regular expression to be monitored
   • If needed, enter a description
   • From the Status list, select Enable
   • Click Update.

   To revert to the default monitoring configuration, type [default] as a new entry and click Update.

6. Resume the procedure for all the application classes for which you wish to apply monitoring filters.

Configuring the Jobs Monitoring

By default, IBM Spectrum Protect KM monitors failed and suspicious jobs for 1 day. The successfully completed jobs are not monitored. You can however modify these settings to better suit your requirements.

1. In the Console, right-click the Jobs instance > KM Commands > Configuration > Jobs…

   

2. Indicate how long the jobs in OK, Suspicious, and Failure status will be monitored. If you:

   • decrease the monitoring period, older jobs will be destroyed on the next data collection cycle
   • increase the monitoring period, you will have to force a full data collection to ensure all jobs are instantiated. To force data collection, right-click the Jobs instance > KM Commands > Force Full Collection

3. Select:

   • Keep Monitoring Active Jobs Indefinitely if you want to endlessly monitor active jobs
   • Ignore Inactive Jobs initially if you want the inactive jobs not to be created in the initial discovery cycle. Initially discovering and monitoring inactive jobs may impact the performance of the application
   • Ignore Alerting for Child Jobs if you do not want to be notified when a job failure occurs on child jobs.

4. Click OK.

Configuring the Mount Requests Monitoring

By default, IBM Spectrum Protect KM monitors mount requests for 1 day. You can however modify these settings to better suit your requirements.

1. In the Console, right-click the Mount Requests instance > KM Commands > Configuration > Mount Requests…

   

2. Indicate how long the mount requests will be monitored. If you decrease the monitoring period, older mount requests will be destroyed on the next data collection cycle

3. Click OK.

Configuring the Processes Monitoring

1. In the Console, right-click the Processes instance > KM Commands > Configuration > Processes…

   

2. Indicate how long the processes will be monitored. If you:

   • decrease the monitoring period, older processes will be destroyed on the next data collection cycle
   • increase the monitoring period, you will have to force a full data collection to ensure all processes are instantiated. To force data collection, right-click the Processes instance > KM Commands > Force Full Collection

3. Click OK.

Configuring the Multi-Node Monitoring Mode

When IBM Spectrum Protect is installed in a cluster environment, i.e. active on one cluster node and passive on others, false alarms and duplicate alerts may occur. To avoid such situation, users need to configure IBM Spectrum Protect KM for PATROL in multi-node monitoring mode, if Spectrum Protect is installed in a supported cluster.

To configure the multi-node monitoring mode:

1. In the Console, right-click the server instance > KM Commands > Configuration > Monitoring Mode…

   

2. Select Multi-node Mode.

3. From the Cluster Application pull-down list, select an appropriate cluster application.

4. In the Failover Group Name field, enter the failover group name, which is online on one cluster node and offline on the others. Leave this field blank if you previously selected Veritas Cluster File System. The IBM Spectrum Protect KM will then monitor the entire cluster from the active master system, which is identified by the vxdctl -c mode command. This method requires vxconfigd in enable mode with its clustered state active.

5. Provide the details of all managed nodes of the cluster to be configured in the multi-node mode:

   • PATROL Agent Hostname: host where the PATROL Agent is installed
   • Node ID: the unique ID of the Spectrum Protect server instance as configured in the KM menu IBM Spectrum Protect instance > KM Command > Configuration > Spectrum Protect Servers…
   • Cluster Node Name: the hostname defined in the selected Cluster Application.

6. Click Update.

7. Resume the procedure for all the other Spectrum Protect servers and click Update.

8. Check the Remote PATROL Agent Checking option if there are more than one PATROL Agent involved in the multi-node mode configuration above. If enabled, the KM queries the other PATROL Agents to check the monitoring status of other nodes.

9. Click Close.

10. If the Remote PATROL Agent Checking is Enabled, you will have to provide all the information required to communicate with the PATROL Agents (protocol, port number, credentials, attempts and timeout).

11. Click Close.

The Spectrum Protect server will then be monitored through the master or online node in Active Multi-node Mode. The other nodes, which are standing by for a failover, will be in Passive Multi-node Mode, monitoring only the components that are not visible from the active node.

If a managed node is unable to check the monitoring status of the active managed node, it will change to Temporary Single-node Mode allowing full monitoring. It will remain in Temporary Single-node Mode until it finds the active node in full monitoring mode again.

If the Remote PATROL Agent Checking is Disabled, while there are more than one PATROL Agent involved, the managed node on the master or online node will be in Active Multi-node Mode and all others will be in Passive Multi-node Mode, without checking the monitoring status of the active node. In addition, the above procedure to configure Multi-node Mode (server instance > KM Commands > Configuration > Monitoring Mode) needs to be repeated from each PATROL Agent involved.

Configuring the Node Status

IBM Spectrum Protect KM allows you to manually customize the regular expressions used for the node status:

1. In the Console, right-click the Server instance > KM Commands > Configuration > Node Status…

   

2. Provide the Node Status Alarms and Warnings: Enter the regular expressions to determine the Node Status in Alarm or Warning status. This regular expression will be used on the result from DSMADMC command: “query status” on the Spectrum Protect server

By default, the following “Included” expressions are checked. If detected, they set the parameter TSMNodeStatus to alert, unless the message also contains an “Excluded” expression. Type [default] to reset to the default configuration.

Error Expression		TSMNodeStatus
ANS1017E		Failure
ANS1473E		Failure
ANS8023E		Failure
Availability: Disabled		Failure
Central Scheduler: Inactive		Suspicious
Server License Compliance: FAILED		Suspicious

1. Click Update.

Configuring the Node Type

You may have to manually indicate the type of node monitored if the discovery fails to recognize it:

1. In the Console, right-click the server instance > KM Commands > Configuration > Node Type…

   

2. Select the node type (Server or Storage Agent)

3. Click OK.

Configuring the Domain Backup Restriction Window

IBM Spectrum Protect KM can be configured to trigger an alert when a backup is started during a specific period of time:

1. In the Console, right-click:

   • the Domains instance > KM Commands > Configuration > Domains Backup Restriction… to apply this customization to all policies
   • a Domain instance > KM Commands > Configuration > Domain Backup Restriction… to apply this customization to a specific policy

   

2. Select Enabled.

3. Indicate the Restriction Start and End Time. The restriction window should at least last 5 minutes.

4. Click OK.