sap_ha_pacemaker_cluster Ansible Role

Ansible Lint for sap_ha_pacemaker_cluster

Ansible Role for installation and configuration of Linux Pacemaker for High Availability of SAP Systems run on various Infrastructure Platforms.

Support Status

The support status of this role is: “Technology Preview”

Scope

This Ansible Role provides:

This Ansible Role has been tested for the following SAP Software Solution scenario deployments:

This Ansible Role contains Infrastructure Platform specific alterations for:

Please note, this Ansible Role sap_ha_pacemaker_cluster is acting as a wrapper and generates the parameter definitions for a given SAP System, Infrastructure Platform specific variables and other additional steps to complete the SAP High Availability setup using Linux Pacemaker clusters.

Warnings :warning:

Functionality

All of the following functionality is provided as Technology Preview.

SAP HANA scale-up (performance-optimized) with SAP HANA System Replication, High Availability using Linux Pacemaker 2-node cluster

Platform Usability
:heavy_check_mark: physical server expected to work with any fencing method that is supported by the ha_cluster Linux System Role
:heavy_check_mark: OVirt VM tested and working
:heavy_check_mark: AWS EC2 VS platform detection and awscli setup included, tested and expected to work

SAP NetWeaver (ABAP) ASCS and ERS, High Availability using Linux Pacemaker 2-node cluster

Platform Usability
:heavy_check_mark: physical server expected to work with any fencing method that is supported by the ha_cluster Linux System Role
:heavy_check_mark: OVirt VM tested and working
:heavy_check_mark: AWS EC2 VS platform detection and awscli setup included, tested and expected to work

Requirements

The Ansible Role requires the SAP HANA Database Server or SAP NetWeaver Application Server software installation to already exist on the target host/s.

The target host must have:

N.B. At this time SLES4SAP functionality is not available, until crmsh commands are provided in dependency Ansible Role ha_cluster

The Ansible Control System (where Ansible is executed from) must have:

Prerequisites

All SAP Software must be installed, and all remote/file storage mounts must be available with correct permissions defined by SAP documentation. For SAP HANA High Availability, SAP HANA System Replication must already be installed.

In addition, the following network ports must be available:

SAP Technical Application and Component Port
SAP HANA System Replication  
hdbnameserver
used for log and data shipping from a primary site to a secondary site.
System DB port number plus 10,000
4<sap_hana_instance_no>01
hdbnameserver
unencrypted metadata communication between sites.
System DB port number plus 10,000
4<sap_hana_instance_no>02
hdbnameserver
used for encrypted metadata communication between sites.
System DB port number plus 10,000
4<sap_hana_instance_no>06
hdbindexserver
used for first MDC Tenant database schema
4<sap_hana_instance_no>03
hdbxsengine
used for SAP HANA XSC/XSA
4<sap_hana_instance_no>07
hdbscriptserver
used for log and data shipping from a primary site to a secondary site.
Tenant port number plus 10,000
4<sap_hana_instance_no>40-97
hdbxsengine
used for log and data shipping from a primary site to a secondary site.
Tenant port number plus 10,000
4<sap_hana_instance_no>40-97
Linux Pacemaker  
pcsd
cluster nodes requirement for node-to-node communication
2224 (TCP)
pacemaker
cluster nodes requirement for Pacemaker Remote service daemon
3121 (TCP)
corosync
cluster nodes requirement for node-to-node communication
5404-5412 (UDP)

Execution Flow

The Ansible Role is sequential:

Tips

Check out the role variables of the ha_cluster Linux System Role for additional possible settings that can be applied when using the sap_ha_pacemaker_cluster role.

For example:

Adding ha_cluster_start_on_boot: false to disable the automatic start of cluster services on boot.

Sample

Please see a full sample using multiple hosts to create an SAP S/4HANA Distributed deployment in the /playbooks directory of the Ansible Collection sap_install.

License

Apache 2.0

Author Information

Red Hat for SAP Community of Practice, Janine Fuchs, IBM Lab for SAP Solutions


Role Input Parameters

Minimum required parameters for all clusters:

Additional minimum requirements depend on the type of cluster setup and on the target platform.

sap_ha_pacemaker_cluster_aws_access_key_id

AWS access key to allow control of instances (for example for fencing operations).

Mandatory for the cluster nodes setup on AWS EC2 instances.

sap_ha_pacemaker_cluster_aws_region

The AWS region in which the instances to be used for the cluster setup are located.

Mandatory for cluster nodes setup on AWS EC2 instances.

sap_ha_pacemaker_cluster_aws_secret_access_key

AWS secret key, paired with the access key for instance control.

Mandatory for the cluster setup on AWS EC2 instances.

sap_ha_pacemaker_cluster_aws_vip_update_rt

List one more routing table IDs for managing Virtual IP failover through routing table changes.

Multiple routing tables must be defined as a comma-separated string (no spaces).

Mandatory for the VIP resource configuration in AWS EC2 environments.

sap_ha_pacemaker_cluster_cluster_name

The name of the pacemaker cluster.

Inherits the ha_cluster LSR native parameter ha_cluster_cluster_name if not defined.

If not defined, the ha_cluster Linux System Role default will be used.

sap_ha_pacemaker_cluster_cluster_nodes

List of cluster nodes and associated attributes to describe the target SAP HA environment.

This is required for the HANA System Replication configuration.

Synonym for this parameter is sap_hana_cluster_nodes.

Mandatory to be defined for HANA clusters.

Example:

sap_ha_pacemaker_cluster_cluster_nodes:
- hana_site: DC01
  node_ip: 192.168.5.1
  node_name: nodeA
  node_role: primary
- hana_site: DC02

sap_ha_pacemaker_cluster_cluster_properties

Standard pacemaker cluster properties are configured with recommended settings for cluster node fencing.

When no STONITH resource is defined, STONITH will be disabled and a warning displayed.

Example:

sap_ha_pacemaker_cluster_cluster_properties:
  concurrent-fencing: true
  stonith-enabled: true
  stonith-timeout: 900

sap_ha_pacemaker_cluster_create_config_dest

The pacemaker cluster resource configuration optionally created by this role will be saved in a Yaml file in the current working directory.

Requires sap_ha_pacemaker_cluster_create_config_varfile to be enabled for generating the output file.

Specify a path/filename to save the file in a custom location.

The file can be used as input vars file for an Ansible playbook running the ‘ha_cluster’ Linux System Role.

sap_ha_pacemaker_cluster_create_config_varfile

When enabled, all cluster configuration parameters this role constructs for executing the ‘ha_cluster’ Linux System role will be written into a file in Yaml format.

This allows using the output file later as input file for additional custom steps using the ‘ha_cluster’ role and covering the resource configuration in a cluster that was set up using this ‘sap_ha_pacemaker_cluster’ role.

When enabled this parameters file is also created when the playbook is run in check_mode (--check) and can be used to review the configuration parameters without executing actual changes on the target nodes.

WARNING! This report may include sensitive details like secrets required for certain cluster resources!

sap_ha_pacemaker_cluster_enable_cluster_connector

Enables/Disables the SAP HA Interface for SAP ABAP application server instances, also known as sap_cluster_connector.

Set this parameter to ‘false’ if the SAP HA interface should not be installed and configured.

sap_ha_pacemaker_cluster_extra_packages

Additional extra packages to be installed, for instance specific resource packages.

For SAP clusters configured by this role, the relevant standard packages for the target scenario are automatically included.

sap_ha_pacemaker_cluster_fence_agent_minimal_packages

The minimal set of fence agent packages that will be installed.

sap_ha_pacemaker_cluster_fence_agent_packages

Additional fence agent packages to be installed.

This is automatically combined with sap_ha_pacemaker_cluster_fence_agent_minimal_packages.

sap_ha_pacemaker_cluster_gcp_project

Google Cloud project name in which the target instances are installed.

Mandatory for the cluster setup on GCP instances.

sap_ha_pacemaker_cluster_gcp_region_zone

Google Cloud Platform region zone ID.

Mandatory for the cluster setup on GCP instances.

sap_ha_pacemaker_cluster_ha_cluster

The ha_cluster LSR native parameter ha_cluster can be used as a synonym.

Optional host_vars parameter - if defined it must be set for each node.

Dictionary that can contain various node options for the pacemaker cluster configuration.

Supported options can be reviewed in the ha_cluster Linux System Role [https://github.com/linux-system-roles/ha_cluster/blob/master/README.md].

If not defined, the ha_cluster Linux System Role default will be used.

Example:

sap_ha_pacemaker_cluster_ha_cluster:
  corosync_addresses:
  - 192.168.1.10
  - 192.168.2.10
  node_name: nodeA

sap_ha_pacemaker_cluster_hacluster_user_password required

The password of the hacluster user which is created during pacemaker installation.

Inherits the value of ha_cluster_hacluster_password, when defined.

sap_ha_pacemaker_cluster_hana_automated_register

Parameter for the ‘SAPHana’ cluster resource.

Define if a former primary should be re-registered automatically as secondary.

sap_ha_pacemaker_cluster_hana_duplicate_primary_timeout

Parameter for the ‘SAPHana’ cluster resource.

Time difference needed between to primary time stamps, if a dual-primary situation occurs.

If the time difference is less than the time gap, then the cluster holds one or both instances in a “WAITING” status.

This is to give an admin a chance to react on a failover. A failed former primary will be registered after the time difference is passed.

sap_ha_pacemaker_cluster_hana_instance_nr

The instance number of the SAP HANA database which this role will configure in the cluster.

Inherits the value of sap_hana_instance_number, when defined.

Mandatory for SAP HANA cluster setups.

sap_ha_pacemaker_cluster_hana_prefer_site_takeover

Parameter for the ‘SAPHana’ cluster resource.

Set to “false” if the cluster should first attempt to restart the instance on the same node.

When set to “true” (default) a failover to secondary will be initiated on resource failure.

sap_ha_pacemaker_cluster_hana_resource_clone_name

Customize the cluster resource name of the SAP HANA DB resource clone.

sap_ha_pacemaker_cluster_hana_resource_name

Customize the cluster resource name of the SAP HANA DB resource.

sap_ha_pacemaker_cluster_hana_sid

The SAP HANA SID of the instance that will be configured in the cluster.

The SID must follow SAP specifications - see SAP Note 1979280.

Inherits the value of sap_hana_sid, when defined.

Mandatory for SAP HANA cluster setups.

sap_ha_pacemaker_cluster_hana_topology_resource_clone_name

Customize the cluster resource name of the SAP HANA Topology resource clone.

sap_ha_pacemaker_cluster_hana_topology_resource_name

Customize the cluster resource name of the SAP HANA Topology resource.

sap_ha_pacemaker_cluster_host_type

The SAP landscape to for which the cluster is to be configured.

The default is a 2-node SAP HANA scale-up cluster.

sap_ha_pacemaker_cluster_ibmcloud_api_key

The API key which is required to allow the control of instances (for example for fencing operations).

Mandatory for the cluster setup on IBM Cloud Virtual Server instances or IBM Power Virtual Server on IBM Cloud.

sap_ha_pacemaker_cluster_ibmcloud_powervs_api_type

IBM Power Virtual Server API Endpoint type (public or private) dependent on network interface attachments for the target instances.

Mandatory for the cluster setup on IBM Power Virtual Server from IBM Cloud.

sap_ha_pacemaker_cluster_ibmcloud_powervs_forward_proxy_url

IBM Power Virtual Server forward proxy url when IBM Power Virtual Server API Endpoint type is set to private.

When public network interface, can be ignored.

When private network interface, mandatory for the cluster setup on IBM Power Virtual Server from IBM Cloud.

sap_ha_pacemaker_cluster_ibmcloud_powervs_workspace_crn

IBM Power Virtual Server Workspace service cloud resource name (CRN) identifier which contains the target instances

Mandatory for the cluster setup on IBM Power Virtual Server from IBM Cloud.

sap_ha_pacemaker_cluster_ibmcloud_region

The IBM Cloud VS region name in which the instances are running.

Mandatory for the cluster setup on IBM Cloud Virtual Server instances or IBM Power Virtual Server on IBM Cloud.

sap_ha_pacemaker_cluster_msazure_resource_group

Resource group name/ID in which the target instances are defined.

Mandatory for the cluster setup on MS Azure instances.

sap_ha_pacemaker_cluster_msazure_subscription_id

Subscription ID of the MS Azure environment containing the target instances.

Mandatory for the cluster setup on MS Azure instances.

sap_ha_pacemaker_cluster_nwas_abap_aas_instance_nr

Instance number of the NetWeaver ABAP AAS instance.

Mandatory for NetWeaver AAS cluster configuration.

sap_ha_pacemaker_cluster_nwas_abap_ascs_ers_ensa1

The standard NetWeaver ASCS/ERS cluster will be set up as ENSA2.

Set this parameter to ‘true’ to configure it as ENSA1.

sap_ha_pacemaker_cluster_nwas_abap_ascs_filesystem_resource_name

Name of the filesystem resource for the ASCS instance.

sap_ha_pacemaker_cluster_nwas_abap_ascs_group_stickiness

NetWeaver ASCS resource group stickiness to prefer the ASCS group to stay on the node it was started on.

sap_ha_pacemaker_cluster_nwas_abap_ascs_instance_nr

Instance number of the NetWeaver ABAP ASCS instance.

Mandatory for NetWeaver ASCS/ERS cluster configuration.

sap_ha_pacemaker_cluster_nwas_abap_ascs_sapinstance_automatic_recover_bool

NetWeaver ASCS instance resource option “AUTOMATIC_RECOVER”.

sap_ha_pacemaker_cluster_nwas_abap_ascs_sapinstance_ensa1_failure_timeout

NetWeaver ASCS instance failure-timeout attribute.

Only used for ENSA1 setups (see sap_ha_pacemaker_cluster_nwas_abap_ascs_ers_ensa1). Default setup is ENSA2.

sap_ha_pacemaker_cluster_nwas_abap_ascs_sapinstance_ensa1_migration_threshold

NetWeaver ASCS instance migration-threshold setting attribute.

Only used for ENSA1 setups (see sap_ha_pacemaker_cluster_nwas_abap_ascs_ers_ensa1). Default setup is ENSA2.

sap_ha_pacemaker_cluster_nwas_abap_ascs_sapinstance_instance_name

The name of the ASCS instance, typically the profile name.

Mandatory for the NetWeaver ASCS/ERS cluster setup

sap_ha_pacemaker_cluster_nwas_abap_ascs_sapinstance_resource_name

Name of the ASCS instance resource.

sap_ha_pacemaker_cluster_nwas_abap_ascs_sapinstance_resource_stickiness

NetWeaver ASCS instance resource stickiness attribute.

sap_ha_pacemaker_cluster_nwas_abap_ascs_sapinstance_start_profile_string

The full path and name of the ASCS instance profile.

Mandatory for the NetWeaver ASCS/ERS cluster setup.

sap_ha_pacemaker_cluster_nwas_abap_ers_filesystem_resource_name

Name of the filesystem resource for the ERS instance.

sap_ha_pacemaker_cluster_nwas_abap_ers_instance_nr

Instance number of the NetWeaver ABAP ERS instance.

Mandatory for NetWeaver ASCS/ERS cluster configuration.

sap_ha_pacemaker_cluster_nwas_abap_ers_sapinstance_automatic_recover_bool

NetWeaver ERS instance resource option “AUTOMATIC_RECOVER”.

sap_ha_pacemaker_cluster_nwas_abap_ers_sapinstance_instance_name

The name of the ERS instance, typically the profile name.

Mandatory for the NetWeaver ASCS/ERS cluster setup.

sap_ha_pacemaker_cluster_nwas_abap_ers_sapinstance_resource_name

Name of the ERS instance resource.

sap_ha_pacemaker_cluster_nwas_abap_ers_sapinstance_start_profile_string

The full path and name of the ERS instance profile.

Mandatory for the NetWeaver ASCS/ERS cluster.

sap_ha_pacemaker_cluster_nwas_abap_pas_instance_nr

Instance number of the NetWeaver ABAP PAS instance.

Mandatory for NetWeaver PAS cluster configuration.

sap_ha_pacemaker_cluster_nwas_abap_sid

SID of the NetWeaver instances.

Mandatory for NetWeaver cluster configuration.

Uses sap_swpm_sid if defined.

Mandatory for NetWeaver cluster setups.

sap_ha_pacemaker_cluster_nwas_sapmnt_filesystem_resource_name

Filesystem resource name for the shared filesystem /sapmnt.

Optional, this is typically managed by the OS, but can as well be added to the cluster configuration.

Enable this resource setup using sap_ha_pacemaker_cluster_nwas_shared_filesystems_cluster_managed.

sap_ha_pacemaker_cluster_nwas_shared_filesystems_cluster_managed

Change this parameter to ‘true’ if the 3 shared filesystems /usr/sap/trans, /usr/sap/<SID>/SYS and ‘/sapmnt’ shall be configured as cloned cluster resources.

sap_ha_pacemaker_cluster_nwas_sys_filesystem_resource_name

Filesystem resource name for the transports filesystem /usr/sap//SYS.
Optional, this is typically managed by the OS, but can as well be added to the cluster configuration.
Enable this resource setup using `sap_ha_pacemaker_cluster_nwas_shared_filesystems_cluster_managed`.

sap_ha_pacemaker_cluster_nwas_transports_filesystem_resource_name

Filesystem resource name for the transports filesystem /usr/sap/trans.

Optional, this is typically managed by the OS, but can as well be added to the cluster configuration.

Enable this resource setup using sap_ha_pacemaker_cluster_nwas_shared_filesystems_cluster_managed.

sap_ha_pacemaker_cluster_resource_defaults

Set default parameters that will be valid for all pacemaker resources.

Example:

sap_ha_pacemaker_cluster_resource_defaults:
  migration-threshold: 5000
  resource-stickiness: 1000

sap_ha_pacemaker_cluster_stonith_custom

Custom list of STONITH resource(s) to be configured in the cluster.

This definition override any defaults the role would apply otherwise.

Example:

sap_ha_pacemaker_cluster_stonith_custom:
- agent: stonith:fence_rhevm
  name: my-fence-resource
  options:
    ip: rhevm-server
    password: login-user-password
    pcmk_host_list: node1,node2
    power_wait: 3
    username: login-user

sap_ha_pacemaker_cluster_storage_definition

List of filesystem definitions used for filesystem cluster resources.

Options relevant, see example.

Mandatory for SAP NetWeaver HA cluster configurations.

Reuse sap_storage_setup_definition if defined.

Reuse sap_storage_setup_definition will extract values ‘mountpoint’, ‘nfs_filesystem_type’, ‘nfs_mount_options’, ‘nfs_path’, ‘nfs_server’.

Reuse sap_storage_setup_definition all options are documented under Ansible Role sap_storage_setup.

Note! For this variable, the argument specification does not list options, to avoid errors during reuse of sap_storage_setup_definition if defined.

Example:

sap_ha_pacemaker_cluster_storage_definition:
- mountpoint: /usr/sap
  name: usr_sap
  nfs_path: /usr/sap
  nfs_server: nfs-server.example.com:/
- mountpoint: /usr/sap/trans
  name: usr_sap_trans
  nfs_path: /usr/sap/trans
  nfs_server: nfs-server.example.com:/
- mountpoint: /sapmnt
  name: sapmnt
  nfs_filesystem_type: nfs
  nfs_mount_options: defaults
  nfs_path: /sapmnt
  nfs_server: nfs-server.example.com:/

sap_ha_pacemaker_cluster_storage_nfs_filesytem_type

Filesystem type of the NFS filesystems that are part of the cluster configuration.

sap_ha_pacemaker_cluster_storage_nfs_mount_options

Mount options of the NFS filesystems that are part of the cluster configuration.

sap_ha_pacemaker_cluster_storage_nfs_server

Default address of the NFS server, if not defined individually by filesystem.

sap_ha_pacemaker_cluster_system_roles_collection

Reference to the Ansible Collection used for the Linux System Roles.

For community/upstream, use ‘fedora.linux_system_roles’.

For RHEL System Roles for SAP, or Red Hat Automation Hub, use ‘redhat.rhel_system_roles’.

sap_ha_pacemaker_cluster_vip_client_interface

OS device name of the network interface to use for the Virtual IP configuration.

When there is only one interface on the system, its name will be used by default.

sap_ha_pacemaker_cluster_vip_hana_primary_ip_address

The virtual IP of the primary HANA instance.

Mandatory parameter for HANA clusters.

sap_ha_pacemaker_cluster_vip_hana_primary_resource_name

Customize the name of the resource managing the Virtual IP of the primary HANA instance.

sap_ha_pacemaker_cluster_vip_hana_secondary_ip_address

The virtual IP for read-only access to the secondary HANA instance.

Optional parameter in HANA clusters.

sap_ha_pacemaker_cluster_vip_nwas_abap_aas_ip_address

Virtual IP of the NetWeaver AAS instance.

Mandatory for NetWeaver AAS cluster setup.

sap_ha_pacemaker_cluster_vip_nwas_abap_aas_resource_name

Name of the SAPInstance resource for NetWeaver AAS.

sap_ha_pacemaker_cluster_vip_nwas_abap_ascs_ip_address

Virtual IP of the NetWeaver ASCS instance.

Mandatory for NetWeaver ASCS/ERS cluster setup.

sap_ha_pacemaker_cluster_vip_nwas_abap_ascs_resource_group_name

Name of the NetWeaver ASCS resource group.

sap_ha_pacemaker_cluster_vip_nwas_abap_ascs_resource_name

Name of the SAPInstance resource for NetWeaver ASCS.

sap_ha_pacemaker_cluster_vip_nwas_abap_ers_ip_address

Virtual IP of the NetWeaver ERS instance.

Mandatory for NetWeaver ASCS/ERS cluster setup.

sap_ha_pacemaker_cluster_vip_nwas_abap_ers_resource_group_name

Name of the NetWeaver ERS resource group.

sap_ha_pacemaker_cluster_vip_nwas_abap_ers_resource_name

Name of the SAPInstance resource for NetWeaver ERS.

sap_ha_pacemaker_cluster_vip_nwas_abap_pas_ip_address

Virtual IP of the NetWeaver PAS instance.

Mandatory for NetWeaver PAS cluster setup.

sap_ha_pacemaker_cluster_vip_nwas_abap_pas_resource_name

Name of the SAPInstance resource for NetWeaver PAS.

sap_ha_pacemaker_cluster_vip_secondary_resource_name

Customize the name of the resource managing the Virtual IP of read-only access to the secondary HANA instance.