Reinitialize the PMDB

Procedure for initializing the replicated PMDB on both SMWs.

When troubleshooting SMW-HA, there may be instances in which the SMW requires to be manual reinitialization.

  1. Initialize the replicated PMDB on both SMWs.
    NOTE: The following substeps require switching back and forth between the active and passive SMWs. Pay attention to the command line prompts in the examples.
    1. Enable maintenance mode.
      smw1# maintenance_mode_configure enable
       Maintenance mode was enabled
    2. Initialize the active SMW.
      smw1# pmdb_util ha --init_master
      ...
      [initialize()]:     INFO: -----------------------------------------------
      [initialize()]:     INFO: PMDB Initialization SUCCEEDED
      [initialize()]:     INFO: -----------------------------------------------
      CAUTION: Ensure that the last three lines of the output look as above. Do not proceed until the initialization has finished.
    3. Start the service for the active SMW.
      smw1# systemctl start postgresql
      NOTE: If postgresql is unable to start, it is possible that the cluster was in transition when it was put into maintenance mode at the beginning of this step. If this happens, perform the following steps to allow the cluster to transition, then return to the beginning of step 1a and repeat this procedure.
      smw1# maintenance_mode_configure disable
      smw1# sleep 30
    4. Initialize the passive SMW.
      smw2# pmdb_util ha --init_standby
      [main()]:     INFO: Initializing standby...
      [init_standby()]:     INFO: Initializing HA standby system...
      [init_standby()]:     INFO: Old data directory removed.
      [init_standby()]:     INFO: Synchronizing this standby with master. This might take a while!
      [init_standby()]:     INFO: Initial replication successful! Full output:
      [init_standby()]:     INFO: 		NOTICE:  WAL archiving is not enabled; 
      you must ensure that all required WAL segments are copied through other means to complete the backup
      [init_standby()]:     INFO: Standby successfully initialized!
      CAUTION: Ensure that the last lines of the output look as above. Do not proceed until the initialization has finished.
    5. Start the service for the passive SMW.
      smw2# systemctl start postgresql
    6. Disable maintenance mode.
      smw1# maintenance_mode_configure disable
       Maintenance mode was disabled
      
      smw1# sleep 300
  2. Check the status of the cluster.
    smw1# crm status
    1. Run the following command if processes remain "Stopped" after waiting 300 seconds.
      smw1# clear_failcounts
      smw1# sleep 300
    2. Run the following command to restart all resources if processes remain stopped after waiting another 300 seconds.
      smw1# clean_resources
      smw1# sleep 300
  3. Report the system status.
    smw1# ha_health
    smw1# check_config smw1 smw2 smw1-drac smw2-drac