Pacemaker brought the PostgreSQL process back to running state.There was no disruption of the writer application.
Standby server was marked as failed. Manual intervention was required to start the PostgreSQL process again.There was no disruption of the writer application.
Patroni brought the PostgreSQL process back to running state.There was no disruption of the writer application.
Stop the PostgreSQL process
Pacemaker brought the PostgreSQL process back to running state.There was no disruption of the writer application.
Standby server was marked as failed. Manual intervention was required to start the PostgreSQL process again.There was no disruption of the writer application.
Patroni brought the PostgreSQL process back to running state.There was no disruption of the writer application.
Reboot the server
Standby server was marked offline initially. Once the server came up after reboot, PostgreSQL was started by Pacemaker and the server was marked as online. If fencing was enabled then node wouldn’t have been added automatically to cluster.There was no disruption of the writer application.
Standby server was marked as failed. Once the server came up after reboot, PostgreSQL was started manually and server was marked as running.There was no disruption of the writer application.
Patroni needs to be started after reboot, unless configured to not start on reboot. Once Patroni was started, it started the PostgreSQL process and setup the standby configuration.There was no disruption of the writer application.
Stop the framework agent process
Agent: pacemakerThe PostgreSQL process was stopped and was marked offline.There was no disruption of the writer application.
Agent: repmgrdThe standby server will not be part of automated failover situation.PostgreSQL service was found to be running.There was no disruption of the writer application.
Agent: patroniIt did not stop the PostgreSQL process.patronictl list did not display this server.There was no disruption of the writer application.
Pacemaker brought the PostgreSQL process back to running state. Primary got recovered within the threshold time and hence election was not triggered.There was downtime in the writer application.
repmgrd started health check for primary server connection on all standby servers for a fixed interval. When all retries failed, an election was triggered on all the standby servers. As a result of the election, the standby which had the latest received LSN got promoted. The standby servers which lost the election will wait for the notification from the new master node and will follow it once they receive the notification.Manual intervention was required to start the postgreSQL process again.There was downtime in the writer application.
Patroni brought the PostgreSQL process back to running state. Patroni running on that node had primary lock and hence election was not triggered.There was downtime in the writer application.
Stop the PostgreSQL process and bring it back immediately after health check expiry
Pacemaker brought the PostgreSQL process back to running state. Primary got recovered within the threshold time and hence election was not triggered.There was downtime in the writer application.
repmgrd started health check for primary server connections on all standby servers for a fixed interval. When all the retries failed, an election was triggered on all the standby nodes. However, the newly elected master didn’t notify the existing standby servers since the old master was back.Cluster was left in an indeterminate state and manual intervention was required.There was downtime in the writer application.
Patroni brought the PostgreSQL process back to running state. Patroni running on that node had primary lock and hence election was not triggered.There was downtime in the writer application.
Reboot the server
Election was triggered by Pacemaker after the threshold time for which master was not available. The most eligible standby server was promoted as the new master. Once the old master came up after reboot, it was added back to the cluster as a standby. If fencing was enabled, then node wouldn’t have been added automatically to cluster.There was downtime in the writer application.
repmgrd started election when master connection health check failed on all standby servers. The eligible standby was promoted. When this server came back, it didn’t join the cluster and was marked failed. repmgr node rejoin command was run to add the server back to the cluster.There was downtime in the writer application.
Failover happened and one of the standby servers was elected as the new master after obtaining the lock. When Patroni was started on the old master, it brought back the old master up and performed pg_rewind and started following the new master.There was downtime in the writer application.
Stop the framework agent process
Agent: pacemakerThe PostgreSQL process was stopped and it was marked offline.Election was triggered and new master was elected.There was downtime in writer application.
Agent: repmgrdThe primary server will not be part of the automated failover situation.PostgreSQL service was found to be running.There was no disruption in writer application.
Agent: patroniOne of the standby servers acquired the DCS lock and became the master by promoting itself.The old master was still running and it led to multi-master scenario. The application was still writing to the old master.Once Patroni was started on the old master, it rewound the old master (use_pg_rewind was set to true) to the new master timeline and lsn and started following the new master.
Network isolate the master server from other servers (split brain scenario)
Corosync traffic was blocked on the master server.PostgreSQL service was turned off and master server was marked offline due to quorum policy.A new master was elected in the majority partition.There was a downtime in the writer application.
All servers have the same value for location in repmgr configuration:repmgrd started election when master connection health check failed on all standby servers.The eligible standby was promoted, but the PostgreSQL process was still running on the old master node.There were two nodes running as master. Manual intervention was required after the network isolation was corrected.The standby servers have the same value for location but the primary had a different value for location in repmgr configuration:repmgrd started election when master connection health check failed on all standby servers.But, there was no new master elected since the standby servers had location different from that of the primary.repmgrd went into degrade monitoring mode. PostgreSQL was running on all the nodes and there was only one master in the cluster.
DCS communication was blocked for master node.PostgreSQL was demoted on the master server.A new master was elected in the majority partition.There was a downtime in the writer application.
Network-isolate the standby server from other servers
Corosync traffic was blocked on the standby server.The server was marked offline and PostgreSQL service was turned off due to quorum policy.There was no disruption in the writer application.
repmgrd went into degrade monitoring mode.The PostgreSQL process was still running on the standby node.Manual intervention was required after the network isolation was corrected.
DCS communication was blocked for the standby node.The PostgreSQL service was running, however, the node was not considered for elections.There was no disruption in the writer application.