{"id":19326,"date":"2024-10-29T16:23:27","date_gmt":"2024-10-29T10:53:27","guid":{"rendered":"https:\/\/opstree.com\/blog\/?p=19326"},"modified":"2024-10-29T20:11:52","modified_gmt":"2024-10-29T14:41:52","slug":"setup-cross-cluster-replication-for-data-migration-in-elasticsearch","status":"publish","type":"post","link":"https:\/\/opstree.com\/blog\/2024\/10\/29\/setup-cross-cluster-replication-for-data-migration-in-elasticsearch\/","title":{"rendered":"Setup Cross Cluster Replication for Data migration in Elasticsearch"},"content":{"rendered":"

\u00a0<\/h1>\r\n

\"Elasticsearch\" Cross Cluster Replication (CCR) is a powerful feature in Elasticsearch that facilitates seamless data migration and disaster recovery by allowing real-time replication of data from a remote source cluster to a target cluster. In this guide, we’ll cover setting up CCR to migrate data from a Remote Cluster<\/em> in Singapore to a Local Cluster<\/em> in Mumbai. This step-by-step process involves configuring both clusters, creating follower indices, and monitoring the replication status. By the end, you\u2019ll have a reliable setup for cross-regional data replication, enhancing both data redundancy and accessibility across geographical locations.<\/p>\r\n

\"CCR\"<\/p>\r\n

Pre-requisites<\/h2>\r\n
    \r\n
  1. \r\n
      \r\n
    1. Should have \u201c<\/span>manage<\/b>\u201d cluster privilege on the local cluster\u00a0<\/span><\/li>\r\n
    2. A license on both the cluster that includes cross-cluster replication.\u00a0<\/span><\/li>\r\n
    3. In local cluster, all <\/span>master<\/i><\/b> nodes should also have a <\/span>remote-cluster-client <\/i><\/b>role. It must also have one <\/span>data<\/i><\/b> role and <\/span>remote-cluster-client <\/i><\/b>role.<\/span><\/li>\r\n<\/ol>\r\n<\/li>\r\n<\/ol>\r\n

      Steps:<\/h2>\r\n
        \r\n
      1. \r\n
          \r\n
        1. Establish trust between your clusters. This can be done in two ways: API key or TLS certificate authentication.<\/span><\/li>\r\n
        2. To setup API key authentication you may want to <\/span>Enable the remote cluster server on every node of the remote cluster. In elasticsearch.yml s<\/strong><\/span>pecify the domain of your remote cluster (singapore)\"Cross<\/li>\r\n
        3. Generate a certificate authority (CA) and a wildcard server certificate\/key pair that would work on all nodes of your cluster. Mention the .p12 <\/b>certification in your elasticsearch.yml.\"Cross<\/li>\r\n
        4. Make sure to add the SSL keystore password to the Elasticsearch keystore to keep your password secure.\u00a0<\/span><\/li>\r\n
        5. Restart the cluster.\u00a0<\/span><\/li>\r\n
        6. For additional instructions, please refer to this document: <\/span>API key authentication<\/span><\/a><\/li>\r\n
        7. You can check your cluster connection from <\/span>Stack Management > Data > Remote Clusters<\/b>\"Cross<\/li>\r\n
        8. Configure privileges by creating separate roles on the local and remote clusters, and then create a user with the required roles.<\/li>\r\n
        9. Finally create a follower index in your local cluster to replicate data from a selected index in the remote cluster.<\/span>\"Cross<\/li>\r\n
        10. Elasticsearch initializes the follower by transferring Lucene segment files from the leader index to follower, setting the index status to Paused, then switches to Active once replication is done.\"Cross<\/li>\r\n
        11. Even if you add more data to the leader index, the follower will continue to replicate these updates as well.<\/span><\/li>\r\n<\/ol>\r\n<\/li>\r\n<\/ol>\r\n

          Additional Considerations for CCR<\/h2>\r\n