How to rename or replace an Elasticsearch index with zero downtime
Recently I worked on a project where I had to replace an ES index using a new dataset every day. The data was coming from a public repository and the requirement was to simply remove the existing index and create a new index using the updated dataset.
However, the problem was that the indexing process took around 45 minutes and if I’d simply dropped the existing index, the service that’s dependent on the index would be unavailable during that time.
One possible solution for this would have been to index the new data into a new index called index-new, delete the old one and rename the index-new to its expected name. However, ES currently doesn’t offer a renaming capability.
Solution
Let’s get down to the solution then. The solution involves using an alias. For those of you who are unfamiliar with aliases,
Aliases are secondary names for a group of data streams or indices.
We can substitute an alias for a lengthy index name or a group of indices which share a common set of data and requires querying as a whole( such as a group of monthly data indices).
So, first we need to add our existing index to a new alias and use the service that access this index to use the alias to query the data instead of directly using the index name.
After that, we can index the new dataset using a new index name. The best approach here would be to use current date or timestamp as a suffix (product-index-2021-07-18).
Once the new dataset is successfully indexed, we would have two indices on our hand. Now the best part,
we can add the new index and remove the old index from the alias in a single command. ES guarantees that this operation would run atomically and there won’t be a situation where the service trying to query a non-existing index. This is what the official docs has to say
And this is what the command looks like,
The takeaway is that, just before the above command, the service will be querying the old index and just after the command, it will be querying the new index. However, no logic need to be placed in the service to configure that.
Hope this short article would be quite useful if you also find yourself in the same predicament.
