Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I would like to know if there is a way to specify to elastic search that I don't mind missing or erroneous indices on my search query. In other words I have a query which tries to query 7 different indices but one of them might be missing depending on the circumstances. What I want to know is that if there is a way to say, forget the broken one and get me the results of the other 6 indices?

    SearchRequestBuilder builder = elasticsearchClient.getClient().prepareSearch(indices)
            .setQuery(Query.buildQueryFrom(term1, term2))
            .addAggregation(AggregationBuilders.terms('term')
                                        .field('field')
                                        .shardSize(shardSize)
                                        .size(size)
                                        .minDocCount(minCount));
As an example query you can find the above one.
Take a look at the ignore_unavailable option, which is part of the multi index syntax. This has been available since at least version 1.3 and allows you to ignore missing or closed indexes when performing searches (among other multi index operations).
It is exposed in the Java API by IndicesOptions. Browsing through the source code, I found there is a setIndicesOptions() method on the SearchRequestBuilder used in the example. You need to pass it an instance of IndicesOptions.
There are various static factory methods on the IndicesOptions class for building an instance with your specific desired options. You would probably benefit from using the more convenient lenientExpandOpen() factory method (or the deprecated version, lenient(), depending on your version) which sets ignore_unavailable=true,allow_no_indices=true, and expand_wildcards=open.
Here is a modified version of the example query which should provide the behavior you are looking for:
SearchRequestBuilder builder = elasticsearchClient.getClient().prepareSearch(indices)
        .setQuery(Query.buildQueryFrom(term1, term2))
        .addAggregation(AggregationBuilders.terms('term')
                                    .field('field')
                                    .shardSize(shardSize)
                                    .size(size)
                                    .minDocCount(minCount))
        .setIndicesOptions(IndicesOptions.lenientExpandOpen());
Have you tried using Index Aliases?
Rather than referring to individual aliases you can specify a single index value. Behind this can be several indexes.
Here I'm adding two indexes to the alias and removing the missing / broken one:
curl -XPOST 'http://localhost:9200/_aliases' -d '
   "actions" : [
      { "remove" : { "index" : "bad-index", "alias" : "alias-index" } },
      { "add" : { "index" : "good-index1", "alias" : "alias-index" } },
      { "add" : { "index" : "good-index2", "alias" : "alias-index" } }
                Sorry Olly, could not exactly understand what you meant. I might use an alias to combine a query on 7 different indices but I am already able to do that. What I want to do is the ability to ignore one of those indices if it fails.
– ralzaul
                Dec 15, 2014 at 13:04
                An index wouldn't fail for no reason, you must have done something to break it (e.g. it contains old data). In which case you would identify and remove the failed index from the alias.
– Olly Cruickshank
                Dec 15, 2014 at 13:09
                no basically without any error/crash/misbehaviour my index might not get created. So you are telling me to create an alias and check the status of the individual indices in the alias to remove them or not?
– ralzaul
                Dec 15, 2014 at 13:26
                If your index doesn't get created you'll get an error message back from ES.  If you get an error message then don't add the index to the alias. I'm struggling to see where the problem is. If your indexes are randomly disappearing - you have bigger problems.
– Olly Cruickshank
                Dec 15, 2014 at 13:36
                that is a possible solution but not exactly what I was looking for. I was looking for more of an internal method in Java API to discard failing indices directly.
– ralzaul
                Dec 16, 2014 at 11:56
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.