We are using Azure Search .Net SDK(v3.0.4) to merge or upload documents to one of the azure search index.
We are using the below code for the same:
var batch = IndexBatch.MergeOrUpload(documentList);
await searchIndexClient.Documents.IndexAsync(batch);
We are indexing 2 types of subindexes using the same search client. When there is a high load, we are facing lots of task cancelled exceptions and the operation is failing without any hint to inner exceptions. Can someone help me here why this issue might be there?
Stacktrace for reference: System.Threading.Tasks.TaskCanceledException: A task was canceled. at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Rest.RetryDelegatingHandler.<>c__DisplayClass11_0.<b__1>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Rest.RetryDelegatingHandler.d__11.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Net.Http.HttpClient.d__58.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Search.DocumentsOperations.d__23.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Search.DocumentsOperationsExtensions.d__13`1.MoveNext() --- End of stack trace from previous location where exception was thrown ---
@Sonia-2755, Welcome to Microsoft Q&A! Thanks for posting a good question.
As Bruce mentioned on your
SO thread
, the issue could be that you're pushing the service beyond its limits and it's not able to keep up with the request rate. We will follow up with you offline.
Apologies for any inconvenience with this issue. Thanks for your co-operation!
Adding the solution that helped to benefit the community, from our offline discussions.
Issue could be due to starvation of TCP connection pool and adding the connection settings showed improvements. To resolve other exceptions while calling Search service (Task cancelled exceptions, Http Socket exceptions, IndexBatch exceptions, Null reference exceptions, Service too busy exceptions etc), additional modifications/fix were made as per the requirement:
Increase the service replicas.
Decrease load on Search service by reducing calls.
Adding delays.
