Using the .NET 2.2.1 client, I’m seeing this error intermittently on AWS. I’ve experimented with different network configurations and I can’t seem to get past this issue. Where should I look first to troubleshoot this?
Server version is 4.0
I’ve enabled client logging and see the same error in there but didn’t notice anything else that seemed relevant.
I’ve looked at all of the server logs and couldn’t see any issues there either.
Our Connection Pool MinSize is 5 and MaxSize is 10. Previously it was 1/10 and when we increased to 5/10 no improvement was observed.
Increasing the instance size on AWS had no effect.
From what I understand, this doesn’t have anything to do with the server. It seems more like an issue with the client machine not being able to obtain a TCP connection. We’ve observed other possible network connectivity issues in our AWS VPC and are troubleshooting with AWS Support, but so far they haven’t found anything. “tcp ping” tests between the client and server on port 8091 show 0% packet loss, even during the same time period that the .NET client experiences this failure.
Any direction on troubleshooting this would be very appreciated.
I know there have been some instances where people have had to adjust the .NET runtime in number of sockets, but nothing at this low. What is the latency of ops? Also, maybe try a simple tool to see if it sees the same issue, something like
cbc pillowfight
which is in the libcouchbase distribution.
p.s.: This post was not categorized, so it probably didn’t get much notice. I’ve moved it to the .NET SDK category.
Sorry for the delay.
@ingenthr
- Thanks, I haven’t had a chance to check ops or test with “cbc pillowfight” yet, but if/when I do, I’ll report back here. Good to know about tagging, I’ll be sure to do that from now on. Thanks!
@dhalls
- Yes. I’m not sure if this was a good idea or not, but we changed the connection pool settings to maxSize=“30” minSize=“5” and the errors went away. Good luck solving your issue.
@nwood888
-
The error you described happens when the client exhausts it’s internal pool of TCP connections. This is usually an indication of either high-latency, very high OPS, and/or perhaps very high concurrency - all three can be related. The solution is to increase the pool size (MaxSize) until the error goes away, exactly as you did…it’s just a matter of tuning the client to your environment.
-Jeff