REST API calls sometimes return "Connection Failure"

[David]

Hello,

I've been sporadically encountering errors with my REST API calls to backend services. The error response includes:

errordetail: Unknown host: Read timed out
filecontent: Connection Failure
statuscode:  Connection Failure. Status code unavailable.

 

In the past month, the issue has occurred today, July 4, July 2, June 29, June 12. As far as I can tell, nothing has changed on my end. This failure was such a concern on/around April 22nd that I setup behavior to automatically retry the call when it fails. The retry often succeeds within three attempts. This issue by no means occurs on every call - most work fine.

The specific operation that is failing is GET / Users / with a MasterKey and Username filter. I would need to investigate further to determine if other operations are occasionally failing as well.

Any thoughts?

Thanks,

David

[Dimo Mitev]

Hi David,

 Based on the error response sent, it seems that by some reason the invoked requests doesn't reach to our servers at all.

 Would you please let us know from what devices you are calling our API and is it possible to lost the connectivity by some reason ? Do you have some additional information that may be helpful?

 Looking forward for your reply. Thank you in advance.

Regards,
Dimo
Telerik by Progress

 

 

[David]

Hi Doma,

Thanks for the quick response. We're calling from a dedicated Tomcat/Lucee(CFML) server on Amazon Elastic Beanstalk. I can't imagine there would be any connectivity issues. Unfortunately, I don't have any more insight into error logs right now. The most interesting part is that it seems to come in waves - I won't have an issue for a long period of time, then quite a few errors during a certain ~24-48 hour period of time - then the errors just go away. It's also by no means every call even during period where I have the issue. 

Not sure if that info helps any, just trying to nail down the cause of the problem.

 

Thanks,

David

[Martin]

Hi David,

The error you are experiencing signifies for a problem with reaching the address host or resolving the DNS.

There may be a lot of reasons for such error but for now I's suggest that you examine the framework settings of the server making the request :

- connection timeout settings
- the dumps of your server application
- ability to log more verbose errors
- how the requests are configured - are all attributes of the request present at the time when the request fails
- is there a difference when making the request over Http/Https?
- How many connections to the host are opened - Everlive will deny more than 50 connections per IP (though with an HTTP status code and response)

According to our server stats there are sporadically periods with a high number of requests. Is this error produced when your app receives a lot of requests to the Finagle API (which may be then addressing the Everlive API) ? Could you please check also with AWS if it is not a load issue on their side?

I look forward to your reply.

Regards,
Martin
Telerik by Progress

 

 

[David]

Hi Dimo,

Thanks for the great suggestions.

Surprisingly (or maybe not) by simply changing my connection timeout from 10 seconds to 30 seconds appears to have helped. I'm not entirely sure why this would be, since the actual api response takes around 200ms for me. I'm monitoring to see if at times that 200ms response randomly slows down to 10+ seconds.

Thanks,

David

[Martin]

Hi David,

Glad to hear that problem seems to be resolved.

Please notify us if you have further questions or considerations.

Regards,
Martin
Telerik by Progress