FetchS3Object throwing com.amazonaws.SdkClientException

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

FetchS3Object throwing com.amazonaws.SdkClientException

Wyllys Ingersoll
Using Nifi 1.11.4, the FetchS3Object processor occasionally throws the following error:

routing to failure: com.amazonaws.SdkClientException: Unable to execute HTTP request: The target server failed to respond. 
The full stack trace looks like:

com.amazonaws.SdkClientException: Unable to execute HTTP request: The target server failed to respond

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1175)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1121)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)

at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)

at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)

at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)

at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4926)

at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4872)

at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1472)

at org.apache.nifi.processors.aws.s3.FetchS3Object.onTrigger(FetchS3Object.java:159)

at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)

at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)

at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)

at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)

at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond

at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)

at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)

at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)

at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)

at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)

at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)

at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doReceiveResponse(SdkHttpRequestExecutor.java:82)

at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)

at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)

at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)

at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)

at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1297)

at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)

... 23 common frames omitted




The timeout period is plenty long enough and the server is definitely available.  Not sure what would be causing the problem but there were some bugs in the Apache HTTPClient module that are referenced in some forums discussing a similar issue - https://issues.apache.org/jira/browse/HTTPCLIENT-1610

Is this a known problem in the Nifi processor, perhaps using a buggy HTTP Client library? It only happens occasionally, usually when the processor is under heavy load. Looking for suggestions/workarounds.

Thanks
   Wyllys Ingersoll