Saturday, December 7, 2013

Are connectTimeout and readTimeout enough for a good night's sleep?

    When it comes to communication with external resources we need to be sure that reasonable timeouts were set. Network is like a black hole - we send a request and many bad things can happen. Without timeouts our thread pool can saturate because of threads stuck deep inside socket read method. It is usual to set connect timeout and read timeout. But is it enough? Can we have a good night's sleep when connect and read timeouts are set in our application? Let's check!

Let's assume that we use a well known URLConnection from JDK in the following way:
URL url = new URL("http://localhost:8080/timeout/SlowServlet");
URLConnection connection = url.openConnection();
connection.setConnectTimeout(2000);
connection.setReadTimeout(5000);
try (BufferedReader in = new BufferedReader(
                         new InputStreamReader(connection.getInputStream()))) {
  String inputLine;
  while ((inputLine = in.readLine()) != null) {
    System.out.println(inputLine);         
  }
} catch (Exception e) {
  e.printStackTrace();
}
Now, I'm going to simulate some poor connection quality - every 2 seconds a chunk of data (a couple of bytes) is sent from a server to a client:
public class SlowServlet extends HttpServlet {
  protected void doGet(HttpServletRequest request, HttpServletResponse response) 
                       throws ServletException, IOException {
    response.setContentType("text/html");
    ServletOutputStream output = response.getOutputStream();
    for (int i = 0; i < 5; i++) {
      output.print("Chunk!");
      output.flush();
      simulateSlowConnection();
    }
    output.close();
  }
  private void simulateSlowConnection() {
    try {
      TimeUnit.SECONDS.sleep(2);
    } catch (InterruptedException e) {
      e.printStackTrace();
    }
  }
Is is easy to calculate that the whole response gathering will take about 10 seconds (5 chunks of data and 2 seconds delay between each chunk). However, our read timeout is 5 seconds. So after 5 seconds we should receive Timeout exception, right? Unfortunately not. Our reading thread will consume the whole response within about 10 seconds (TimeoutException should have been thrown after about 5 seconds!):
Chunk!Chunk!Chunk!Chunk!Chunk!
Without flush() invocation that exception is thrown:
java.net.SocketTimeoutException: Read timed out
 at java.net.SocketInputStream.socketRead0(Native Method)
If the response is slow we are not safe. If data chunks are sent in intervals shorter than readTimeout we can end up with extremely long response consumption and ... we can be awoken in the middle of the night because of thread pool saturation. This is a real nightmare. What can be done to overcome slow response? Let's try something like this:
final URL url = new URL("http://localhost:8080/timeout/SlowServlet");
final HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setConnectTimeout(2000);
connection.setReadTimeout(5000);
final ScheduledExecutorService executor = Executors.newScheduledThreadPool(2);
final Future<Void> handler = executor.submit(new Callable<Void>() {
  public Void call() throws Exception {
    try (BufferedReader in = new BufferedReader(
        new InputStreamReader(connection.getInputStream()))) {
      String inputLine;
      while ((inputLine = in.readLine()) != null) {
        System.out.println(inputLine);
      }
    } catch (Exception e) {
      e.printStackTrace();
    }
    return null;
  }
});
  
executor.schedule(new Runnable() {
  public void run() {
    connection.disconnect();
    handler.cancel(true);
    executor.shutdownNow(); 
  }
}, 5000, TimeUnit.MILLISECONDS);
handler.get();
executor.shutdownNow();
After 5 seconds disconnect(), cancel(true) and shutdownNow() methods will be invoked. The most important is disconnect() method, because cancel(true) attempts to cancel execution of a task - it just invokes interrupt() method on a thread. Unfortunately, it would give us nothing without disconnect().
If you prefer Apache HttpClient, something similar can be created:
final HttpClient client = new DefaultHttpClient();
client.getParams().setParameter("http.connection.timeout", 2000);
client.getParams().setParameter("http.socket.timeout", 3000); 
final HttpGet request = new HttpGet("http://localhost:8080/timeout/SlowServlet");
final ScheduledExecutorService executor = Executors.newScheduledThreadPool(2);
final Future handler = executor.submit(new Callable() {
  public Void call() throws Exception {
    HttpResponse response = client.execute(request);
    try (BufferedReader in = new BufferedReader(
        new InputStreamReader(response.getEntity().getContent()))) {
      String inputLine;
      while ((inputLine = in.readLine()) != null) {
        System.out.println(inputLine);
      }
    } catch (Exception e) {
      e.printStackTrace();
    }
    return null;
  }
});

executor.schedule(new Runnable() {
  public void run() {
    request.abort();     
    handler.cancel(true);
    executor.shutdownNow();
}}, 5000, TimeUnit.MILLISECONDS);
handler.get();
executor.shutdownNow();

To sum up, we ended up with three timeouts:
  • connection timeout,
  • read timeout,
  • request timeout (I like this name).
As you can see, slow response can be tricky to handle.

2 comments :

  1. Wondered why you used a scheduled thread for your main work of reading the web page? If you do your "abort" task first, can't you simply read from the main thread?
    Also, are you sure you need this if you use HttpClient - I believe I had to resort to this method when I used the URLConnection (Sun), but HttpClient did timeout properly.

    ReplyDelete
    Replies
    1. Hi mate! I use ServiceExecutor for doing the main job - the execution is not deferred - I use submit(...) instead of schedule(...) method [which in fact is: schedule(task, 0, TimeUnit.NANOSECONDS)]. I believe it is a little bit more safe to hit external dependency not in the main processing thread.

      HttpClient without abort() is stuck (http.connection.timeout and http.socket.timeout are not enough) - I retested it a few minutes ago.

      Cheers!

      Delete