Errors in reading text file from S3 via AWS Java SDK

Question

Im trying to read a text file from AWS S3 via JAVA SDK v2 and send it back to a client via HTTP (using com.sun.net.httpserver.HttpServer). I want to read the contents as string. But my simple code below doesn't work.

What is the problem? How to fix it?

import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
//...
Region region = Region.US_WEST_2;
String bucketName = "file-store";

S3Client s3Client = S3Client.builder().region(region).build();
//...
class GetFileHandlerV2 implements HttpHandler {
@Override
public void handle(HttpExchange he) throws IOException {
GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                                         .bucket(bucketName)
                                         .key(id + "/files/" + id + ".txt")
                                         .build();
OutputStream os = he.getResponseBody();
s3Client.getObject(getObjectRequest, ResponseTransformer.toOutputStream(os));
os.close();
//...
}
}

Here are the errors:

java.io.IOException: response headers not sent yet
        at sun.net.httpserver.PlaceholderOutputStream.checkWrap(ExchangeImpl.java:433) ~[jdk.httpserver:?]
        at sun.net.httpserver.PlaceholderOutputStream.write(ExchangeImpl.java:448) ~[jdk.httpserver:?]
        at java.io.InputStream.transferTo(InputStream.java:772) ~[?:?]
        at comcast.labs.objectstore.FileRetriever$GetFileHandlerV2.handle(FileRetriever.java:112) [FileRetriever-1.0.jar:?]
        at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) [jdk.httpserver:?]
        at sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:82) [jdk.httpserver:?]
        at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:80) [jdk.httpserver:?]
        at sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:692) [jdk.httpserver:?]
        at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) [jdk.httpserver:?]
        at sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:664) [jdk.httpserver:?]
        at sun.net.httpserver.ServerImpl$DefaultExecutor.execute(ServerImpl.java:159) [jdk.httpserver:?]
        at sun.net.httpserver.ServerImpl$Dispatcher.handle(ServerImpl.java:442) [jdk.httpserver:?]
        at sun.net.httpserver.ServerImpl$Dispatcher.run(ServerImpl.java:408) [jdk.httpserver:?]
        at java.lang.Thread.run(Thread.java:835) [?:?]

Answer 1

Please, consider reviewing the HttpExchange documentation. It provides the typical sequence of steps in the life-cycle of a HttpExchange.

Specifically, it indicates that before writing to the body of the response, using the OutputStream returned by getResponseBody, the method sendResponseHeaders must be invoked in order to actually start sending information to the client. From the javadoc:

Starts sending the response back to the client using the current set of response headers and the numeric response code as specified in this method....

Following your example, please, try something like this:

@Override
public void handle(HttpExchange he) throws IOException {
  GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                                         .bucket(bucketName)
                                         .key(id + "/files/" + id + ".txt")
                                         .build();

  // Optional, and according to your file extension
  he.getResponseHeaders().set("Content-type", "text/plain");

  // Set the `responseLength` to zero, from the docs: chunked transfer encoding
  // will be used and an arbitrary amount of data may be sent.
  he.sendResponseHeaders(200, 0);

  // The rest of your code
  OutputStream os = he.getResponseBody();
  s3Client.getObject(getObjectRequest, ResponseTransformer.toOutputStream(os));
  os.close();
  //...
}

You can read the whole information returned by S3Client first and then wrap it on your handler as well:

@Override
public void handle(HttpExchange he) throws IOException {
  // Fist, download actual object from S3 in the way you consider appropriate
  // This fragment of code could be refactored and be defined in its own
  // class/method
  GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                                         .bucket(bucketName)
                                         .key(id + "/files/" + id + ".txt")
                                         .build();
  ByteArrayOutputStream bos = new ByteArrayOutputStream();
  s3Client.getObject(getObjectRequest, ResponseTransformer.toOutputStream(bos));
  
  byte[] bytes = bos.toByteArray();

  // Perform the actual exchange

  // Optional, and according to your file extension
  he.getResponseHeaders().set("Content-type", "text/plain");

  he.sendResponseHeaders(200, bytes.length);
  // Please, perform the optimizations (buffering, etcetera) that you
  // consider necessary when writing the information to the user
  he.getResponseBody().write(bytes);
  he.close();
}

This second approach will allow you to refactor your code if required, differentiating the logic that has to do with the interaction with S3 from the one related to your HTTP wrapping code.