Chunked HTTP with Ajax and Scala

April 16, 2014

In Web programming, it is often handy to be able to process an HTTP request as a stream while sending back periodic status updates to the client. Consider an application that performs some time-consuming computation on incremental subsets of request data, or an application that handles the uploading of a large file.

Using Java servlets, we can write a simple server that reads a file from a multipart POST request, processes the file, and keeps the client updated with the current progress:

def doPost(req: HttpServletRequest, res: HttpServletResponse) {

  import java.io.File
  import java.io.FileOutputStream
  import java.util.UUID

  res.setContentType("plain/text")
  res.setCharacterEncoding("UTF-8")

  val in = req.getInputStream
  val length = req.getContentLength

  val cacheFile = new File("/tmp/chunky-" + UUID.randomUUID.toString)
  try {
    val out = new FileOutputStream(cacheFile)

    def percentOf(x: Int, y: Int): Int = (x.toDouble / y.toDouble * 100).toInt

    val buffer = Array.fill[Byte](1024)(0)

    def processChunk(skipped: Int): Unit =
      // read a chunk from the request
      in.read(buffer) match {
        case x if x <= 0 => ()
        case read =>
          val progress    = percentOf(read + skipped, length)
          val oldProgress = percentOf(skipped, length)

          // check for integer increase in % progress
          if (progress > oldProgress) {
            // write progress to the response
            res.getWriter.write(progress.toString + "%" + "\n")
            // force-write the response buffer to the client
            res.flushBuffer
          }

          out.write(buffer, 0, read)
          processChunk(skipped + read)
      }

    processChunk(0)
    out.close
  } finally {
    cacheFile.delete()
  }

}

We can test this service using curl with the --no-buffer option, observing the return of each incremental percentage point:

$ curl --no-buffer -X POST -F file=@test.bmp localhost:8080/upload
1%
2%
3%
4%
...

To interact with this service from JavaScript, we need an XMLHttpRequest:

var xhr = new XMLHttpRequest();

We also need to be able to do something with the output, so let's make a progress bar:

<style>
  div.progress { border: solid 1px; width: 360px; height: 14px; }
  div.bar      { background-color: #3cf; height: 14px; }
  div.label    { margin-top: 6px; float: right; }
</style>
<div class="progress">
  <div id="bar" class="bar" style="width: 0%">
    &nbsp;
  </div>
  <div id="label" class="label">
    &nbsp;
  </div>
</div>

Finally, we can make a request and watch for interesting XMLHttpRequest#upload events:

function upload() {
  var file = document.getElementById('file').files[0];
  var formData = new FormData();
  formData.append('file', file, file.name);

  var xhr = new XMLHttpRequest();

  function updateProgress (e) {
    if (e.lengthComputable) {
      var progress = Math.round(e.loaded / e.total * 100).toString() + '%';
      document.getElementById('bar').style.width = progress;
      document.getElementById('label').innerHTML = progress;
    }
  }

  function transferComplete(e) {
    document.getElementById('label').innerHTML = "Done!";
  }

  function transferFailed(e) {
    alert('Transfer failure.');
  }

  function transferCanceled(e) {
    alert('Transfer cancelled.');
  }

  xhr.upload.addEventListener("progress", updateProgress, false);
  xhr.upload.addEventListener("load", transferComplete, false);
  xhr.upload.addEventListener("error", transferFailed, false);
  xhr.upload.addEventListener("abort", transferCanceled, false);

  xhr.open("post", "/upload", true);
  xhr.send(formData);
}

Connecting this to an input of type file, we get something like this:

File upload progress
File upload progress

This article was inspired by Servlet/Javascript chunking on Stack Overflow.