Downloading Large Files with Finch in Elixir

In this TIL, we will explore how to download large files using the Finch HTTP client library in Elixir. Finch is a highly efficient, concurrent HTTP client that utilizes the powerful concurrency features provided by Elixir.

The following code demonstrates how to download a large zip file using Finch:

path = "/tmp/big-file.zip"
file = File.open!(path, [:write, :exclusive])
url = "https://domain.com/url/big-file.zip"
request = Finch.build(:get, url)

Finch.stream(request, App.Finch, nil, fn
  {:status, status}, _ ->
    IO.inspect("Download assets status: #{status}")

  {:headers, headers}, _ ->
    IO.inspect("Download assets headers: #{inspect(headers)}")

  {:data, data}, _ ->
    IO.binwrite(file, data)
end)

File.close(file)

This code snippet downloads a large zip file, storing it in the /tmp directory. We first build the Finch request using Finch.build/2, and then stream the response by calling Finch.stream/4. By streaming the response, we can efficiently handle large files without consuming excessive memory. The data chunks are written directly to the file using IO.binwrite/2 as they are received, making it an ideal approach for downloading large files.

Stay up to date