Sudhakar Rayavaram
Problem solver (And maker),
Inquisitive (Root to most of my problems),
Software craftsman (Don't ask for estimates)

Works at TarkaLabs

Tech guy behind

22 Nov 2016
CSV export of Ecto models

This post is about converting Ecto model to a csv using this CSV library

Ecto is a database DSL for Elixir and is part of the Phoenix framework. Recently we had a requirement to export one of the tables as CSV (you can never escape such export requirement as long as there is excel right?)

Here is an ecto model we wanted to export

 1   defmodule Message do
 2     use MyApp.Web, :model
 4     schema "messages" do
 5       field :message, :string
 6       field :status, :string
 7       field :uuid, Ecto.UUID
 8       timestamps()
 9     end
10   end

It is a pretty simple model backed by table called "messages" which has the columns message, status and uuid

Get the data to be exported from the database

1 ...
2 messages = Message order_by(:inserted_at) |> Repo.all
3 ...

And trying to convert it to CSV

1 messages
2   |> CSV.encode
3   |> Enum.to_list

does not work because the CSV encoder does not know how to deal with the Message type. So, define an encoder function for the Message type

 1 defmodule Message do
 2   ...
 3   defimpl CSV.Encode, for: Message do
 4     def encode(cm, env \\ []) do
 5       [cm.message, cm.status, cm.uuid]
 6         |> -> CSV.Encode.encode(v, env) end)
 7         |> Enum.join(",")
 8     end
 9   end  
10 end

We are creating an array of all the needed column values and encoding them individually. Then, joining them to form the CSV for the given message struct

But this is not enough. CSV.encode/2 expects a stream of data in a tabular format, and encodes it to RFC 4180 compliant CSV lines. By that it means the data should be in the format

1 [
2   ["row1-col1-data","row1-col2-data","row1-col3-data"],
3   ["row2-col1-data","row2-col2-data","row2-col3-data"]
4 ]

So, wrap the array of Ecto Message objects into an array of array having a message object per row

1 [
2   [message1],
3   [message2]  
4 ]

Do that and pass it to the csv encoder

1 ...
2 messages = Message order_by(:inserted_at) |> Repo.all
3   |> -> [m] end)
4 messages
5   |> CSV.encode
6   |> Enum.to_list
7 ...

The resulting string will be the CSV of Ecto message instances