elixir snippets

Web Microframework Benchmark

Tagged elixir, clojure, golang, benchmark, ruby  Languages 

Results on an old MacBook Air:

Ruby 2.1.0 + Hobbit + Hat + puma (-t 8 -w 2) - ~100-150 req/s (Hat = Hobbit app template with i18n, asset pipeline, etc)
Ruby 2.1.0 + Hobbit + Hat custom + puma (-t 8 -w 2) - ~1500 req/s (Hat without asset pipeline)
Ruby 2.1.0 + Hobbit + puma (-t 8 -w 2) - ~1600 req/s
Ruby 2.1.0 + rack + puma (-t 8 -w 2) - ~1600 req/s
Golang 1.3.1 + net/http - ~2700 req/s
Elixir 1.0.0 + Phoenix 0.4.1 - ~1300 req/s
Clojure 1.6.0 + ring 1.3.1 - ~5000 req/s
Clojure 1.6.0 + ring 1.3.1 + slim - ~270 req/s

YMMV.

How to implement tagging with Ecto and Elixir

Tagged ecto, elixir, tagging, tags  Languages elixir
defmodule Snippets.Tagging do
  import Ecto.Query
  alias Ecto.Changeset
  alias Snippets.Repo
  @doc """
  Tags a model.

  ## Model

    defmodule Snippets.Tag do
      import Ecto.Query
      use Ecto.Schema

      schema "tags" do
        field :name
        many_to_many :snippets, Snippets.Snippet, join_through: "snippet_tags"
      end
    end

    defmodule Snippets.Snippet do
      import Ecto.Query
      use Ecto.Schema
      schema "snippets" do
        ...
        field :tag_list, :string, virtual: true
      end
    end

  ## Schema

    create table(:tags) do
      add :name, :text, null: false
    end
    create index(:tags, ["lower(name)"], unique: true)

    create table(:snippet_tags) do
      add :tag_id, references(:tags)
      add :snippet_id, references(:snippets)
    end
    create index(:snippet_tags, [:tag_id, :snippet_id], unique: true)

  ## Code

    snippet
    |> Changeset.cast(conn.params, @required_params, @optional_params)
    |> Tagging.changeset(Snippets.Tag, :tags, :tag_list)
    # Supports multiple types of tags
    |> Tagging.changeset(Snippets.Language, :languages, :language_list)

  """
  def changeset(changeset, model, association, tag_list_attr) do
    # Parse tags into Enum
    tag_list = changeset.changes
                |> Map.get(tag_list_attr, "")
                |> String.split(",")
                |> Enum.map(&(String.strip(&1)))
                |> Enum.reject(fn(name) -> name == "" end)
                |> Enum.map(&(String.downcase(&1)))
                |> Enum.uniq
                |> Enum.sort
    # Find existing tags
    existing_tags = from(t in model, where: t.name in ^tag_list) |> Repo.all
    # Create or find all tags
    tags = Enum.map(tag_list, fn(name) ->
      # Initialize new tag. Equivalent to:
      # new_tag = %Snippets.Tag{name: name}
      new_tag = struct(model, name: name)
      tag = Enum.find(existing_tags, new_tag, fn(existing_tag) ->
        existing_tag.name == name
      end)
    end)
    tag_changeset = Enum.map(tags, &Ecto.Changeset.change/1)
    # Add tags to changeset
    changeset |> Changeset.put_assoc(association, tag_changeset)
  end
end

Fixing "enif_send: env==NULL on non-SMP VMAborted"

Tagged elixir, erlang, smp  Languages bash, erlang

I received the following error while testing an Elixir application on a Debian VPS having only 1 CPU:

enif_send: env==NULL on non-SMP VMAborted

Usually this means your Erlang VM was compiled without SMP support.

In my case, the issue was caused by the VPS having only 1 CPU, as explained in the documentation for erl:

$ man erl
...
-smp [enable|auto|disable]

    -smp enable and -smp starts the Erlang runtime system with SMP support enabled. This may fail if no runtime system with SMP support is available.
    -smp auto starts the Erlang runtime system with SMP support enabled if it is available and more than one logical processor are detected.
    -smp disable starts a runtime system without SMP support.
...

Also see http://erlang.org/doc/man/erl_nif.html#enif_send

I fixed the issue by enabling SMP-mode using the —erl switch:

MIX_ENV=prod iex --erl "-smp enable" -S mix

Next, I tried to fix the issue by using the ELIXIR_ERL_OPTIONS environment variable:

ELIXIR_ERL_OPTIONS="-smp enable"

This works fine when running the application with mix. However, with exrm this option does not work.

exrm

If you’re deploying your app to production using exrm then you have to add the switch to running-config/vm.args after each deploy, or in rel/vm.args for a permanent solution:

# Enable SMP on 1 CPU machines
-smp enable

See this issue on Github for more information about why you can’t use ELIXIR_ERL_OPTIONS=”-smp enable”: https://github.com/bitwalker/exrm/issues/90

Also see the exrm documentation for more details: https://exrm.readme.io/docs/release-configuration

Logging to a file with Elixir

Tagged elixir, file, logging  Languages elixir

mix.exs:

  def application do
    [
      applications: [:logger, :logger_file_backend, ...],
      mod: {Snippets, []}
    ]
  end

  def deps do
    [
      {:cowboy, "~> 1.0.0"}, # web server
      {:logger_file_backend, "~> 0.0.7"} # writes log messages to a file, LOL
      ....
    ]
   end

config/dev.exs:

config :logger, :error_log,
  path: "log/app.log",
  level: :debug

Pagination with Elixir and Ecto

Tagged ecto, elixir, pagination  Languages elixir
defmodule Pagination do
  import Ecto.Query
  alias Snippets.Repo
  #
  # ## Example
  #
  #    Snippets.Snippet
  #    |> order_by(desc: :inserted_at)
  #    |> Pagination.page(page: 0, per_page: 10)
  #
  def page(query, page: page, per_page: per_page) do
    count = per_page + 1
    result = query
              |> limit(^count)
              |> offset(^(page*per_page))
              |> Repo.all
    %{ has_next?: (length(result) == count),
       has_prev?: page > 0,
       list: Enum.slice(result, 0, count-1) }
  end
end

Elixir Deployment

Tagged deploy, deployment, elixir, exrm, rollback, upgrade, distillery  Languages elixir, bash

Setup

Development environment:

  1. Sign up for a Gitlab account, or use your favorite git server
  2. Create a (private) repository
  3. Create a deploy key that allows read-only access to your private repository (optional, if public)
  4. Commit code to the repository

Pre-requisites

You will need Elixir:

$ sudo apt-get install elixir

Code

Add distillery to the list of dependencies:

def deps do
   ...
   {:distillery, "~> 0.9.9"},
   ...
 end

Run mix release.init and follow the instructions:

$ mix release.init

An example config file has been placed in rel/config.exs, review it,
make edits as needed/desired, and then run `mix release` to build the release

Next, run mix release:

You can run it in one of the following ways:
  Interactive: rel/xxx/bin/xxx console
  Foreground: rel/xxx/bin/xxx foreground
  Daemon: rel/xxx/bin/xxx start

Deployment

Production environment:

  1. Clone project
$ cd /var/www
$ git clone git@gitlab.com:christianhellsten/aktagon-snippets.git
  1. Create a release
$ MIX_ENV=prod mix do deps.get, clean, compile
$ MIX_ENV=prod mix release --verbosity=verbose
  1. Create a monit start script
#
# Run "ps -ef | grep beam, if you get status "Does not exist" from "monit status"
#
check process snippets MATCHING "rel/snippets"
    start program = "/bin/su - deploy -c '/var/www/aktagon-snippets/rel/snippets/bin/snippets start'"
    stop program = "/bin/su - deploy -c '/var/www/aktagon-snippets/rel/snippets/bin/snippets stop'"
  1. Start server
$ sudo monit start snippets

Deploying a new version

First increment the version number. Then perform the hot upgrade:

$ git pull
$ export MIX_ENV=prod
$ mix do compile
$ mix release --upgrade 

$ rel/snippets/bin/snippets upgrade "0.0.2"

Rolling back to a previous version

How to perform hot downgrades:

$ rel/snippets/bin/snippets downgrade "0.0.1"

See https://hexdocs.pm/distillery/getting-started.html

Phoenix

See Using Distillery With Phoenix.

Alternatives

Annotated Plug and Elixir Example

Tagged elixir, erlang, genserver, otp, plug, cowboy  Languages bash, elixir

This is an annotated example of a web application written using Elixir and Plug.

Create mix.exs:

#
# Mix is a build tool that provides tasks for creating, compiling, and testing
# Elixir projects, managing its dependencies, and more.
# https://github.com/hexpm/hex_web/blob/master/mix.exs
#
defmodule PlugExample.Mixfile do
  # In order to configure Mix, a developer needs to use Mix.Project in a module
  # and define a function named project that returns a keyword list with
  # configuration.
  # http://elixir-lang.org/docs/stable/mix/Mix.Project.html
  use Mix.Project

  # Configure project.
  def project do
    [app: :Router,
      version: "0.0.1",
      elixir: "~> 1.2",
      deps: deps]
  end

  #
  # Returns applications that should be started.
  #
  # In OTP, application denotes a component implementing some specific
  # functionality, that can be started and stopped as a unit, and which can be
  # re-used in other systems as well.
  # http://erlang.org/doc/man/application.html
  #
  def application do
    [
      applications: [:logger, :plug], # 3rd-party applications
      mod: {PlugExample.Supervisor, []} # own application
    ]
  end

  #
  # Returns project dependencies:
  # https://hex.pm/
  #
  def deps do
    [
      {:cowboy, "~> 1.0.0"}, # Erlang web server
      {:plug, "~> 1.0"} # Elixir's rack (Ruby), wsgi (Python), servlet (Java) API
    ]
  end
end

Create lib/router.exs

defmodule PlugExample.Router do
  #
  # Plug ships with a router that allows developers to quickly match on
  # incoming requests and perform some action:
  # https://hexdocs.pm/plug/readme.html#the-plug-router
  #
  use Plug.Router
  #
  # This module defines a Plug.Conn struct and the main functions for working with Plug connections.
  # https://hexdocs.pm/plug/Plug.Conn.html
  #
  import Plug.Conn
  # Logs request to STDOUT
  plug Plug.Logger
  # Matches request to a route
  plug :match
  # Dispatches request to a route
  plug :dispatch

  #
  # Start a GenServer, i.e., a server instance:
  #
  # http://elixir-lang.org/getting-started/mix-otp/genserver.html
  # https://blog.drewolson.org/understanding-gen-server/
  #
  def start_link do
    # Cowboy adapter:
    # https://hexdocs.pm/plug/Plug.Adapters.Cowboy.html
    {:ok, _} = Plug.Adapters.Cowboy.http PlugExample.Router, []
  end

  #
  # HTTP GET /
  #
  # Use conn.private to access route options:
  #   conn.private[:protected]
  #
  get "/", private: %{protected: false} do
    conn |> send_resp(200, "Hello world")
  end

  get "/favicon.ico" do
    conn |> send_resp(200, "LOL")
  end
  
  # Forward /about to another router's get "/" route
  # forward "/about", to: PlugExample.About

  #
  # A catch-all route
  #
  match _ do
    conn |> send_resp(404, "Not found")
  end
end

Create lib/supervisor.ex

#
# A supervisor is a process which supervises other processes, called child
# processes. Supervisors are used to build a hierarchical process structure
# called a supervision tree, a nice way to structure fault-tolerant applications.
#
# http://elixir-lang.org/docs/stable/elixir/Supervisor.html
#
defmodule PlugExample.Supervisor do
  #
  # In Elixir (actually, in Erlang/OTP), an application is a component
  # implementing some specific functionality, that can be started and stopped as a
  # unit, and which can be re-used in other systems.
  # http://elixir-lang.org/docs/stable/elixir/Application.html
  # http://erlang.org/doc/man/application.html
  # http://erlang.org/doc/design_principles/applications.html
  #
  use Application

  #
  # The type argument passed to start/2 is usually :normal unless in a
  # distributed setup where application takeovers and failovers are configured.
  #
  # start/2 typically returns {:ok, pid} or {:ok, pid, state} where pid
  # identifies the supervision tree and state is the application state.
  #
  def start(_type, _args) do
    #
    # Convenience functions for defining a supervision specification.
    # http://elixir-lang.org/docs/stable/elixir/Supervisor.Spec.html
    #
    import Supervisor.Spec, warn: false
    # A list of children (workers or supervisors) to supervise
    children = [
      worker(PlugExample.Router, [])
    ]
    #
    # If a child process terminates, only that process is restarted:
    # http://elixir-lang.org/docs/stable/elixir/Supervisor.html
    #
    opts = [strategy: :one_for_one, name: PlugExample.Supervisor]
    #
    # Start supervisor by passing the children to start_link/2.
    # You may want to use a module-based supervisor.
    Supervisor.start_link(children, opts)
  end
end

Start the server:

$ mix deps.get
$ iex -S mix

Continue reading: How to deploy an Elixir web application.

How to send emails with Mailgun and Elixir

Tagged httpotion, mailgun, elixir  Languages elixir

This example shows how to send emails with Mailgun and Elixir:

url = "https://api.mailgun.net/v3/xxx.mailgun.org/messages"
headers = [
  "User-Agent": "My App",
  "Content-Type": "application/x-www-form-urlencoded"
]
params = %{
  "from" => "christian@xxx.mailgun.org",
  "to" => "christian@aktagon.com",
  "subject" => "Hello Christian",
  "text" => "Hello"
}
body = URI.encode_query(params)
auth = { "api", "key-xxx" }
response = HTTPotion.post url, [body: body, headers: headers, basic_auth: auth]
if !HTTPotion.Response.success?(response) do
  throw(response.body)
end
Poison.decode!(response.body)

Replace the API key and mailgun URL with your own API key and URL.

Dependencies

How to fix "the response was already sent" error in Elixir

Tagged phoenix, plug, elixir  Languages elixir

If you get an “(Plug.Conn.AlreadySentError) the response was already sent” error:

Server: localhost:4000 (http)
Request: POST /api/v1/xxx
** (exit) an exception was raised:
    ** (Plug.Conn.AlreadySentError) the response was already sent
        (plug) lib/plug/conn.ex:332: Plug.Conn.put_status/2

Verify that you have not defined the action plug more than once:

plug :action

Multi-tenancy in Ecto and Phoenix

Tagged elixir, multi-tenancy, postgres, phoenix, ecto  Languages bash, elixir

Creating a new tentant

A new tenant requires a namespace, which is a schema in Postgres, and a prefix in Ecto:

$ psql -U postgres database_x
> create schema aktagon; 

Querying data

import Ecto.Query
email = "christian@aktagon.com"
q = from(m in User, where: m.email == ^email)
Repo.all(%{q | prefix: "aktagon"})

Documentation: https://hexdocs.pm/ecto/Ecto.Query.html#module-query-prefix

Inserting data

Repo.insert(
  Ecto.put_meta(
   %User{ email: "christian@aktagon.com" },
   prefix: "aktagon"
  )
)

Migrations

$ mix ecto.migrate --prefix "aktagon"

Notes

  • (KeyError) key :__meta__ not found

I got this error when passing a changeset to Ecto.put_meta instead of a User struct.