How to use PostgreSQL advisory locks with SQLAlchemy and Python

Tagged python, advisory lock, pg_try_advisory_lock, sqlalchemy  Languages python

from models import Session
from sqlalchemy import func, select

def execute(session, lock_fn, lock_id, scope):
    Executes the lock function
    return session.execute(select([lock_fn(lock_id, scope)])).scalar()

def obtain_lock(session, lock_id, scope):
    Obtains the advisory lock
    lock_fn = func.pg_try_advisory_lock
    return execute(session, lock_fn, lock_id, scope)

def release_lock(session, lock_id, scope):
    Releases the advisory lock
    lock_fn = func.pg_advisory_unlock
    return execute(session, lock_fn, lock_id, scope)

def with_lock(my_func, lock_id, scope=1):
    Executes my_func if the lock can be obtained.
    session = Session()
    obtained_lock = False
        obtained_lock = obtain_lock(session, lock_id, scope)
        if obtained_lock:
        if obtained_lock:
            release_lock(session, lock_id, scope)


from advisory_lock import with_lock

def run():
    print("It runs")

if __name__ == '__main__':
    with_lock(run, 300000)

UFW + Docker = No firewall

Tagged docker, elasticsearch, gotcha, iptables, ufw, wtf  Languages 

TLDR: Docker can and will override existing iptable rules and expose your services to the Internet

This means you have to think twice when installing Docker on a machine that is only protected by an iptables-based firewall such as UFW. You might think you are protected by your firewall, but you very likely are not. This is probably one of the more common reasons why Elasticsearch servers, which are unprotected by default, are exposed to the internet.

For details, see:

Solution 1: External firewall

One solution is to use a firewall provided by the hosting provider (DO, AWS, GCP, etc.).

Solution 2: Disable Docker’s iptables “feature”

Disable iptables in Docker by adding the following switch:


Solution 2: Listen on private IPs

This is perhaps the easiest to implement and easiest to forget: expose your containers and services on one of the following private IP address ranges:

  • to
  • to
  • to

Note that binding to will not work with Docker Swarm.

Autocomplete / Typeahead

Tagged autocomplete, javascript, typeahead, bootstrap, jquery  Languages javascript, html

Javascript (jQuery):

function autocomplete(options) {
  var input = $(options.input);
  var resultsElem = $(options.results);
  var url = options.url;
  var cache = {};
  var result = null;
  var minQueryLength = options.minQueryLength;
  var debounce = null;
  var waitingFor = null; // Ajax responses might not arrive in order
  var show = function() {
    if(result == null) {
  var hide = function() {
  // Hide when clicking outside autocomplete elements
  // Stop propagation so that event does not reach the body handler 
  var renderResults = function(query, results) {
    if (query !== waitingFor) {
    if (results != undefined) {
    } else {
  var search = function() {
    var query = input.val();
    console.log("Searching for '" + query + "'")
    waitingFor = query;
    if (query in cache) {
      console.log('Using cache')
      renderResults(query, cache[query]);
    } else {
      console.log('Not using cache')
      if (query.length > minQueryLength) {
        $.ajax({url: url, data: {query: query}, type: 'GET'}).then(function(data) {
          result = data;
          cache[query] = result;
          renderResults(query, result);
      } else {
        renderResults(query, null);
  var searchWithDebounce = function(e) {
    debounce = setTimeout(search, 100);
  input.on('blur', hide);
  input.on('focus', show);
  input.on('keyup', searchWithDebounce);

  input: '#autoComplete',
  results: '#autocomplete-results',
  url: '/search',
  minQueryLength: 2

HTML (Bootstrap 4):

<form action="/search" class="form-inline ml-2 my-2 my-lg-0 ml-auto mr-auto" id="site-search" method="GET">
  <input aria-label="Search" autocomplete="off" class="autocomplete form-control mr-sm-2" id="autoComplete" name="query" placeholder="Search" tabindex="1" type="search">
  <div class="autocomplete shadow" id="autocomplete-results"></div>

SCP via an SSH jump host

Tagged jump host, rsync, scp, ssh  Languages bash

I prefer using rsync to scp when copying files via jump hosts, for example, to target-host via the intermediate host jump-host:

rsync -v -e 'ssh -A -t [email protected] ssh -A -t target-host' nasty.file :/tmp

rsync syntax is stable across versions and platforms while the syntax of SCP is different across platforms and versions.

man ssh:

     -A      Enables forwarding of the authentication agent connection.  This can also be specified on a per-host basis in a configuration

             Agent forwarding should be enabled with caution.  Users with the ability to bypass file permissions on the remote host (for the
             agent's UNIX-domain socket) can access the local agent through the forwarded connection.  An attacker cannot obtain key material
             from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into
             the agent.
     -t      Force pseudo-terminal allocation.  This can be used to execute arbitrary screen-based programs on a remote machine, which can be
             very useful, e.g. when implementing menu services.  Multiple -t options force tty allocation, even if ssh has no local tty.

Idempotency with Rails and Turbolinks

Tagged idempotency, turbolinks, d3.js  Languages ruby

To achieve idempotency with Rails and Turbolinks you have many options including these:

  • Option 1: Destroy DOM elements

Destroy existing elements that were created on the first page load. This works well with most libraries, including D3.js.

The is also the easiest to implement if you put everything under one root DOM element.

  • Option 2: Set a flag

Set a flag. Check if the flag is set before running the script. Note that you will have to use delegated event listeners or attach event listeners again. This is hard to do with, for example, D3.js.

document.addEventListener("turbolinks:load", function() {
  $('.linechart').each(function(_ix, el) {
    // Option 1: Destroy existing elements that were created on the first page load
    el = $(el);
    el.empty(); // empty removes all child elements. Normally this would be put in the script itself
    // Option 2: Set a flag to avoid calling the script twice 
    //if(el.attr('data-initialized') == null) {
    //  console.log("not initialized")
    //  linechart(el)
    //} else {
    //  console.log("initialized")
    // // Attach event listeners again
    // linechart.attachDelegatedEventListeners(el);
    //el.attr('data-initialized', true);