How to debug and fix chromedriver and selenium errors

Tagged capybara, selenium, refused, connection, chromedriver  Languages bash

You an enable the chromedriver log to debug and fix chromedriver and Selenium errors such as “unknown error: net::ERR_CONNECTION_REFUSED”.

First start the chromedriver in the background:

nohup chromedriver --port=12345 --verbose --log-path=/tmp/chromedriver.log &

Then configure Selenium to use the chromedriver instance:

Capybara::Selenium::Driver.new(@app, browser: :chrome, options: ..., url: 'http://localhost:12345')

Next run the tests and inspect the chromedriver log.

In my case the issue was that an unreachable URL, see the unreachableUrl property in the output below:

[1659101781.048][DEBUG]: DevTools WebSocket Event: Page.frameNavigated B30A4B9FE0908FDACE0522341267520A {
   "frame": {
      "adFrameStatus": {
         "adFrameType": "none"
      },
      "crossOriginIsolatedContextType": "NotIsolated",
      "domainAndRegistry": "",
      "gatedAPIFeatures": [  ],
      "id": "B30A4B9FE0908FDACE0522341267520A",
      "loaderId": "595BC7FEFD2C1ECCE31157FF7972F889",
      "mimeType": "text/html",
      "secureContextType": "InsecureScheme",
      "securityOrigin": "://",
      "unreachableUrl": "http://localhost:3030/",
      "url": "chrome-error://chromewebdata/"
   },
   "type": "Navigation"
}

Thanks: https://makandracards.com/makandra/301588-how-to-enable-chromedriver-logging

Shadow DOM and shadow root

Tagged attachshadow, shadowroot, dom, root, shadow  Languages javascript

https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_shadow_DOM

Shadow DOM is very important for web components because it allows specific sections of the HTML document to be completely isolated from the rest of the document. This means CSS styles that are applied to the DOM are not applied to the Shadow DOM, and vice versa.

The shadow DOM can only be accessed through the reference returned by attachShadow:

const shadowComponent =  document.getElementById('component').attachShadow({mode: 'open'}); 
const p = document.createElement('p'); 
p.setAttribute('class', 'title'); 
p.innerText = 'Lorem ipsum'; 
shadowComponent.appendChild(p);

The shadow root can be styled with CSS:

const css = document.createElement('style'); 
css.innerText = ` 
  .title { 
      font-size: 2rem; 
      font-weight: bolder; 
  }
`; 
shadowComponent.appendChild(css);

Use templates and slots make it easier to create components: https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_templates_and_slots

Reference: https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_shadow_DOM

Also see the Awesome Styled Components repository and the Styled Components project for inspiration: https://github.com/styled-components/awesome-styled-components https://styled-components.com/

Details HTML element with open attribute

Tagged collapsible, details, no-javascript, open  Languages html

The details HTML element can be used as a no-Javascript alternative to e.g. the Bootstrap collapsible element:

<details open>
  <summary>This is always shown</summary>
  <p>This is hidden.</p>
</details>

Pigeonhole principle

Tagged hash, cryptography, crc, sha256, md5  Languages 

When working with hashing functions and cryptography the pigeonhole principle is essential knowledge: https://en.wikipedia.org/wiki/Pigeonhole_principle

In mathematics, the pigeonhole principle states that if n items are put into m containers, with n > m, then at least one container must contain more than one item.

For example, given that the population of London is greater than the maximum number of hairs that can be present on a human’s head, then the pigeonhole principle requires that there must be at least two people in London who have the same number of hairs on their heads.

This means that in order to create a completely collision-free hashing function, every message would have to have a hashed output of the same length as the input.

Simple backup script with GPG encryption

Tagged gpg, backup, tar  Languages bash

Save to a file named backup and run chmod +x backup:

#!/bin/bash
#
# Uses tar to store files in one TAR file.
# Uses gpg to encrypt the TAR file.
#
# Example:
#
# ./backup /folder-to-backup
#
#
export DIR_NAME=$1
export BACKUP_DATE=`date +"%Y%m%d-%H%M"`
export BACKUP_FILE=$DIR_NAME-$BACK_UPDATE.tar
tar -cvf $BACKUP_FILE $DIR_NAME
#
# TODO:
# --exclude="*.log"
# -r = recipient (only one who can decrypt)
#
gpg --encrypt -r [email protected] $BACKUP_FILE

EXPLAIN ANALYZE with ActiveRecord

Tagged analyze, explain, activerecord  Languages ruby

ActiveRecord example:

module ActiveRecord
  module ConnectionAdapters
    module PostgreSQL
      module DatabaseStatements
        def explain_analyze(arel, binds = [])
          sql = "EXPLAIN ANALYZE #{to_sql(arel, binds)}"
          PostgreSQL::ExplainPrettyPrinter.new.pp(exec_query(sql, "EXPLAIN", binds))
        end
      end
    end
  end
end

puts Job.where("slow = true").explain_analyze
...

Raw SQL example:

puts ActiveRecord::Base.connection.select_all(sql).to_a.map { |x| x.fetch("QUERY PLAN") }.join("\n")

See: https://github.com/rails/rails/blob/b8787b92e7c94b8575118b5f259dd47b5de4772f/activerecord/lib/active_record/connection_adapters/postgresql/database_statements.rb#L7-L10

Converting a string to a number with CRC

Tagged crc, number, string, conversion  Languages ruby

CRC can be used to convert a string into a number:

require 'zlib'
integer = Zlib.crc32('snippet')
=> 2518453461
Zlib.crc32('cat')
=> 2656977832

The CRC algorithm is normally used to detect changes or errors in large chunks of data.

Waiting for Socket (IO) to be readable / writable in Ruby

Tagged ruby, io-wait, readable, writable, socket  Languages ruby

Option 1: Use IO.select

Use IO.select with read_nonblock, write_nonblock, and connect_nonblock.

Option 2: Use io-wait

require 'io/wait'
r, w = IO.pipe
w.wait_writable(0.1)

Example from https://bugs.ruby-lang.org/issues/12013?tab=history:

# 30 second wait
IO.select([mysock],[mysock], nil, 30)
# as opposed to (60 second wait)
require 'io/wait'
mysock.wait_readable(30) && mysock.wait_writable(30)

How to simulate TCP read, write, and connect timeouts

Tagged tcp, timeout, connect, read, write  Languages 

How to simulate TCP read, write, and connect timeouts:

  • Connect timeout

Option 1: Drop all SYN packets with firewall or iptables rules

Option 2: Try connecting to a non-routable IP, e.g., 10.0.0.0

  • Read timeout

Read from a socket to which the client or server is not writing while keeping the socket open on both ends.

  • Write timeout

You’re (almost) out of luck. There are OS-level buffers, so the write timeout might never happen.

Option 1: Use https://github.com/openresty/mockeagain and LD_PRELOAD to mock syscalls

Reference:

https://github.com/openresty/programming-openresty/blob/master/testing/testing-erroneous-cases.adoc

https://github.com/openresty/mockeagain