Channel Cancel Culture

I wrote a background job that backs up a git repo after every commit, with a poll interval, such that many commits submitted rapidly only triggers one backup process. The tests have been intermittently failing on CI, for seemingly odd reasons: Removing the test git directory during setup Error: Received unexpected error: unlinkat data/testGitOrg: directory not empty Trying to backup after the original repo has already been removed (maybe) target OID for the reference doesn't exist on the repository I added some more logging, and it looks like a race condition is in play.
Read more →

Openvpn CRL

The certificate revocation list is for blocking certificates/clients that were previously given access to the VPN. I originally created a crl.pem file via easy-rsa when we needed to block access for a customer that had moved on. This morning, we had a widespread VPN outage. I immediately panicked because, if an outage is caused by a problem on the customer’s end, we’re basically fucked, in that someone needs to get on a plane to rectify the problem, per customer.
Read more →

Gets Ruby

TIL that you need to make sure arguments are cleared during a ruby script, if you want to collect input from the user. So, instead of arg = ARGV[0] print "what is your quest? " quest = gets.chomp do arg = ARGV.pop print "what is your quest? " quest = gets.chomp Otherwise, quest will receive whatever ARGV has left in it.
Read more →

Git Historical Checkout

TIL that you cannot simply do git checkout <branch>@{2021-01-01}, or at least not according to this article. Instead, do git checkout ‘git rev-list -1 –before="Jan 17 2014" master’. The “'"s should be backticks.
Read more →

No Port Forwarding With Container Networking

If you need to use one container’s network across containers (e.g., –network container:vpn), you cannot also forward ports. I wasted an hour trying to figure out why I couldn’t forward port 3000 to 80 (-p 80:3000) when using container networking. After being thoroughly frustrated, I changed the default port of the underlying service to use 80, switched the port docker switch to -p 80:80, and now it works fine…
Read more →

Bash: Subshell Variable Scope

TIR (today I remembered) that, if you use a loop with a while read, that creates a subshell, and any variables you declared beforehand won’t be accessible. I had to change it to a for with a backticked command.
Read more →

Puppet Ssl Woes

I have run into this one at least once before, so I’m annoyed enough with having to google again to write down the problem and solution: When running puppet agent on one of our nodes, we are greeted with the following (excerpted and sanitized): Error: Request to https://[host]:[port]/puppet/v3 failed after 0.174 seconds: SSL_connect returned=1 errno=0 state=error: sslv3 alert certificate unknown Wrapped exception: SSL_connect returned=1 errno=0 state=error: sslv3 alert certificate unknown Warning: Unable to fetch my node definition, but the agent run will continue: What is actually happening?
Read more →

Tmux Current Path

Finally got annoyed enough with new tmux panes/windows not opening on the directory I’m currently on: bind c new-window -c "#{pane_current_path}" bind % split-window -h -c "#{pane_current_path}" bind ‘"’ split-window -v -c "#{pane_current_path}"
Read more →

Unbuffered

TIL that python STDOUT is buffered by default (at least for python3). You can get docker logs output by passing the -u flag. I added this to my shebang, so I only have to do ./main.py to run my script.
Read more →

SSL

Some notes so maybe I don’t forget how SSL certs work again: certificates are arranged in a chain from CA down the CA cert is special only because it is at the top of the chain you need to use the higher level cert to create lower level ones if you want them in the chain if you don’t need a separate authority, you can just use a self signed cert as the trusted root certificate (synonymous with CA?
Read more →

K8s Port Forwarding

I have redis in a pod, and wanted to connect to it to see some usage stats this morning. My initial, naive attempt was to hit redis directly, thinking maybe the k8s master maps these internal hostnames out to the host; no bueno. Next I tried connecting to the pod directly via kubectl exec -it pod/redis-123812739 sh but once I get in there I cannot telnet because that isn’t installed. I could’ve installed telnet, but I didn’t want to add custom stuff to a k8s-managed container.
Read more →

Replication

I spent the last couple of days figuring out if I can replicate a master server to a standby (formerly slave) server, without directly connecting to the master from the standby. The master servers are on premise in this case, and behind a VPN. We need to be able to query this standby, and collect records
Read more →

assert.Len

TIL that you can replace assert.Equal(t, 3, len(someSlice)) with assert.Len(t, someSlice, 3) Why they mix up which is the expected param is beyond me, but otherwise I like not having to add the len().
Read more →

Equal Values

TIL that assert.EqualValues allows one to compare numerals of differing types for equality: package main import ( "testing" "github.com/stretchr/testify/assert" ) func TestFoo(t *testing.T) { assert.EqualValues(t, int64(1), uint(1)) assert.EqualValues(t, int64(1), 1.0) } These asserts should both pass.
Read more →

Streaming

I often get down on how verbose Go is as compared to Ruby. However, this is an example where Go (and go-gin) is quite terse for what it’s doing: chanStream := make(chan string) go func() { defer close(chanStream) for i := 0; i < 10; i += 1 { chanStream <- fmt.Sprintf("msg %d", i) } }() c.Stream(func(w io.Writer) bool { if msg, ok := <-chanStream; ok { c.SSEvent("message", msg) return true } return false }) In this code we create a streaming HTTP response, such that the main server is not tied up while sending down data to the client.
Read more →

Docker Compose Override

Note to self: if I am not seeing env var overrides that I specify in a docker compose override file, check the following: is the file spelled correctly (e.g., hyphens vs periods and singular!) docker-compose.override.yml is it in the same directory as docker-compose.yml? is the dk convenience alias/script directly citing the config files, such as docker-compose -f docker-compose.yml? If so, you need to add another -f with the override.
Read more →

Git Timestamps

I just ran into a scenario where the git log was not telling me the correct order of commits. This mis-ordering happened because git only keeps/considers the commit timestamps up to the nearest second. So, if you have a script that is creating a bunch of commits, you could have several that are associated with one second, which will result in the mis-ordering. To get a more accurate order, use the –graph option on git log.
Read more →

k8s Monitoring

Some commands that are useful for monitoring k8s: > kubectl get namespace NAME STATUS AGE c66-system Active 83d default Active 83d kube-node-lease Active 83d kube-public Active 83d kube-system Active 83d menopon Active 83d > kubectl get pods –namespace=menopon NAME READY STATUS RESTARTS AGE redis-649f7db499-sl6h4 1/1 Running 0 74d web-69c576797d-7x7h2 1/1 Running 0 74d web-69c576797d-mjgfs 1/1 Running 0 74d web-69c576797d-zbfkz 1/1 Running 0 74d also, look to /var/log/pods for pod logs:
Read more →

Mui Class Override

If you need to override some CSS in a component that you didn’t author, but aren’t sure what the options are, just use a dummy one and check the dev tools console for errors. The available class names should be listed in the error message. const classes = useStyles() <ExternalComponent classes={{dummy: classes.myCssClass}}/>
Read more →

try/finally in Bash

If you need to do some cleanup whether or not a part of a bash script ran successfully, but also want to exit immediately if other stuff in that script fails, use a subshell: set -e # exit immediately on error touch fileThatAlwaysGetsRemoved # if this fails it exits without the cleanup ( # <— subshell set -e executableThatLooksForTheFileAndMaybeFails ) rm fileThatAlwaysGetsRemoved
Read more →