I've written a few Tampermonkey userscripts to improve websites that I regularly use, and I wanted to share some patterns that I have found useful.
Generally I iterate on the scripts in the Tampermonkey Chrome Extension editor, and then push to GitHub for versioning and backup.
I somewhat recently ran into an issue where our system was incorrectly creating duplicate records. Here's a writeup of how we found and fixed it.
After reading through the request logs, I saw that we were receiving intermittent duplicate requests from a third-party vendor (applicant tracking system) for certain webhook events. We already had a check to see if records exist in the database before creating them, but this check didn't seem to prevent the problem. After looking closely, the duplicate requests were coming in very short succession (< 100 milliseconds apart) and potentially processed by different processes, so the simple check would not reliably catch the duplicate requests.
Today I will tell you some stories of how I made mistakes in our production environment, and how I am trying to help prevent future mistakes using iTerm.
At work we are mid-journey to having more automation around our deployments, provisioning, backups, monitoring, and so forth. But at the moment, we have some things that are typically done manually. Within recent memory, I was SSHed into our QA (staging) box and for some reason wanted to rename the database. A few minutes later, someone came down and said "production's down!" 1 (Production is the end-user visible environment, the one thing that we don't want to be down.) I was thinking, "hmm, we haven't changed anything recentl… wait, was I actually on the QA box?" Sure enough, what I renamed was the production database on the production environment! A minute later service was restored, but this was the most downtime this quarter during the day (a handful of minutes.)
As part of our postmortem on this issue, we identified that switching my terminal profile whenever I thought I would be in a production-like environment would be useful. For example, if I am going to be SSHing into a QA box, I might create a new profile that has a different background color. This would help disambiguate the two environments.
The other day after hours, I was switching back and forth between QA and production SSH environments to try to debug a problem on the QA side. I again thought that I had SSHed into the QA environment but I didn't read my SSH command well enough when cycling between those environments (using
There are two problems with the profile switching approach: I need to remember to switch profiles when I am SSHing, and I need to be SSHing into the right environment for the given profile. These are error-prone enough that I don't think the manual profile switching approach is workable long-term. Again, in a perfect world, we would have everything already automated and some way of making all of our changes through well-tested or peer-reviewed means. But there has to be a stopgap solution.
Today I want to share a tool that I have found indispensable for finding and fixing intermittent tests in test suites. It's a little script I wrote, called ntimes.
Based on the commit logs to my dotfiles repository, until about 2014, to run the same command many times, I would press up in my terminal and press enter. While effective, this approach has the disadvantage of requiring me to be present at the machine and do manual work. I thought: there must be a better way.
So, probably by cribbing from somewhere and adding my own extensions, I made a script that could run an arbitrary command-line command multiple times and report a summary at the end. To use it, I would use something like:
In our Slack organization and Github pull request review, I have noticed a small pattern of using parentheses to soften or clarify the statements that we make. Sometimes it used by someone in a position of authority to emphasize that a comment is an idea, not a directive. Other times, it represents that things are not critical to address, but might be something that we want to look into.
I originally wanted to call this post "The Shipley Parenthetical" since Kyle uses it all of the time. Didn't know if he thought of it this way / approves though. :)
In this post I'll give a few examples of how this works and some thoughts on it.
Just capturing a thought that would be helpful or is on the commenter's mind, but wanting to be clear that there is no specific action item around this.