Photo by Nemanja Jeremic on Unsplash
1. Install dconf
I already had this installed
$ sudo apt install dconf-tools
2. Disable caps lock (reenable by pressing both shift keys at once)
$ setxkbmap -option "caps:none"
$ setxkbmap -option "shift:both_capslock"
I did only disable for now.
-- UPDATE
This works but running this automatically is of course what was missing in this post so I added step 3
3. Add this to your .bashrc file
vim .bashrc
See also: How to disable the Caps Lock key on Mac