Recently I set up a Zendesk support phone number voicemail box on CameraLends. It takes a message from every caller and sends an email of a transcript and link to the audio recording. Up until today, it had been nothing but spam. But today I got a message from a woman saying she tried to sign up 7-8x and it didn’t work.
I immediately dug into my logs. After finding no anomalies around the time of the bug report, I tried to repro — and was unable to sign up. Some quick debugging yielded two bugs: a misnamed text input that was blanking out the email field AND controller code that wasn’t setting a `flash[:error]`. It was a quick fix, but an embarassing kick in the pants. I’ve written specs but don’t run them regularly, and I’m sick of basic functionality breaking. It’s time to protect me from myself.
Step one: fix the broken specs
There were some cobwebs to shake off. Luckily, a few test failures were from code that didn’t exist anymore. The other tests were straightforward to fix, except one. It didn’t feel right to disable Yet Another test, but I didn’t want to get hung up on the one failure instead of getting automated testing completely set up. If you’re in a similar situation, I think you should make the same decision: automated testing of 90% of your tests is a better end-of-day win than spending most of the day maybe fixing one test case.
Step two: add a regression spec
You can’t enumerate all possible bugs, but you can easily prevent yourself
from making the same mistakes over and over.
The sign up bug was caused by two input[type=text] fields with the same name,
meaning the second field was the value that the browser submitted with the
form, blanking out the first field. There was a similar spec testing signup,
but it supplied the parameters as a hash (a common pattern in RSpec testing)
and could not catch this bug. For this specific bug, I ended up using:
expect(response.body).to have_css('input[name="user[email]"]', count: 1)
This matcher will ensure that exactly one (not zero, or more than one) field
has the name in question. Certainly not bulletproof, but a lot better than
Step three: automated testing in the cloud
As I’ve proven to myself, tests are worthless if not run regularly. Initially I just wanted a service that would pull from my Heroku repository and do a nightly run. All of the products in this space seem to prefer Github/BitBucket and using a post-receive hook on the repo. I ended up going with Drone.io at $25/month for unlimited builds (plus 50 free builds on the test plan) but Codeship is very popular as well ($50/month for the same plan, and mode pirate related language than you’d care for).
Since I only have repos on my workstation and at Heroku, I needed to set up either a Github account or BitBucket. I chose BitBucket simply because they have free private repositories. After linking the repo in Drone.io, I had to set up a build script. This took about an hour because my gemset takes ~6 minutes to install and it took 8 builds for it to finally come back green — not an ideal feedback loop. To save anyone running Rails on Heroku the headache, here’s my build script:
psql -c "CREATE USER cameralends WITH PASSWORD 'secret';" -U postgres
psql -c "CREATE DATABASE cameralends_test OWNER cameralends;" -U postgres
psql -c "GRANT ALL PRIVILEGES ON DATABASE cameralends_test TO cameralends ;" -U postgres
RAILS_ENV=test bundle exec rake db:schema:load
bundle exec rspec
Now all I have to do is push to the BitBucket remote in git, and in about seven minutes I receive an email with my build status. Super easy!
Had I known setting up automated testing “in the cloud” was going to be so easy, I probably would have done this sooner. I think I was fighting putting my code on Github/BitBucket for some silly reason — but the need for automated testing outweighs any of the concerns I had that I can’t even recall. The email build notifications is just what I need to stay motivated and test regularly.