Posts from May 2017

As Hootsuite’s products have grown, we have started to move away from a single monolithic project to a suite of microservices. Each of these microservices has an API that we use to hook all these disparate systems together. Like any large software system, we need to test these services end-to-end in order to ensure that they function correctly as a whole and to quickly find issues when one service changes in a way that affects its integration with others. To that end, we created Wasabi.

What is Wasabi?

Wasabi is a NodeJS app designed for end-to-end API testing. It uses Mocha to run tests and Chai as an assertion framework. We also designed a custom HTTP client module, as well as a custom Consul module that we use to lock the fixtures used in our tests.

Why did we create Wasabi?

Hootsuite had a number of previous methods to test our APIs, and none of them really did what we wanted. One of our first attempts was a combination of PyUnit tests attached to an older custom HTTP client that never worked very well. It had big problems trying to run tests concurrently and cleaning up our test setup after a failed test, and pretty quickly we decided we needed something better.

Another solution that we used for awhile was Runscope, a third-party product that allows users to create tests through a GUI. Runscope worked fairly well, but because it was a third-party product, we weren’t able to integrate it with our source control and we had to rely on external support more than we liked (plus it came with a monthly fee).

We also have a number of Scala integration tests, many of which are still in use. These were our best attempt at writing good tests before Wasabi, but parsing JSON is a lot more cumbersome in Scala than in Javascript. Eventually, we decided we needed something better, so we created Wasabi.

Wasabi was designed to solve all the problems that existed in our previous tests. It takes care of the concurrency and cleanup problems that plagued our Python solution, and developing it in-house means we don’t have to rely on an external product that’s hard to integrate with our code. It’s much easier to parse JSON responses with Wasabi then it is with Scala, which makes our tests easier to write and understand. Wasabi can perform end-to-end testing for all of our external APIs more reliably than previous solutions, and it has a fun emoji to use on Slack.

How to write end-to-end API tests?

Regardless of whether you use Wasabi or some other API testing framework, the steps required to write API tests are fairly consistent. There are two major elements to API testing: creating fixtures and writing the tests themselves.

Fixtures:

Fixtures refer to the data used in the API tests. At Hootsuite, that usually means Hootsuite accounts with access to specific social networks or organizations, but it could be whatever type of data is controlled by a given API. The important thing about fixtures is that they are fixed. Your tests will be verifying that the fixtures contain the data that you expect, so it’s important that they are not changed outside of tests, or you risk your tests failing due to outside interference. At Hootsuite, we have designated test accounts on each of our production environments that are used for specific test suites only and aren’t touched otherwise. We also lock each test account using Consul before running tests that hit them, so we can be certain to avoid outside interference while our tests are running.

Writing the tests:

After you have your fixtures, you can move on to writing the tests themselves. A happy path Hootsuite API test can generally be broken up into the following components:

  • Locking and verifying the account used for the test.
  • Authenticating the test account.
  • Echoing the API (i.e. hitting an endpoint that does nothing except respond with the same message you sent it. Useful for troubleshooting network problems when your tests aren’t passing).
  • Setting up external dependencies, such as bins for webhooks.
  • Hitting your endpoint.
  • Verifying that the response has the shape and content that you expect.
Below is an example Wasabi test that hits all of those steps. If you’re not using Wasabi, some of the specifics might be a little different, but the general structure will be very similar.

Example Wasabi test

The meat of the test is the last step, verifying the response. This step should be overly detailed, so you will know exactly what’s wrong when your test fails. Good practice is to start with verifying that the response looks as you expect at a high level, and then work your way down. A typical verification hierarchy would look like this:

  • HTTP response code is correct
  • response has a body
  • the body contains a data array
  • the data array is not empty
  • the data array has the fields you expect
  • each field has the content you expect
What are the current issues with Wasabi?

Wasabi runs end-to-end tests over Hootsuite’s internal network. That means it is heavily reliant on the stability of that network to run correctly. Slow connections can cause timeouts (this is especially a problem on our terrible wifi). Upgrades to seemingly unrelated projects can cause tests to break unexpectedly, which is especially a problem in our constantly-changing dev environment. Often, it’s difficult to tell if a new test failure is due to changes in the code or due to unrelated changes in our infrastructure. These aren’t necessarily problems with Wasabi itself, but they do place restrictions on how we can use it. We don’t run Wasabi tests on our dev environment at all, instead using it as a final check on production before we push an update to all users. This has helped lower the time we spend fighting with failing tests, while still ensuring that our product is tested thoroughly before release.

About the Author

Jacob Brunette is a Software Developer Co-op with the Platform team at Hootsuite. Jacob studies computer science at the University of British Columbia. Connect with him on LinkedIn.

Take a look at this piece of code:

1
2
10 PRINT "Hola, amigo!"
20 END

That was the very first two lines of code that I ever wrote! I was 7 years old, and when I ran this program my life completely changed!

This is why I love Hackathons! It brings back the exact same feeling of excitement that, as a 7-year-old, I got when I realized that I could make a computer do pretty much whatever I asked it to do! That’s why you should “hack” too! It’s all about seeing things that we do on a daily basis (like coding) but imagining that we are doing it for the first time so that 7-year-old passion always comes forward and shine on everything you do.

This week and next is our 2017 Hootsuite Hackathon, and for us, that’s the perfect opportunity to unleash that curiosity, determination, and passion! and be a 7-year-old again! I’ve been lucky enough to participate in the last two company Hackathons, and they have brought some of the best experiences in my career. They will always be my “favorite time of the year!

I really recommend participating in Hackathons whenever you can. It’s lots of fun! No idea is silly when it comes to hackathons, that’s the fundamental principle of brainstorming and how true innovation begins.

Here are a few tips that I learned from previous hackathons and that I wanted to share for anyone thinking about joining one:

  • If you are having a hard time coming up with ideas, pair up with our Product and Design team members; they invest a lot of their time brainstorming on how the future of our products may look like, so they could have some cool stuff in their “ideas drawer” for you!
  • Try to work with people from other teams. This is a great opportunity to get to know talented people that don’t necessarily interact with you on a daily basis. #newfriends
  • Hackathons are not just for Engineers. Last year I worked with designers and the year before we brought up Marketing. It is amazing what fresh eyes can bring to your project. We are always surrounded by so much talent outside of engineering and a lot of that talent is more than willing to participate.
  • Have tons of fun!
Hack away!

 

About the Author

Rei Colina is a Lead Software Developer. He leads the Labs Team at Hootsuite. Get in touch via Twitter @rei_colina