API Design at Hootsuite
Jon Slow sees that his favorite social media platform is missing a feature that would make it great: the ability to directly add pictures from his Google Drive account. He asks the company support if they’re going to add this feature, and gets a negative response, so Jon decides to do it himself. With his new programming boot camp certificate in hand, he gets to work, feverishly coding day and night. Without any official support from the platform, he has to get creative, hacking a solution together, spoofing HTTP calls, spending hours on Stack Overflow, until finally, it works. His third-party add-on, with no support from the platform creators, successfully connects to his Google Drive (sometimes). Suddenly, the platform creators push a minor, unrelated bug fix, and all of Jon’s work comes crashing down. His integration crashes immediately and permanently. Months of work, gone. If only there had been an API he could use.
What is an API?An API (Application Programming Interface) is a well-documented set of functions that allow one program to interact with another. Essentially, an API is a contract between the API provider and the client program: the client knows that it can call a predefined function with certain types of parameters, and the provider promises to return a specifically formatted response for the client to use.
At Hootsuite, we allow third-party developers to create apps that integrate with our social media platform in a variety of ways. For some, we have created inbound APIs that third-party apps can call to take advantage of Hootsuite features. For others, we define an outbound API that an app must implement, and then Hootsuite will invoked predefined endpoints in that API. Recently, we created functionality to support a new type of app, media apps, which are used to find and attach images or gifs from an external content source (like Giphy or Google Drive) to a social media post created in Hootsuite. As part of that process, we defined an outbound API for media apps to implement, and we decided to share our decision-making process for the API design here. Once it’s in wide usage, it’s very difficult to significantly change an API, so we tried to be very deliberate with our design process in an attempt to get it right the first time.
Goals for the API design
- It has to work. The most important part! The functions in the API have to return data in a format that lets us at Hootsuite do what we need with it. For this specific project, that meant the app had to return a list of media, with each item containing information such as file type, height, width, and size, in a format readable by the Hootsuite dashboard.
- It has to be easy for us to work with. We don’t want every app to return data in a different format, as that increases the amount of code we have to write to support it as well as the possibility of unexpected bugs. We expect every app to return data in almost exactly the same format, to minimize the work needed on our end to support different integrations.
- It has to be easy for devs to work with! None of this process works if we can’t convince developers to create apps for our platform! Because we are requiring developers to conform to our API design, we want to make sure that it isn’t a pain to do so. Ideally, our API specifications are easy to understand and implement, so that the development process for 3rd-party devs is as smooth and enjoyable as possible.
- It has to be RESTful. REST is a web API design standard in common usage across a ton of web applications. By making our API RESTful, it will be similar to other APIs that developers may have come across previously, and will help them understand how it’s meant to be used. RESTful design includes a number of best practices that make an API easier to use and maintain.
The APIWith those goals in mind, after a few meetings, we produced this spec for our API (still in alpha). This is the API that we expect all apps with media library functionality to implement. The main component is the “media” endpoint, which Hootsuite will call in order to retrieve the media that the app provides. The documentation defines both the type of requests that Hootsuite will make, as well as the format of the responses it expects. It defines which fields in both the request and the response are optional, which are required, and which are sometimes optional and under what conditions. There are also some optional OAuth endpoints, for apps that require the user to login before they will provide media. More detail about the OAuth setup can be found here if you’re interested.
While we’re pleased with our final API, it’s still not perfect, and its flaws come from places where our goals conflicted with each other. One example is the “cursor”. When an app has more than one page’s worth of media that fits a specific query, it returns a cursor in addition to the rest of the payload. The cursor is a unique key that Hootsuite can send to the app to get the next page of results.
During our internal planning discussions, we thought about providing the app developer a guarantee that our software would return all the search parameters from the first page in addition to the cursor when asking for the second page. We eventually decided against it, as we thought it would add too much complexity in designing and implementing our internal services that interface with media apps.
As part of this project, we developed our own app that implemented the API as a proof of concept. During the development process, we realized that only sending a cursor, without the related search params, made it a lot harder for the app to generate said cursor. We had to encode all the search parameters in the cursor, send it to Hootsuite, and then decode the cursor that Hootsuite sends back when it asks for the next page of results. Not impossible, but a great deal more complicated than the alternative when search params are sent: “page2”.
This problem is typical of API design. Two of our major goals (“make it easy to use for external devs”, “make it easy to use for internal devs”) conflicted, and we had to make a decision about what was most important to the team. Maybe it was the wrong decision! We won’t really know until we show the API to more external developers and get some feedback. At that point, maybe we could change it.
Changing an API after releaseChanging an API isn’t as easy as it might seem at first. As I said earlier, an API is a contract between two pieces of software, promising each other that they will both act a certain way. Changing the rules of that contract can cause a lot of problems. If the API implementation changes, every program that uses the API will probably have to change as well, and given that the API provider and consumer are usually developed by different teams or companies, it might be a long time before everything is back in sync. In the meantime, software will break and customers will be unhappy.
In order to avoid that, there are a few steps to take when updating your API. The most important is to make non-breaking changes. These are changes that will allow all apps currently using your API to continue using it in exactly the same way without malfunctioning. Generally, this restricts your changes to adding optional features: either new methods that API users can call, or optional parameters to existing methods that will add functionality.
Breaking changes, then, are basically anything else: removing functions, changing the required parameters that they accept, or the type or form of the data they return. If you really need to make a breaking change, then you’ll need to make a new version of the API, release it, try to convince developers to do the work to switch to the new version, and continue to support the old version for as long as possible (some might say forever) while developers are switching. The easiest way to do this is to version your endpoints (e.g. currently our media endpoint is called “v1/media”, and if we made breaking changes, we would create a new “v2/media” endpoint while leaving the old one alone). Even so, it’s a major headache, and the main reason why it’s important to be thoughtful with your initial design.
FinaleJon Slow has been ruined. His integration is broken beyond repair. Every time he tries to use it, his computer gives off a little puff of smoke in despair. His programming career is at death’s door. Suddenly, an update arrives: the platform creators have released an API for media integrations! Reinvigorated, Jon sits at his computer. An hour later, his Google Drive has successfully connected to the platform. He can quickly and easily attach images to his posts. It’s everything he dreamed of and more. Jon Slow’s life is complete.
About the Author
Jacob Brunette is a Software Developer Co-op with the Platform team at Hootsuite. Jacob studies computer science at the University of British Columbia. Connect with him on LinkedIn.