A codebase with many active contributors grows and evolves like an organism. File systems that store code are shaped like trees, and all of the changes that happen over time are akin to organic growth and decay.
On the sixth anniversary of the Hootsuite’s codebase, Specialist Engineer Bill Monkman spent a weekend building a beautiful visual artifact that conveys those organic qualities and much more. Using Gource, Andrew Caudwell’s amazing code repository visualization tool, Bill has given us a macroscopic, time-lapsed view of every change, big or small, that has gone into making our product what it is today.
For those of us who have been fortunate enough to be a part of the team that builds Hootsuite products, being able to see our codebase grow up from its humble beginnings is a moving experience. Now with over 10m users, Hootsuite engineers get to work under the hood of a codebase that runs on hundreds of Amazon instances, and processes millions of social messages every day.
In this video, we are able to see not only how our code has changed but also the evolution of our team. We see some of the major technology decisions we’ve made along the way (and their consequences). We see big wins, and even a few things we might have done sooner or a bit differently.
When we showed this video to people outside of our team they also saw something of the beauty in a large codebase evolving over time. So, we decided to share it more broadly with our technical community and spark some conversation.
What we hope is evident is that software engineering is not just about putting headphones on and going into the zone for protracted periods of time. While there is some of that to be sure, we believe that building software is a team sport and a creative endeavour, the output of which has a life of its own.
About the Author: Geordie Henderson is the VP of Engineering at Hootsuite. Follow Geordie on Twitter @geordie_h.
Collective action is a wonderful thing. Meetups and open source projects are two examples of people getting together to have a conversation about the things they care about, with the aim to do something positive for themselves and their community. How can this collective action happen inside your organization? At Hootsuite, the answer is Guilds.
Guilds are a self-organizing group of people with common interests. It is a natural forum for social interactions that build relationships that, in turn, promote cooperation, cohesion, and productivity. 
Guilds provide a horizontal communication layer across our Product Engineering teams. Our engineers, testers, and other staff use them to set their own missions, to establish technical roadmaps, to take on joint tasks for their grassroots initiatives, and to promote education through experiential learning. This collective action benefits their members, their craft, and our organization.
Sometimes a wonderful tool comes along that makes a kludgy process radically better. Packer (thanks again Hashicorp!) and ServerSpec are great examples. The 1-2 punch of Packer + ServerSpec combined with the automation abilities of Jenkins made a significant impact on our automated server image creation. This combination reduced our time-to-deploy, took our visibility from ‘translucent’ to ‘transparent’, improved our traceability, and generally made our Ops Engineers much, much happier. Read on to find out how these tools can make you happy too.
When writing Android code, there are many situations in where you’ll need to make network calls. There are several different ways of performing these network calls, and after reviewing them all, we’ve determined some are good, some are bad, and one in particular is just plain ugly. We’ve compared and contrasted several methods, and along with some code examples, will provide some basic insight into using what we think is the best method of making network calls in an Android application.
Ow.ly is our URL shortening and click analytics service, and a critical part of Hootsuite because it handles several billion clicks a month. We love Ow.ly, and we’re delighted that our customers feel the same way – but the booming popularity of Ow.ly introduced scaling issues for our Engineering Team. We could have mitigated these issues by keeping with our current infrastructure and code framework, but it turns out that’s a lot of effort and not cost-effective. Instead, we searched for a solution that would allow us to scale Ow.ly, keep our AWS costs in check, and allow us to build new functionality … all while maintaining 100% uptime. Easy, right? Kinda :).
We turned to Typesafe’s Play Framework to refactor and scale Ow.ly’s services. Today, 100% of Ow.ly’s API, the web UI, and the URL redirect services are all running off the Scala-based framework.
After our successful relaunch, our friends at Typesafe produced a case study on our work, our research, and how we decided the Play Framework was the best solution to our scaling issues. Check out the case study to learn more, and visit the Typesafe blog for more innovative technology solutions.
At Hootsuite we’re always on the lookout for new technologies that might help us, whether it be a new kind of database, new programming language or an improvement to an existing process.
Something that has been in our sights for a while is Distributed Configuration. This is a puzzle piece we’re currently missing, and one that will help guide and support many of the tasks that we’re going to face in the next couple years. In this post we’ll recount our experience with implementing Consul to fill that need.
We’re improving our build and deploy pipeline, but don’t yet have a way to handle dynamic configuration of services. This part of the system will see some heavy work in the next year, and Distributed Config will influence a lot of the design of that system.
This May I found myself on a newly formed Engineering team: CoreUX, a group focused on defining and maintaining the experience of Hootsuite’s users and the front-end tools we use. Our team started out as 12 people working on multiple projects, and we’ve nearly doubled in size since. The team is a mix of old and new employees – some of us hardly knew each other. We needed to find a way to bond quickly and stay connected as a team.
After two days of non-stop work, Hootsuite Engineers Anubhav Mishra and Luke Kysow won the Vancouver DevFest Hackathon in Vancouver, BC. Luke shares his experiences below (originally posted on Medium):
From Saturday morning, November 8th through to Sunday night, Anubhav Mishra and I worked to build LeapSnap at the DevFest Hackathon in Vancouver. The idea for LeapSnap came from a Sci-Fi short I watched where everyone had chips implanted that would store their memories via video. Whenever they wanted to share their memories with friends, they would “cast” them onto any screen within view. While sitting at the dinner table, they could then wirelessly control the TV, cycle through to the memory they wanted to share, and then play it. Read More →
The DataLab team at Hootsuite is responsible for storing the constant firehose of data that our 10 million users generate for product intelligence analyses. All useful insights stem from a hardy storage-and-retrieval system that we can rely on. Recently, we migrated our primary storage back-end from Apache Hadoop to Amazon’s Simple Storage Solution (S3) in order to scale our ability to analyze data. What follows is a short retelling of the journey we took and the lessons learned while handling the nozzle.
Software-as-a-service organizations live and die on service reliability, performance, and uptime. Hootsuite monitors these key metrics for all our services, and effective monitoring is especially vital as we steadily migrate to a service oriented architecture. In this post, we outline ways our Analytics teams monitors Hootsuite services.