Software-as-a-service organizations live and die on service reliability, performance, and uptime. Hootsuite monitors these key metrics for all our services, and effective monitoring is especially vital as we steadily migrate to a service oriented architecture. In this post, we outline ways our Analytics teams monitors Hootsuite services.
During my time here as a member of Hootsuite’s Customer Happiness team, I’ve come to understand our techniques for debugging a large scale web application. I want to share the five best techniques I’ve learned for debugging issues on a web application as massive as Hootsuite, though they can really apply to debugging web applications of any size.
Functional Reactive Programming has been adopted in many programming communities, and for good reason. Trying to manage multiple asynchronous calls usually results in a mess of code that is not only tricky to debug, but difficult to maintain and build upon — not to mention the many “gotchas” surrounding the use of AsyncTasks on various different versions of the Android SDK. This kind of code usually ends up being the bane of an Android developer’s existence.
Enter the Android module for RxJava.
By using a souped up version of the Observer pattern, we gain the ability to create incredibly powerful chains of logic that can be specifically run against various threads (UI, background, etc) and that can be further modified by users of these observables to tune them exactly for the UI they are trying to populate.
Reposted from our Hootsuite Blog
After attending Vancouver’s Polyglot Unconference in May 2013, Beier Cai and Rahim Lalani approached me with an idea: kickstart short, unconference-style talks on technical topics within our Engineering group. I loved this riff on the traditional lunch-and-learn and decided to set a short time limit of just 5 minutes per talk.
Why so short?
Short is all you need to start a conversation. Short also defines our modern day attention spans, and short is hard because this constraint requires you to think critically and creatively about how to convey only essential information. We started out with only technical topics but soon expanded to include non-technical topics and now include speakers from other parts of our organization as a way to expose our audience to different roles and functions. Read More →
At Hootsuite, we analyze user data to help us answer questions about product usability and give us insight into trends. Recently, we designed and implemented a RESTful API that provides access to our aggregated data sets and shields our data consumers from changes in infrastructure that might be happening in the backend. We built our RESTful API using the Play Framework with Scala and we believe it was a great decision for multiple reasons.
Currently, all of our logs are stored in HDFS and S3. We use Apache Spark to aggregate raw logs into more meaningful and usable data sets. For instance, one of our Spark jobs calculates an aggregate count of Hootsuite’s unique daily users across all our different platforms and plan types.
The aggregated data sets are stored in a central RDBMS and are used for data visualizations on dashboards around the office and reports which serve multiple internal stakeholders. Since we are continuously experimenting with different data processing pipelines and data stores, we decided that it would be necessary to provide a constant and defined way to access our data through an API. Providing an API also makes it easier to clearly communicate which data sets are available and makes documentation a much easier process.
We came across multiple features that demonstrated the advantages of using Play to design and implement our API: Read More →
Software-as-a-service organizations live and die on service reliability and uptime. Having an “on call” team (formal or otherwise) that can quickly react to incidents and outages is mission critical to the business. Hootsuite is no different – our customers around the globe rely on Hootsuite.com being available 24/7 – but even software engineers need sleep. So, who holds down the fort if the unthinkable happens outside business hours?
This year we overhauled our On Call practice to version 3.0. This post walks through the evolution of our On Call model: the stark reality of 1.0, the noble intentions of 2.0, and the practicality of 3.0.
On Call 1.0
Our first On Call system was informal and happened reactively, out of necessity. A handful of original software engineers and operations engineers who built Hootsuite.com were unofficially on call all. the. time. No sleep allowed, no knowledge transfer, no metrics, and no visibility into incidents or issues or remedies. Read More →
For my 8-month co-op term at Hootsuite, I have been working on the Streaming team. The Streaming infrastructure allows services to receive real-time updates for a large user set. Streaming is responsible for delivering push notifications, analytics, and interaction history to subscribed Hootsuite users. One example is receiving a push notification on your mobile device for a Twitter retweet or mention from the Hootsuite application. Streaming is a distributed system and every complex distributed application needs a coordination and orchestration system of some sort, so the team decided to integrate ZooKeeper into the Streaming infrastructure back in 2011. ZooKeeper is a software project of the Apache Software Foundation, providing an open source distributed configuration service, synchronization service, leader election, and naming registry for large distributed systems.
One of my projects during my co-op term included fixing the previous implementation of ZooKeeper, as there were a number of things that were not properly working. Originally, ZooKeeper was embedded within the Streaming application, and there were a number of issues with this implementation.
In 8 months, I’ll be graduating from the University of British Columbia with a degree in Physics and Computer Science. I’ll be looking for jobs shortly thereafter, armed with a résumé to convince future employers that I am an amazing iOS developer. Hootsuite will be at the top of my ‘Work Experience’ section, along with a list of the various technologies I used and the tasks I was assigned. It’ll probably look something like this:
Hootsuite – iOS Developer Co-op
January – August 2014
Worked with Xcode 5 and 6, Objective-C, Swift, and iOS 6, 7, and 8. Contributed to new features and bug fixes for the Hootsuite app, and helped it become featured in the iOS App Store. Gave presentations, collaborated with teammates, and wrote automated tests using KIF. Used Facebook, Twitter, and Google APIs, communication patterns such as KVO, delegation, and notification, and Interface Builder with auto-layout.
Do you hate coming to the office, firing up your favourite text editor, getting ready to do some work, and .. you forgot to start your Vagrant machine? Now you have to open up the terminal, type vagrant up, wait 90+ seconds for the beast to load and .. it doesn’t work because you forgot to connect to the VPN and Puppet cannot correctly provision the box without it. So, you do that too.
Now that everything is setup – you code the entire day, pack your things, arrive home, start browsing threads on Hacker News and… the low battery warning comes up because you forgot to close Vagrant.
We’ve really enjoyed reading up the posts from Mark Otto, Ian Feather, Chris Coyer and others about CSS on the projects they work on. I’ve personally learned a lot from those individuals and picked up some more tricks in those posts about how CSS works on large projects.
This is a little bit about how CSS works at Hootsuite and process we have found work best for us at this point in time. Read More →