Lightning Talks: Bring your Organization Closer Together in 5 Minutes

Written by: on September 19, 2014 Posted in:

Reposted from Hootsuite Blog

Listening to Lightning Talks

After attending Vancouver’s Polyglot Unconference in May 2013, Beier Cai and Rahim Lalani approached me with an idea: kickstart short, unconference-style talks on technical topics within our Engineering group. I loved this riff on the traditional lunch-and-learn and decided to set a short time limit of just 5 minutes per talk.

Why so short? Short is all you need to start a conversation. Short also defines our modern day attention spans, and short is hard because this constraint requires you to think critically and creatively about how to convey only essential information. We started out with only technical topics but soon expanded to include non-technical topics and now include speakers from other parts of our organization as a way to expose our audience to different roles and functions. Continue…

Our Experience with Building a RESTful Data API using Play

Written by: on September 17, 2014 Posted in:

At Hootsuite, we analyze user data to help us answer questions about product usability and give us insight into trends. Recently, we designed and implemented a RESTful API that provides access to our aggregated data sets and shields our data consumers from changes in infrastructure that might be happening in the backend. We built our RESTful API using the Play Framework with Scala and we believe it was a great decision for multiple reasons.

Currently, all of our logs are stored in HDFS and S3. We use Apache Spark to aggregate raw logs into more meaningful and usable data sets. For instance, one of our Spark jobs calculates an aggregate count of Hootsuite’s unique daily users across all our different platforms and plan types.

The aggregated data sets are stored in a central RDBMS and are used for data visualizations on dashboards around the office and reports which serve multiple internal stakeholders. Since we are continuously experimenting with different data processing pipelines and data stores, we decided that it would be necessary to provide a constant and defined way to access our data through an API. Providing an API also makes it easier to clearly communicate which data sets are available and makes documentation a much easier process.

We came across multiple features that demonstrated the advantages of using Play to design and implement our API: Continue…

24/7: The Evolution of On Call at Hootsuite

Written by: on September 4, 2014 Posted in:

Software-as-a-service organizations live and die on service reliability and uptime. Having an “on call” team (formal or otherwise) that can quickly react to incidents and outages is mission critical to the business. Hootsuite is no different – our customers around the globe rely on Hootsuite.com being available 24/7 – but even software engineers need sleep. So, who holds down the fort if the unthinkable happens outside business hours?

This year we overhauled our On Call practice to version 3.0. This post walks through the evolution of our On Call model: the stark reality of 1.0, the noble intentions of 2.0, and the practicality of 3.0.

On Call 1.0

Our first On Call system was informal and happened reactively, out of necessity. A handful of original software engineers and operations engineers who built Hootsuite.com were unofficially on call all. the. time. No sleep allowed, no knowledge transfer, no metrics, and no visibility into incidents or issues or remedies. Continue…

Distributed Coordination with ZooKeeper

Written by: on August 27, 2014 Posted in:

For my 8-month co-op term at Hootsuite, I have been working on the Streaming team. The Streaming infrastructure allows services to receive real-time updates for a large user set. Streaming is responsible for delivering push notifications, analytics, and interaction history to subscribed Hootsuite users. One example is receiving a push notification on your mobile device for a Twitter retweet or mention from the Hootsuite application. Streaming is a distributed system and every complex distributed application needs a coordination and orchestration system of some sort, so the team decided to integrate ZooKeeper into the Streaming infrastructure back in 2011. ZooKeeper is a software project of the Apache Software Foundation, providing an open source distributed configuration service, synchronization service, leader election, and naming registry for large distributed systems.

One of my projects during my co-op term included fixing the previous implementation of ZooKeeper, as there were a number of things that were not properly working. Originally, ZooKeeper was embedded within the Streaming application, and there were a number of issues with this implementation.

Continue…

Life as a Hootsuite Co-Op

Written by: on August 14, 2014 Posted in:

In 8 months, I’ll be graduating from the University of British Columbia with a degree in Physics and Computer Science. I’ll be looking for jobs shortly thereafter, armed with a résumé to convince future employers that I am an amazing iOS developer. Hootsuite will be at the top of my ‘Work Experience’ section, along with a list of the various technologies I used and the tasks I was assigned. It’ll probably look something like this:

Hootsuite – iOS Developer Co-op
January – August 2014

Worked with Xcode 5 and 6, Objective-C, Swift, and iOS 6, 7, and 8. Contributed to new features and bug fixes for the Hootsuite app, and helped it become featured in the iOS App Store. Gave presentations, collaborated with teammates, and wrote automated tests using KIF. Used Facebook, Twitter, and Google APIs, communication patterns such as KVO, delegation, and notification, and Interface Builder with auto-layout.

Continue…

Automate everything – startup and shutdown the apps you need in seconds

Written by: on August 7, 2014 Posted in:

Do you hate coming to the office, firing up your favourite text editor, getting ready to do some work, and .. you forgot to start your Vagrant machine? Now you have to open up the terminal, type vagrant up, wait 90+ seconds for the beast to load and .. it doesn’t work because you forgot to connect to the VPN and Puppet cannot correctly provision the box without it. So, you do that too.

Now that everything is setup – you code the entire day, pack your things, arrive home, start browsing threads on Hacker News and… the low battery warning comes up because you forgot to close Vagrant.

Continue…

CSS at Hootsuite

Written by: on August 5, 2014 Posted in:

We’ve really enjoyed reading up the posts from Mark Otto, Ian Feather, Chris Coyer and others about CSS on the projects they work on. I’ve personally learned a lot from those individuals and picked up some more tricks in those posts about how CSS works on large projects.

This is a little bit about how CSS works at Hootsuite and process we have found work best for us at this point in time. Continue…

DOMObserver – react to changes in the DOM

Written by: on July 15, 2014 Posted in:

What: DOMObserver – a JavaScript library for easily tracking DOM mutations.
Code: https://github.com/uberVU/dom-observer/
Demo: http://ubervu.github.io/dom-observer/

Here at Hootsuite, we love to open source components that we develop for ourselves and might be useful to others. We feel that this is a great way to help to give back and contribute to the Open Source ecosystem.

While performing some refactoring on our fronted code, we managed to isolate a component that we use for observing DOM changes, as well as reacting to those changes. The reason we developed this code was that alternatives such as mutation-summary did not do all we wanted, and even now, they don’t offer fallback support, which we need since we support older versions of IE as well.

Enter DOMObserver, a JavaScript library that lets you observe and react to DOM changes in an easy and friendly manner. It uses either the new MutationObserver API or the older Mutation Events API, whichever the browser supports, and offers you control in an unified way. We use it for managing the widgets we show on a page, but you can use it for any changes to the DOM that you care about.

DOMObserver exposes an API that lets you set handlers for when:

  • a node is added to the DOM
  • a node is removed from the DOM
  • a node’s attributes are changed

The handlers receive the mutation target as a parameter, so you know which node changed. Furthermore, if you want to do more advanced things, the handlers also receive the mutation (event) object, so you can tell exactly what has changed.

As far as performance goes – you can set the handler on a specific DOM node, so there is no need to listen on the whole document. Also, for attribute changes you can filter the attributes you care about (so for example you only get notified when data-loaded is changed, but not for src changes). When you are no longer interested in the DOM changes (say your app reaches a constant, immutable state), you can simply shutdown the observer.

Check out the demo to get a hands-on feel of what DOMObserver can do for you, and make sure to check out the README file on GitHub for more information.


uberVU is part of the Hootsuite team. Read more about it here

GridList – building responsive dashboards with resizable widgets

Written by: on July 6, 2014 Posted in:

What?

[GridList][1] is a JS library for creating two-dimensional, resizable and responsive lists that we built to help create highly customisable dashboards. The library is used in one of our flagship features we call [Boards][2] and was developed with the needs of this project in mind.

Why a grid?

Our strongest point is our data, this is what we do best, collect and analyse great volumes of data. Data alone is not enough. It is critical to show the result of all the effort of collecting and analysing in an easy to comprehend way. We needed a way to make it easy to discover patterns and take action on them.

We needed a data visualisation tool that would be:

  • easy and intuitive to use;
  • easy to customise and manage;
  • responsive – meaning that it will look great no matter what screen it is shown on, a mobile phone or a big 4k display;

A dashboard is a way of visualising data that, if properly built, would tick all the boxes of our requirements. We are passionate about dashboards and went through a few iterations and versions of displaying data in a dashboard form to understand what makes a dashboard great.

Given all of these it was clear that we needed to use a grid system that supported drag and drop. We had two options, find and adapt an existing one or make our own. Continue…

Github-changelog – robust notification system built on top of github

Written by: on June 24, 2014 Posted in:

What is it?

Github-changelog provides a mechanism to communicate to the users of your web app that updates are available and that they should reload to see the changes. Github repository: Check it out here
Demo: See the demo

The source of the need

As developers, most of us strive to reach the Holy Grail of continuous deployment, so that any member of the team can push fixes and features as soon as these are built. Mean and lean. For most of us gone are the days of huge releases, replaced by an unpredictable and continuous flow of fixes and features. And these can be many. Etsy, a company we admire for their well oiled deployment system, managed to have 517 individual deploys in a single month. Although we are not quite at that scale yet, we manage to have 15+ daily deploys on a productive day. Continue…