At Hootsuite, RxJava and RxAndroid are some of the core libraries we use when building apps for Android. They provide us with a powerful API for writing beautiful and functional asynchronous code. However, the asynchronicity that these libraries provide can also make it more difficult to write unit tests. Many common ways of testing asynchronous code like thread sleeps or using a CountDownLatch are inconsistent and messy, especially when applied to RxJava. In order to avoid these problems, we must look deeper into the RxJava API.

There are two main components to test independently in an asynchronous system: the process submission and the process execution. In RxJava this means that we must test both creation of Observables and the execution of Subscribers. Fortunately, RxJava (and RxAndroid) come with several tools to make this easier.

rxjava and junit Read More →

Terraform is a tool developed by HashiCorp that allows you to build your infrastructure using code. At Hootsuite, we are using Terraform to build new infrastructure in an auditable and maintainable way.

Terraform makes spinning up infrastructure less painful and making changes less scary. By describing infrastructure as code, spinning a new server turns into submitting a pull request and rolling back to a previous state of infrastructure becomes as easy as reverting a commit. Terraform is not limited to AWS, it can provision a whole suite of AWS products and can integrate with a growing list of providers including Digital Ocean, OpenStack and more.

Using Terraform as a systems developer is a good start for remodelling your infrastructure as code. However, to really scale you need to be able to have multiple people work on your terraform stacks. This problem is solved by Terraform’s remote state. Remote state allows you to store the state file for a stack in some third party storage provider so it can be shared across developers. This is in contrast to what you would have if you did not use remote state, that is, each developer with their own statefile. As you could imagine, this would get really messy as people start to clobber each other’s changes.

Terraform provides users with a couple of options when it comes to remote state backends including: S3, Consul and HTTP. S3 is a particularly interesting backend to use since you can version the contents of buckets. Conceivably, then you could also version control states of your infrastructure. Read More →

Up until recently, loading images on Hootsuite’s Android app had been done via Volley, an asynchronous networking library. We now use a library purpose-built for image loading and caching called Glide. Given the recent change, it seemed worthwhile to examine first, what pain points Glide addresses and then do a walkthrough of how it’s used.

Why the change? 


Volley is an HTTP library and that handles the main pain points surrounding loading images – namely, networking and caching – but is not made solely for that purpose. The problem? Loading a remote image into an ImageView can get verbose. Glide can do the job in one line and is a nice wrapper for an HTTP client. Glide uses HttpUrlConnection as its default network stack, but can be set to use Volley or OkHttp. Two other key benefits of Glide are, smoother loading when scrolling through a view that contains images (see the example below); and perhaps more importantly, animated GIF support. Viewing and posting media to social networks is central to Hootsuite so using a library that makes loading images a better experience for users and easier for programmers makes a lot of sense. Read More →

The Problem

A result of migrating our monolithic system towards microservices in the last couple of years is 20+ internal services (so far). Many of these services lacked great API documentation, the kind where our developers could easily grasp all endpoints, behaviours, and error codes. We needed three things for our API documentation: a common and easy way to document each endpoint of every service, a centralized location to access all this documentation, and a way to automatically update after changes to an endpoint. Swagger to the rescue.

Swagger

swagger

Swagger is a specification and complete framework implementation for describing, producing, consuming, and visualizing RESTful web services. It also comes with Swagger UI, a great documentation viewer for endpoints. It lists endpoints and details of each one, and also provides a UI that can actually run test queries against test servers. Another bonus is that Swagger documentation is done with annotations in source code and so reduces overhead of managing multiple document sources. Sounds good!

  • Swagger documentation library for Play framework? Check.
  • Swagger documentation library for arbitrary Java web apps? Check.
  • Swagger documentation for endpoints in general for Scala? Nope.
sbt-swagger

No Swagger for Scala endpoints? OK, let’s help. We made sbt-swagger Scala Library as an SBT plugin. Our sbt-swagger sbt plugin generates swagger-ui compatible JSON data based on the Swagger & JAX-RS (jsr311) annotations in your code. Any Scala applications that provide APIs can benefit from this plugin.

Today, we released it as open source. We have been using sbt-swagger in our products for documenting, exclusively for our internal Scala microservice component that provides our internal protocol over ZeroMQ.

Using sbt-swagger

Three steps:

  1. Add sbt-swagger config/dependency in your SBT build file
  2. Add docs in your Scala code with JAX-RS (jsr311) annotations
  3. run sbt swagger
Read More →

Hootsuite is no stranger to tearing down monoliths. In fact, over the past year we’ve built fifteen microservices to go along with the deconstruction of our monolith. The extraction of each microservice came with its own complications, but they have all generally followed one development blueprint.

A monolith is the result of legacy code becoming fossilized after years of coding with ongoing contributions from multiple people. This makes the code incredibly hard to refactor as it is always in use, and always depended on. In this case, microservices are used as a tool to tear apart the monolith to address the problem of unreliability. However, a microservice in general is not limited in its purpose or functionality.

Hootsuite's Microservices

Hootsuite’s Microservices

Scoping & Prioritization

The first major decision when tearing down your monolith is which part to break off first. Do you want to slice it horizontally or vertically, julienne it or dice it? At Hootsuite, before writing any code, all business objects and use cases were outlined to determine what might make a good microservice. Models that frequently interact with each other were carefully grouped so that a tangled web of distributed services would not be created.

Example

It is clear that business objects like a Team and a Member of a Team should be migrated together. However, if each Team must belong to a single Organization, then it would make sense to also pull out the Organization object instead of leaving it inside the monolith. Read More →

Working at Hootsuite has helped enforce one ideal: security matters. In Hootsuite’s continuous implementation and deployment environment, our developers take security to heart and this is reflected in our code base. Working hand-in-hand with our developers, the security team strives to improve by staying updated and evolving with latest practices. As part of our commitment to staying relevant, we continuously look for tools that may simplify or remove our pain points. One of these pain points is the management of the large amounts of code flowing through the pipeline, and ensuring that they reach our security standards: with thousands of lines of code coming through every day, how do we guarantee those standards are met?

Read More →

Hootsuite Co-ops

The Story of Ira Needles

When Ira G. Needles arrived in the Waterloo region in 1925 to take a job as assistant sales manager at B.F. Goodrich, he hid the fact that he was university educated. At the time, the business world considered it “snooty” to have a higher education.

His education didn’t hurt him, however, and Needles gradually rose within the ranks of the tire giant, and by 1951 he was appointed president of B.F. Goodrich Canada.

However, in the summer of 1956, Needles’ two separate worlds – industry and academia – would finally come together in a radical speech he made to the Rotary Club of Kitchener-Waterloo. Needles’ speech would ultimately transform the nature of education in Canada.

During the talk, entitled “WANTED: 150,000 Engineers – The Waterloo Plan,” Needles presented a new kind of education that would involve studies in the classroom as well as training in industry.

Courtesy of the University of Waterloo public library file on Ira Needles

Thank Goodness for Ira

Needles and two others would then go on to found the future University of Waterloo (first known as the Waterloo College Associate Faculties) in 1957 and admit the first 75 students, all of whom were also co-ops. Sixty years later, roughly 80,000 students in Canada enroll in co-operative education each year.

A co-op program has, in my mind, no downside for students, employers, or the institutions that support it. Students apply their learning, test-drive life in industry, fund their education, and return to their studies to challenge some of the theoretical principles based on this experience. Employees, like us, get an opportunity to cement our own knowledge by teaching students and at the same time build a pipeline of young, bright talent. Lastly, demand for an academic institution’s programs grow as the demand for their graduating students grows.

How Do You Measure the Success of a Co-op Program? Read More →

Love technology? Are you a high school student in grade 11 or 12, or home schooled? Do you live in BC? 

Hootsuite’s technologists would love to help you develop your technical skills and to provide you with experience around people, process, and technology by working with you over the summer. You’ll pair with a mentor and work at our Vancouver office side-by-side with a passionate, egoless team having fun building something bigger than themselves. Experience what it’s like to make a difference in people’s lives by building the products our customers use to turn messages into meaningful relationships.

There are opportunities at Hootsuite in all aspects of technology including software (both mobile and web), operations, security, and IT.

This is a paid position with a competitive salary.

Application instructions are at the end of this post.

This is the second summer of this program. You can meet the High School students who worked with us last year and read about their stories and accomplishments.

Passive learning creates knowledge. Active practice creates skill.James Clear

Photo courtesy of @alexrousse_ (instagram)

Read More →

The Platform team at Hootsuite creates and manages microservices and is always obsessive over response times. We write all of our code with that as a guideline, so users can get the best experience. To achieve this, we rely on many cool open source projects like Scala and Akka, and do our best to give back to the communities whenever we can. In keeping with that, we are happy to announce we are open sourcing our Scala circuit breaker.

Circuit Breakers, Responsiveness, and Complexity

So, what is a circuit breaker? A circuit breaker in the context of a service helps deal with situations in which a service’s dependency (can be a DB, another service, a web site, a toaster, etc) stops working as expected and causes slow interactions with end users. The circuit breaker helps mitigate the problem by detecting slow or failed dependencies, and stops the sending of further requests. This fail-fast behaviour informs the users that there is a problem quickly, as well as lowering the resources allocated to communicating with the broken dependency. After being tripped, the circuit breaker should not remain tripped, or the dependency would be quarantined forever. To mitigate this situation, the circuit breaker can periodically retry the dependency and/or wait for predefined amount of time to see if the dependency is available again. Without the circuit breaker, the server would continue to try to use the unavailable dependency, and users would see pages timing out or returning an error after many seconds.

Scala circuit breaker

Read More →

How do we share our libraries internally?

In 2015, our Android team released v3 of Hootsuite’s app with a new UI following Material Design guidelines. We also gave users the Publisher feature, Instagram integration and enhanced Twitter Search. The development of these features relied on many popular open source libraries and tools, such as RxAndroid, Retrofit, Google Design, Robolectric, and many more. With the success of these libraries, our Android development team was inspired to build our own libraries to allow other teams to consume. A great idea, with a surprising complication: How do we share our library internally?

Publishing libraries to Maven Central using Gradle has been covered before by Chris Banes and others. His solution is visualized in the following diagram.

Chris Banes Maven Gradle

Our need was to be able to publish libraries to a private Maven repository like Artifcatory. We tweaked @chrisbanes‘s solution and made it available to our developers to enable them to write libraries independently and share them between teams.

Hootsuite Maven Gradle

In this post, I’m going to guide you in setting up your library to share your own Maven repository.

The Template Library Project

Most Android library projects share a similar directory structure, so we prepared a Library Template that will act as the reference for this guide. We encourage you to fork this project if you’re starting fresh.

Read More →