The biggest idea in frontend development today is DOM as a function of state. It’s a game-changing concept that proved particularly effective for Hootsuite streams, which are essentially a function of social network data and a user’s interaction with that data.
Today, Hootsuite streams are built in React and use Flummox to manage application state. But that’s just an implementation detail – the core of our product lies in the way we organize our data and the functions that transform it into views. In this article, I’ll be presenting a glimpse of the application state driving Hootsuite Streams and more generally, things to consider when designing global state trees for functional frontends such as React.
Transitioning from a monolithic application to a set of microservices can help increase performance and scalability, but it can also drastically increase complexity. Layers of inter-service network calls for add latency and an increasing risk of failure where previously only local function calls existed.
Jim is a Senior Engineer on Hootsuite’s Platform team. When he’s not working on building a rapidly expanding collection of Scala/Akka micro services, he likes cycling and writing/playing music. Follow him on Twitter @jimriecken.
“It’s 10:00PM, do you know where your microservices are?”
During our SOA transition at Hootsuite, we have noticed that visibility into our service relationships, dependencies and status is paramount to keeping our team, our build pipeline and application running smoothly.
In this presentation I’d like to share with you an API we baked into our Service Oriented Architecture that enables us to explore our applications service dependency graph in real time.
Can you guess what these two scenarios have in common?
Imagine seeing Google or Facebook working on ways to provide internet connectivity to remote regions, and then launching project Seed—an off-the-grid web server that solves the same problem on a smaller scale.
Jenkins plugins allow you to extend the Jenkins build system to do almost anything. Jenkins in Italian? Sì. Replace Jenkins with Chuck Norris? You bet. Almost every feature used in building code on Jenkins is a plugin – in fact, even things like LDAP logins can be handled through the Jenkins plugin system. However, Jenkin’s near-limitless customization is also its drawback: with so much possibility, there is also an inherent complexity.
Jenkins plugins work by creating or extending an extension point, which hooks into a specific part of the build process. The catch: these extension points can differ significantly in what they do and how they work. In this post I specifically talk about implementing a Notifier plugin, as it showcases how you can use multiple extension points in a singular plugin while not being significantly more complex than the HelloWorldBuilder plugin described in the official Jenkins plugin tutorial.
At Hootsuite, RxJava and RxAndroid are some of the core libraries we use when building apps for Android. They provide us with a powerful API for writing beautiful and functional asynchronous code. However, the asynchronicity that these libraries provide can also make it more difficult to write unit tests. Many common ways of testing asynchronous code like thread sleeps or using a CountDownLatch are inconsistent and messy, especially when applied to RxJava. In order to avoid these problems, we must look deeper into the RxJava API.
There are two main components to test independently in an asynchronous system: the process submission and the process execution. In RxJava this means that we must test both creation of Observables and the execution of Subscribers. Fortunately, RxJava (and RxAndroid) come with several tools to make this easier.
Terraform is a tool developed by HashiCorp that allows you to build your infrastructure using code. At Hootsuite, we are using Terraform to build new infrastructure in an auditable and maintainable way.
Terraform makes spinning up infrastructure less painful and making changes less scary. By describing infrastructure as code, spinning a new server turns into submitting a pull request and rolling back to a previous state of infrastructure becomes as easy as reverting a commit. Terraform is not limited to AWS, it can provision a whole suite of AWS products and can integrate with a growing list of providers including Digital Ocean, OpenStack and more.
Using Terraform as a systems developer is a good start for remodelling your infrastructure as code. However, to really scale you need to be able to have multiple people work on your terraform stacks. This problem is solved by Terraform’s remote state. Remote state allows you to store the state file for a stack in some third party storage provider so it can be shared across developers. This is in contrast to what you would have if you did not use remote state, that is, each developer with their own statefile. As you could imagine, this would get really messy as people start to clobber each other’s changes.
Terraform provides users with a couple of options when it comes to remote state backends including: S3, Consul and HTTP. S3 is a particularly interesting backend to use since you can version the contents of buckets. Conceivably, then you could also version control states of your infrastructure.
Read More →
Up until recently, loading images on Hootsuite’s Android app had been done via Volley, an asynchronous networking library. We now use a library purpose-built for image loading and caching called Glide. Given the recent change, it seemed worthwhile to examine first, what pain points Glide addresses and then do a walkthrough of how it’s used.
Why the change?
Volley is an HTTP library and that handles the main pain points surrounding loading images – namely, networking and caching – but is not made solely for that purpose. The problem? Loading a remote image into an ImageView can get verbose. Glide can do the job in one line and is a nice wrapper for an HTTP client. Glide uses HttpUrlConnection as its default network stack, but can be set to use Volley or OkHttp. Two other key benefits of Glide are, smoother loading when scrolling through a view that contains images (see the example below); and perhaps more importantly, animated GIF support. Viewing and posting media to social networks is central to Hootsuite so using a library that makes loading images a better experience for users and easier for programmers makes a lot of sense.
Read More →
A result of migrating our monolithic system towards microservices in the last couple of years is 20+ internal services (so far). Many of these services lacked great API documentation, the kind where our developers could easily grasp all endpoints, behaviours, and error codes. We needed three things for our API documentation: a common and easy way to document each endpoint of every service, a centralized location to access all this documentation, and a way to automatically update after changes to an endpoint. Swagger to the rescue.
Swagger is a specification and complete framework implementation for describing, producing, consuming, and visualizing RESTful web services. It also comes with Swagger UI, a great documentation viewer for endpoints. It lists endpoints and details of each one, and also provides a UI that can actually run test queries against test servers. Another bonus is that Swagger documentation is done with annotations in source code and so reduces overhead of managing multiple document sources. Sounds good!
Swagger documentation library for Play framework? Check.
Swagger documentation library for arbitrary Java web apps? Check.
Swagger documentation for endpoints in general for Scala? Nope.
No Swagger for Scala endpoints? OK, let’s help. We made sbt-swagger Scala Library as an SBT plugin. Our sbt-swagger sbt plugin generates swagger-ui compatible JSON data based on the Swagger & JAX-RS (jsr311) annotations in your code. Any Scala applications that provide APIs can benefit from this plugin.
Today, we released it as open source. We have been using sbt-swagger in our products for documenting, exclusively for our internal Scala microservice component that provides our internal protocol over ZeroMQ.
Add sbt-swagger config/dependency in your SBT build file
Add docs in your Scala code with JAX-RS (jsr311) annotations
Hootsuite is no stranger to tearing down monoliths. In fact, over the past year we’ve built fifteen microservices to go along with the deconstruction of our monolith. The extraction of each microservice came with its own complications, but they have all generally followed one development blueprint.
A monolith is the result of legacy code becoming fossilized after years of coding with ongoing contributions from multiple people. This makes the code incredibly hard to refactor as it is always in use, and always depended on. In this case, microservices are used as a tool to tear apart the monolith to address the problem of unreliability. However, a microservice in general is not limited in its purpose or functionality.
Scoping & Prioritization
The first major decision when tearing down your monolith is which part to break off first. Do you want to slice it horizontally or vertically, julienne it or dice it? At Hootsuite, before writing any code, all business objects and use cases were outlined to determine what might make a good microservice. Models that frequently interact with each other were carefully grouped so that a tangled web of distributed services would not be created.
It is clear that business objects like a Team and a Member of a Team should be migrated together. However, if each Team must belong to a single Organization, then it would make sense to also pull out the Organization object instead of leaving it inside the monolith.
Read More →