Accelerating cross platform development with serverless microservices

For products that are delivered on the Web, iOS and Android, there have been myriad approaches to reducing development time and the duplication of work. Sharing code between major platforms has in some ways felt like the search for the holy grail – hope remains, but as of today, has yet to be found. As long as there have been multiple platforms, there has been a desire to write once, deploy everywhere. Developers are obsessed with efficiency by nature and nothing feels more inefficient than developing the same thing twice. In this post we look at accelerating cross platform development by harnessing serverless microservices. Based on our own experiment with AWS Lambda, we will cover the motivations, benefits, and drawbacks as well as continuous integration and delivery of this approach.

Strategies for sharing tried in the past

Leading up to today, we’ve spent time researching, prototyping and building cross platform, shared source solutions using a variety of approaches.

Way back, we started off by sharing Web views. In many cases, they lacked responsiveness parity with native views, and so were perceived as a degradation in user experience. The code was often difficult to maintain, and the Web view didn’t offer integration with native OS capabilities. They worked for single views or sub-views, but they often weren’t a solution for an entire app. Our core Hootsuite app, as of today, still has some of these embedded Web views.

Next we built using Cordova/Phonegap, which essentially wraps a Web view and enables us to have access to the most popular OS capabilities such as camera and Bluetooth. Over time, with smart optimizations, we could almost achieve native performance but we had to put in the work. Some of the apps Hootsuite has in the market right now are built this way.

We prototyped with React Native, which produces native interfaces orchestrated by Javascript. We ended up spending a lot of time on interface layouts and troubleshooting performance problems having to do with limited threads. In the end, we could get a result which was indiscernible from purely Native but it involved hair pulling and riding the bleeding edge of the open source project. It didn’t feel natural, or safe, and we felt like we were swimming upstream. We ended up shelving our React Native experiments without releasing them to the public.

We looked at using Xamarin, to write once and deploy to both iOS and Android, but we didn’t get the benefits of leveraging our large Web team or our existing iOS and Android teams. C# as a language alternative to Obj-C or Java is reasonably attractive but as a platform it felt optimized towards a completely shared user experience rather than one optimized to feel right for both Android and iOS users. Other than keeping up with latest developments, we didn’t invest in any substantive way in the Xamarin approach.

Lessons learned, the bar moving forward

After a huge amount of effort, some successful deliveries, and also a waste bin full of prototypes, what had we learned?

We learned that we don’t want to be forced to drop back into Javascript. It’s nice to have multiple options when it comes to programming language. For many in our team, returning to a language without type safety was tough to stomach.

We learned that we need the performance of native. There is no tolerance in the marketplace for apps that feel clunky or make you wait.

We learned that too much focus was being placed on sharing code at the user interface layer. Developers who choose cross platform development typically have the expectation that the entire app codebase will be shared. We also know that the lowest tolerance for performance degradation is at the user interface layer. Users want to feel the app responding to their interactions in near real time.

We learned that we don’t want development to feel like a fight. It should feel natural. The question we all ask after trying a new cross platform tool is “would it have been faster to simply write the app twice, once for each platform?” If you spend too much time trying to optimize for the cross platform development environment, it can feel like you’re doing something wrong.

We learned that we want to use IDEs and have powerful and easy debugging tools. Many cross platform tools make it very difficult to do simple things like set breakpoints and step through your code. IDEs developed by the same company as their respective native platform have an unfair advantage. Apple, as example, adds additional support to XCode with each release of its OS. 3rd party tools like Appcode often lag behind.

The experiment

Considering our learnings, and compelled by the ease of deployment and attractive pricing of the serverless Amazon Web Services (AWS) Lambda offering, we decided to try another cross platform experiment. We decided to apply our approach of microservices and services oriented architecture (SOA) from our non-mobile products to our native and mobile Web apps. We decided to focus lower down the stack, and avoided the user interaction layer. Our Android team blazed the trail, by writing and testing locally first. Once the first iteration was ready, we moved into the cloud and then integrated with iOS. In the future we’re looking to exposing the Lambda function to our Web version.

The problem we solved

Our AWS Lambda experiment is tied to our recently released feature, query builder. Query builder allows users to use Twitter’s advanced search features in a clean and intuitive interface. Without an abstraction to build a query, a search for all tweets with attached media and the words “cat” or “dog”, and the word “cute” would need to be typed in as:

(cat OR dog) cute filter:media

The above is only a simple example, our customers twitter searches can be a dizzying sight, hundreds of characters long, full of brackets, boolean operators, and keywords. Query builder allows the user to abstract away from search syntax to build a search graphically.

To facilitate the conversion of the search string that Twitter uses to the graphical user interface and vice versa, we created a JSON model to act as an intermediary. Conversion between JSON and UI is relatively easy and is platform specific. This conversion is performed on each client, iOS and Android.

Layers of Hootsuite Query Builder
An image showing the three layers of query builder: the UI, the JSON model, and the Twitter search string.

Unlike the conversion between the User Interface and the JSON Model, the conversion between JSON Model and Twitter search string can be quite complicated.

Converting to a Twitter search string from the JSON model is fairly easy since we are converting structured data into unstructured data. However, the parsing of the Twitter search string is much more complicated, converting unstructured data into structured data. Our solution uses ANTLR to describe a context-free grammar of Twitter searches. It then parses and builds the JSON model from the search string.

Look for a forthcoming blog post for more details on how we solved that problem. For the purposes of this blog post, however, our function converts from our JSON model to a Twitter search string and vice versa.

What has the development experience been like?

Coming from Android development, creating an AWS Lambda function is a breeze. Android development is currently stuck in purgatory, somewhere between Java 6 and Java 7. With AWS Lambda, you can develop using Java 8. This means that you can use native Lambda expressions rather than the Retrolambda backport commonly used in Android development. Practically speaking, choosing Java for Lambda function development means that an Android developer can use the language they’re already familiar with to get up and running quickly.

One thing that can slow development down is the deployment steps. Rather than simply compiling and running some code, or deploying to a mobile device, Lambda functions need to be deployed in AWS. The code first needs to be compiled into an artifact (zip or jar file). This artifact then needs to be uploaded to AWS Lambda. It can be uploaded directly, or in our case, pushed to AWS Lambda through an Amazon S3 bucket. Finally, to actually invoke your Lambda function, you’ll need some sort of client. This can be achieved using the Java SDK or the Mobile SDK. If all of this sounds daunting, it is. Thankfully, putting in the work early to automate these steps pays dividends and prevents development from getting into deployment hell.

In our deployment of AWS Lambda, we’ve used a number of tools to automate deployment. First, the uploading of the artifact to S3 is handled with a single gradle task. Gradle is a build tool used in Android which makes managing dependencies easier, it also has many plugins, several of which upload files to S3. To manage the deployment of the Lambda code, we use Gradle tasks invoking boto scripts. All of this is covered in the “Automation of delivery” section below.

The Gradle tasks that we’ve created to perform these functions are invoked from our Jenkins Continuous Integration (CI) server. After code is reviewed, a merge into master triggers Jenkins to run a series of tasks and tests which results in new code in AWS Lambda with no further dev action. Easy.

As developers, grepping through log files is one of the first actions we take to debug our code. With normal Java or Android code, this is easy as your IDE usually provides a log output. With AWS Lambda however, there is a little extra work involved. Thankfully, Amazon Cloudwatch provides a way to monitor your Lambda functions. You can configure any number of metric alarms to trigger when your Lambda throws errors, exceeds a usage limit, etc. You can also pore through Cloudwatch logs to see exactly what is going wrong with your Lambda function.

One of the striking differences between the serverless AWS Lambda and a solution with a server, an EC2 instance for example, is the lack of management required. After you have the deployment automated, there is little else to do. Lambda functions only require you to set the amount of RAM you want to provision by the function as well as the number of seconds your function should run before timing out. There are no file permissions, no OS package management to worry about, no security patches to apply. If more people run your function concurrently – Amazon handles it, and if nobody does, no problem, you’re only charged per use.

The Android developer

As mentioned above, coming from Android development, AWS Lambda lets you finally try out those Java 8 features you’ve been hearing so much about. Beyond that, the AWS Android SDK allows the easy creation of a client to invoke Lambda functions. In code, they are called just like any other function meaning you don’t need to use any networking library to invoke the Lambda, it’s all handled by the SDK.

As described in “The experiment” section above, we are building a parser for our “query builder” feature. A user creates a Twitter search graphically on each of the mobile apps (iOS and Android). Each client then converts the UI to a JSON model. This model is then converted to Twitter search string syntax through the AWS Lambda function. A user can also load a saved search to be modified. To do this, the client sends the existing search string to the Lambda function which is converted to the JSON model and sent back to the client which then shows the model as UI.

Below you can see some Java code declaring an AWS Lambda function. We have two Lambda functions, one in a Staging environment, and the other in Production. The data structure that we use is the same for both request and response, LambdaDataTransferObject.

The above code is all that you need to declare a Lambda function in Android. The @LambdaFunction annotation works with the Android SDK to mark the method as a Lambda function. The qualifier that is passed to each annotation is the alias of the Lambda which will be invoked (more on that in the “Delivering to multiple app versions” section below).

Since we have two Lambda functions, we use another class to choose in which environment to invoke the Lambda function. This isn’t strictly required but allows us to switch between Lambda environments at runtime, useful for testing without reinstalling the app.

The above class simply provides a layer of abstraction away from the Lambda function so that the caller of the invokeLambda method doesn’t know in which environment the Lambda function is being invoked. In addition to the environment, this class requires a LambdaInvokerFactory instance. This instance is used to implement the Lambda interface in the previous code example. Below is the class which is used to call our function and which provides the LambdaInvokerFactory instance.

Recall from the “The experiment” section above, our Lambda function converts Twitter search parameters (query string, location, language) into a model of the query we use internally and vice versa. Thus there are two functions below which invoke the Lambda, controlling the direction of the conversion.

In the above code, you can see that the only things needed to invoke a Lambda function on Android is the interface as previously shown along with:

  1. AWS credentials to invoke the function.
  2. A LambdaInvokerFactory instance specifying the Android context and AWS region of the Lambda function.
  3. The aforementioned AwsLambdaEnvironmentInvoker which lets us choose which environment in which we invoke the lambda function.
The two methods, generateQueryObject and generateSearchParams each invoke the Lambda function to perform the conversion between SearchParams and Query.

In using the Android SDK, you can easily combine AWS Lambda invocation with other Android libraries. In the above code, for example, we are using observables from RxJava to handle calling the Lambda function asynchronously.

Automating deployment

As mentioned above, the deployment of AWS Lambda function code can be automated using Gradle tasks and Jenkins jobs. However, those tools only cover maintaining Lambda functions and deploying code to them. The creation of the Lambda function is done using Terraform from Hashicorp. Terraform allows you to create infrastructure using code, making it easy to review and to deploy.

Below is a sample of the HCL (Hashicorp Configuration Language) that is used to create our Lambda function.

The code above is all that is needed to create the Lambda function. We specify the resource type in HCL to be aws_lambda_function which provides a number of parameters to set. Since we are deploying our Lambda through S3, we specify the bucket and key where the initial lambda function code can be found, if we were deploying directly to Lambda we would provide a file name here.

Since we are managing the updating of function code through Jenkins, we need to tell Terraform to ignore if the s3 key ever changes. Since artifacts are versioned, new keys will have a different filename which should be ignored by Terraform.

Next we provide the function name which matches the Java code sample shown above. The runtime is set, Java8, as well as the timeout of the function and the Identity Access Management (IAM) role which will be attached to the function.

Finally, the handler is the fully qualified method name in Java which will be called when the Lambda function is invoked. Setting up is that easy, you can then run your Terraform code and have a Lambda function ready to be invoked.

Beyond the Terraform code for creating the Lambda function, we also use Terraform to create the SNS topic for our function as well as the Cloudwatch alarms which will be sent to that SNS topic.

As mentioned above, we use Jenkins to maintain and deploy Lambda code. To allow this, we also create the Jenkins user and its attached policies with Terraform. That policy will need to include these commands which are invoked through boto. In order to automate the manual steps, we use the create-alias, update-alias, publish-version, and update-lambda-code commands among others. How we use them to properly deploy and version our Lambda function is described in the “Automation of delivery” section below.

Finally, our Jenkins user needs to be able to run the unit and end-to-end tests so the policy also needs to allow for the invocation of the Lambda functions.

What has the performance been like?

Hello World Performance

In order to assess performance, a simple AWS Lambda function was called one thousand times on both iOS and Android, with the round trip time of each request and response being recorded. We were worried about how much delay would be added to the user experience, as well as how much deviation we would see in the delay. The data that was gathered is shown in the figures below.

Execution Times (Android) Execution Times (iOS)

Platform Min. Time (ms) Mean Time (ms) Max. Time (ms) Standard Deviation (ms)
Android 73 120 407 13
iOS 70 127 330 24
From the data above, both Android and iOS had an average total execution time (time from the sending of the request to the receiving of the response) of around 120ms. While there was slightly more variance in the iOS execution times, both are grouped quite tightly around the mean with few outliers. The test was executed on WiFi in Vancouver, with our Lambda function in us-east-1, North Virginia . Our summary was that even in the worst case, the delay was acceptable.

Unit tests vs. end-to-end tests

After implementing the Lambda function for our feature, the function was again tested for execution time. First, 98 unit tests were run locally on the source code 10 times each, and their execution times averaged. Next, those unit tests were run as end-to-end tests through the actual Lambda function 10 times each as well. The results of each test is shown below:

Test Execution Times per Environment

Environment Min. Time (ms) Mean Time (ms) Max. Time (ms) Standard Deviation (ms) Coefficient of Variation
Local 14.3 16.2 18.4 1.3 7.9%
AWS Lambda 610.0 742.3 840.1 70.4 9.5%
Each test suite contained 98 tests, the execution time shown above is per test. While the local tests ran in well below one hundred ms on average, the end to end tests ranged between a few hundred milliseconds up to three seconds, with an average execution of around seven hundred milliseconds. The coefficient of variation shows that the AWS Lambda tests had a slightly higher variance than the local unit tests but still remained relatively constant.

Averaging the above results we can see how much time invoking the lambda function adds in network delays and overhead.

Average Test Execution Times

The average per test execution time was 16.24 ms when run locally and 742.32 ms when run through the AWS Lambda function. Many of these tests use much more complex inputs than the majority of real-world use cases and so a sub-second average execution time is quite good, we consider these tests a success.

Delivering to multiple app versions

The need for multiple versions

When developing mobile applications, it is common to have a portion of your userbase running older versions. It takes time for users to update to the latest version, if they ever do, and support should be maintained while they are on older versions.

At Hootsuite, we manage versioned features through the use of dark launch flags. This gives us the ability to roll-out, and sometimes more importantly, roll-back, code to our desktop and mobile users. However, dark launch flags are just that, flags, they can only turn on and off code that is already built into the app.

With AWS Lambda, our problem was twofold. First, we wanted to make sure that we could set specific versions of our AWS Lambda function for each version of the application. Second, we wanted to be able to do something that even dark launch flags can’t do, fix unforeseen bugs in older versions of our mobile apps. Before delving into how we achieved that, here is a typical use case of a dark launch flag which highlights the problem.

Let’s consider the case of deploying a new feature, the launch of Instagram on our platform for example. While we are developing we put the new Instagram code behind a dark launch flag. Unless the flag is enabled on our server, the user of the app will never see the new Instagram code.

Now, when we’re ready to enable access to Instagram to all our users, we put out a release of the app with all the Instagram code inside. With the release we turn on the dark launch flag for Instagram and users can now use the feature.

If we now see bugs in our Instagram implementation, our only choice is to either roll-back access to Instagram and fix the bugs in a new release, or to allow users to use the buggy version of the Instagram code. What we’d like to be able to do is fix the bugs in older versions of the application without forcing users to upgrade to see the fix. The problem is shown in the diagram below.

Mobile apps locked to version

On the right we have some service that is being invoked from the mobile application, part of the Hootsuite backend. The service is versioned and locked to an app version. After releasing version A of the app and version 1 of the service, some time goes by and new versions, B of the app, and 2 of the service are released.

It’s at this time however, that the bug is discovered in Version 1 of the service. Since application versions are locked to service versions, there is no way to fix the bug for users of version A of the app. Instead we must release a new version of both the app and the service and then wait for users to upgrade the application on their device as shown in the diagram below.

Mobile apps forced to upgrade

This is problematic as before they update, version A users are stuck with a buggy version of the service and experience problems using the app. With AWS Lambda however, this limitation can be overcome through the use of function aliases in addition to versioning.

AWS Lambda versions and aliases

Both aspects of our versioning problem are solved through the use of AWS Lambda versioning and aliases.

Lambda versions allow us to take snapshots of the code in the $LATEST bucket (where all versions of the function start after deployment) and then create an immutable numbered version of the Lambda function. However, coupling these function versions to application versions would only ensure that we could have older versions of the app invoking older versions of the Lambda function. In order to modify which function version a specific app version invokes, we need to use aliases.

Aliases are pointers to Lambda function versions. Unlike versions, aliases are mutable. When a function version is published, an alias can be created which points to that version. This alias is then embedded into the application so that that version of the application always invokes that alias. Now however, in the case that a bug is discovered in an old version of the Lambda function. We can patch the bug, publish a new version of the function and then update the alias to point to the new function version.

Below is a graphic showing the same situation as above but including the aliases offered by AWS Lambda.

Mobile apps avoid lock by using alias

Here we have the same problems as we did before, a bug was discovered with an older version of the Lambda function code (version 1) after a newer version was released. Similar to the previous example, each version of the app is locked to a particular alias of the Lambda function. However, the aliases themselves are pointers to published versions of the Lambda code, and can be changed.

In this case, we can take the code we have in Version 1 of the Lambda function, patch the bug and then publish version 3 of the function. As long as we keep the same request/response structure, we can update alias A to point to the new function version.

Alias allows iteration

When an alias is updated to point to a new function version, the transition is seamless, no application restart is required. The next invocation will call the new Lambda function version with the bug fixed. Through the use of aliases and versions, the Lambda function code is decoupled from the app version. This means that as a developer, you have complete control.

As mentioned previously, the Lambda function API needs to remain the same for version A of the application so that it uses the same request and response structure. However, the Lambda versions are independent from each other, Version 2 and Alias B could have a new updated API.

Automation of delivery

Creating infrastructure

At Hootsuite, we use HashiCorp’s Terraform to build and maintain AWS infrastructure through code. As mentioned in the “What has the development experience been like?” section above, our AWS Lambda experiment uses Terraform to create the Lambda functions, as well as the IAM resources that we use to invoke, update, and maintain the Lambda. To facilitate testing and stability, we created two identical Lambda functions, one in a staging environment and one in production.

Deploying to AWS Lambda

There are a number of unit tests which are run on the source code before the artifact is built to sent to AWS Lambda. After these tests pass, the source code is packaged into a .zip file which is uploaded to an S3 bucket.

The artifact in S3 has to be pushed to AWS Lambda by invoking the update function code command. When this command is invoked, the artifact is sent to the $LATEST bucket of the target Lambda function. From there, we run the same unit tests as end-to-end tests to ensure that communication with the Lambda function is working as expected.

The AWS Lambda Java artifact is built using Gradle. By using Gradle and Jenkins we created a single Jenkins job which is triggered when there is a merge to the master branch of our source code repository. This job uses Gradle tasks which trigger boto scripts and unit tests. The job performs the following actions:

  1. Runs the local unit tests on the source code now pushed to the master branch.
  2. Creates the .zip artifact
  3. Uploads the .zip artifact to the staging S3 bucket.
  4. Pushes the artifact from S3 to $LATEST in the staging Lambda function.
  5. Runs the end-to-end tests on the newly updated code.
After this job runs, the developers can then proceed to perform the same set of actions on the production Lambda. After those two jobs are run, the latest source code is in the $LATEST of both staging and production Lambda functions.

Versioning and aliases

After the code is in the $LATEST of each Lambda function, there are other Jenkins jobs which can be triggered. These jobs also invoke boto scripts and are used to:
  1. Publish a new version from the $LATEST code.
  2. Create a new alias pointing to a specified version.
  3. Update an existing alias to point to a new version of the Lambda function.
Using these three Jenkins jobs, complete control over the delivery of multiple Lambda function versions to multiple application versions can be achieved.

The benefit to using Jenkins jobs to update and maintain the Lambda functions is in security and auditability. Since users have to log into Jenkins, a log of which users triggered which jobs is maintained. Additionally, the credentials used by boto to invoke AWS CLI commands (the same credentials generated earlier through Terraform) are kept on the machine hosting the Jenkins server only. Thus, no developer has the ability to modify the AWS Lambda functions from their own machine. Finally, a record of the output of all the jobs are kept on the machine so it’s easy to see when each version was published, or alias modified.

Management with Jenkins

All the automation of delivery is handled with six Jenkins jobs. A screenshot of our Jenkins dashboard is shown below with those six jobs.

Build dashboard

Moving down through the list, we have:

  1. Creation of aliases: This job is run manually when new aliases need to be created, usually when we release a new version of one of our mobile applications.
  2. Deploying to production: This job is a copy of the staging job below it but it triggered manually by a developer after ensuring that the staging deployment is successful.
  3. Deploying to staging: This job runs every time new code is pushed to master. As mentioned above, it runs local unit tests, creates the artifact, pushes the artifact to S3, pushes the S3 artifact to Lambda, and then runs the end to end tests on the new deployment.
  4. Publishing a Lambda version: This job is run manually to take a snapshot of the code in $LATEST and create a new function version from it. An alias can be created or updated to point to that new version.
  5. Running tests: This job allows developers to specify an environment and alias to run the test suite against.
  6. Updating an alias: This job is run manually to update an alias to a new function version.

Environment variables and serverless application model

AWS re:Invent 2016 introduced many new products and services integrating with AWS Lambda. Two enhancements to Lambda itself are Environment Variables and Serverless Application Model (SAM).

Environment variables

As mentioned earlier in this section, we have deployed two Lambda functions, one for our staging environment and one for our production environment. These environments can each be targeted for running tests and can be invoked from our application. This is handled through Gradle properties and multiple interfaces in our source code.

Amazon has now introduced environment variables in AWS Lambda functions directly. There is a great blog post written on how to use the environment variables to simplify the use of AWS Lambda. While this is certainly a tool that we will be integrating into our Lambda functions in the future, it is not a replacement for having one Lambda function per environment.

Having a separate staging Lambda function means that versions and aliases are managed separately as well. This allows for rapid changes to the staging environment, testing out new configurations before pushing the stable code to the production Lambda.

AWS serverless application model

The SAM released by Amazon is designed to help integrate Amazon API Gateway, Amazon DynamoDB, and AWS Lambda together, allowing them to be configured using a proprietary syntax.

The focus of our AWS Lambda experiment so far has been on invoking a Lambda function using the Android and iOS SDKs. In the future we will be expanding our experiment to include AWS Gateway as mentioned in the “Sharing beyond Android/iOS with Amazon API Gateway” section below. At that time, SAM will likely prove a valuable tool for configuring those services together, we look forward to testing it out.

What parts of an app benefit from this model?

The user interface layer stays native, and doesn’t benefit from this model. The use of microservices is really a best fit for encapsulating business logic or things like aggregating and integrating third party services where you need to be able to adapt to changes outside of your control without having to deploy new versions and orphan those who don’t update their apps. Being cloud based, this model is only appropriate for scenarios where you are able to require network availability. There is a small amount of latency introduced by going to the cloud, so again you’ll need to select interactions that can tolerate or obfuscate the delay. You’re trading off response time and a connectivity requirement for reduction of duplication and maintainability. If thinking of your app as a sandwich, this works best for the what’s between the slices of bread – avoid the OS specific interface and data layers.

What are the upsides to this model?

We’ve found a number of upsides to sharing cross platform using serverless microservices.

Cost

Lambda has no set cost. It is a true pay-as-you-go service model. There are no servers, physical or virtual, required. The cost savings for most periodic workloads like Web requests can be dramatic as compared to AWS EC2 or Digitalocean Droplets. Lambdas are particularly attractive in the context of releasing new features or products where the usage may grow gradually over time or where usage may be low at first as users first discover the feature or as marketing of the feature ramps up. If no one uses your app or feature, you don’t pay anything.

Scale

Lambda is auto scaling by default. Without having to perform any complex setup yourself, you get a service that will scale to handle any load from one to one billion requests per second. To be fair, your AWS account will have limits on how many Lambdas you can execute in parallel, but those limits can be raised or removed upon request. The point is, you do not need to worry about adding or removing capacity as usage goes up or down. If your app catches fire, it scales automatically and in almost all cases you will pay much less than you would if spinning up your own servers.

Library driven development

In our experiment we were able to utilize third party libraries to support our development, without having to embed them directly into our app. For Android, this means no bloating our APK size, or cutting into our method count. Library driven development also forces us to think about what does and does not make sense as a library, and what the responsibilities of each library are and are not. Libraries are more easily maintained in general and work better for us as we scale our development team size.

What are the downsides to this model?

Including the AWS SDK

In order to invoke AWS Lambda directly in iOS and Android applications without using Amazon API Gateway, the AWS Mobile SDK has to be added to the projects. This has the drawback of increasing application size and the number of dependencies in each mobile app.

Android

In developing Android applications you have to be mindful of the number of methods in your application. In order for the application to be a single .dex file, you have to have fewer than 65535 methods. Libraries included in an Android project count toward this limit and including the AWS SDK adds at least three thousand methods.

This drawback is ameliorated by only including the modules of the SDK that your app requires. Amazon has split up their Android SDK into modules. While every app which uses the SDK is required to include the AWS SDK core module, our implementation on Android only needs to include the Lambda module from the tens of other modules offered in the AWS SDK. This strategy saves us from having to import thousands of unused methods. Even with the two libraries, some of these methods can be later eliminated from the application by using the Proguard tool but this only reduces the number of added methods and requires additional development time to configure properly.

iOS

While iOS doesn’t carry the same method limit that Android does, there are still some drawbacks. Any library which you add to an application will increase its size. The AWS library is quite large and unlike on Android, it is not split up into core and other modules, meaning you need to frequently build the full library. The library will add some heft to your IPA. Additionally, as any other library you need to maintain updates to the library.

Limited language support

Currently, AWS Lambda only supports four languages: Node.js (JavaScript), Python, C#, and Java. At Hootsuite, the services in our SOA architecture are mostly written in Go and Scala. Existing Lambdas we have in production utilize Node.js. Across our entire development team, we have skills in all of the current Lambda languages.

In addition to our current Java Lambda function, we have experimented with running Go code in a Lambda function. Although this language isn’t currently supported, it can be wrapped in a Python function using a library such as go-lambda. The library uses a Python function to pass data back and forth to arbitrary Go code.

Within the Hootsuite Mobile team, the current language support means that AWS Lambda is much easier for Android developers to pick up and immediately be productive in comparison to iOS developers.

Additional network calls

Invoking any AWS Lambda function requires making at least one network call (two if you need to make a call to AWS Cognito to get temporary credentials). Thus, the benefits Lambda offers of cross-platform invocation and smaller app size comes at the cost of a network call.

Our implementation of AWS Lambda is used with a feature that already requires an active connection to the internet and so this drawback is not significant. However, for an application that is expected to be able to be used offline, an AWS Lambda solution may not work.

Sharing beyond Android/iOS with Amazon API Gateway

Our experiment with Lambda had the goal of invoking a Lambda function from both our Android and iOS applications through the Amazon mobile SDK to accelerate cross-platform development. That experiment has been a success. But what if we want to push beyond native apps and provide our Lambda function to customers using our desktop and mobile browser products? Or if we wanted to avoid embedding the native AWS SDKs that are bloating our app sizes? For that we can use Amazon API Gateway. API Gateway is a fully managed service, designed to integrate well with the similarly serverless AWS Lambda.

Using API Gateway, you can easily create an API that exposes your API to standard REST calls. A screenshot of the AWS console with such a configuration is shown below.

Amazon API Gateway Setup

Here, we have created an API called “testLambdaApi” with a “/function” resource. This resource has a POST method attached to it which when invoked sends the request to Lambda and then receives the response to send back to the client. Using the test client you can invoke the API right from the dashboard.

Amazon API Gateway Routing

The request is routed to the “testApi” Lambda function set-up for to test API Gateway. The testApi function then sends the response back to Gateway which returns the correct response to the client.

While API Gateway is a great tool to create REST endpoints for your Lambda functions, the overhead of creating and linking resources can be burdensome depending on the application. If you would prefer to get up and running quickly, there are tools like serverless out there to remove the tedium from exposing your Lambda function using Amazon API Gateway.

Summary

After many months of production use and deployment on both iOS and Android, our assessment is that the experiment was successful. After releasing on Android, we were able to leverage our Lambda to speed up the delivery of our iOS implementation. An unexpected benefit was that we were able to outsource the iOS version to a third party without having to share the intellectual property within our Lambda. In a company like ours where security is always a consideration, being able to selectively share intellectual property meant we could share more quickly.

We ended up developing user experience patterns to ‘hide’ the latency added by having to call to the cloud. Typically it involves calling our Lambda earlier in the workflow so the result is already cached when the user requires it.

Our first experiment involved quite a bit of heavy lifting around how to properly handle versioning, backward compatibility and automated deployments. Now that we’ve laid the foundation, we expect to recoup that investment on all future Lambdas.

Currently, we’re evaluating where the next opportunities are to move business logic from inside our app into the cloud. If sharing this has helped you, or you have any feedback for us, please tweet us @hootsuitemobile.

About the Authors

Paul and Neil

Neil Power is a two-time Hootsuite co-op turned full-time Android developer. Paul Cowles is director of software development for the Hootsuite mobile group. Neil and Paul have a lot of fun working together on the Hootsuite mobile team.