By Po Liu on October 29, 2014
The DataLab team at Hootsuite is responsible for storing the constant firehose of data that our 10 million users generate for product intelligence analyses. All useful insights stem from a hardy storage-and-retrieval system that we can rely on. Recently, we migrated our primary storage back-end from Apache Hadoop to Amazon’s Simple Storage Solution (S3) in order to scale our ability to analyze data. What follows is a short retelling of the journey we took and the lessons learned while handling the nozzle.