7 Tips for Building a Multi-Tenant Cloud Service over AWS

A list of tips for building a multi-tenant cloud service over AWS.

Photo by Kyle Sung on Unsplash

About a year ago, we started to plan and implement our new multi-tenant cloud service at Hysolate. It was quite a challenge: there are so many alternatives to consider and decisions to make in such a short time.

Should we go Serverless? Use Fargate? Kubernetes? And which Identity Provider should we pick? Each of these decisions has a huge impact and is very difficult to revert once your product grows.

We made some mistakes along the road but also had some good calls. More than anything — we learned a ton in the process and decided to share some of our insights.

So here are a few tips:

1. Start with a single region. It will simplify things and let you move very fast. For obvious reasons, supporting multiple regions early on creates an ongoing operational drag that you should strive to avoid as long as you can. In most cases, you will need to support more than one region only when it blocks a significant deal for compliance/regulatory reasons.
2. Looking for an identity provider? Avoid AWS Cognito. We had quite a bad experience with Cognito overall. such as its customizable hosted UI. At the end of the day, the UX was quite bad and we ended up replacing our login page. At the end of the day, Cognito’s hosted UI is barely customizable and has a horrible user experience in many different cases. We managed to work around some of these issues ourselves along the way, but we believe alternatives such as Auth0 or Okta might be better in many cases.
3. Don’t be so terrified of vendor lock-in. In most cases, it is extremely unlikely that you will need to migrate to another cloud provider in the lifetime of a startup. Cloud providers such as AWS can provide you with magical goodies that would be a shame not to take advantage of turn-key solutions that require minimal configuration to hit the ground running and provide value.
4. Avoid the AWS Elasticsearch Service (now Amazon Opensearch Service). We had a very bad experience with AWS ES and ended up moving to the official cloud offering by Elastic (and we’re very happy with it). The provided Elasticsearch and Kibana versions were lagging behind the official releases, and there were also many missing XPack features, such as SAML authentication and automatic index lifecycle management.
5. Familiarize yourself with the awesome powers of Lambda@Edge. This great tool lets you run custom code right at the CDN edge, providing better latency in many use-cases where logic can be executed right at the edge (such as token validations), request manipulations, etc. Running these cuties in a local development environment might be slightly painful, so better use them wisely.
6. Seriously consider ECS Fargate over K8s. While AWS Fargate is still not the mainstream technology in the cloud market, in our opinion, it definitely should be. Our experience with it was fantastic: it just works. In our development environment, we use docker-compose to run our containers locally. We were easily able to eliminate the operational costs and the hassle of maintaining a Kubernetes cluster. The main downside: it’s more expensive.
7. Carefully select your multi-tenant database architecture. Whether it’s a column-based, table-based, or database-per-tenant architecture, understand the pros and cons of each approach. For most B2B startups, going with a database-per-tenant architecture would probably be a safe choice from a regulatory and operational perspective. The pains we encountered were mostly around performing cross-tenant queries and aggregations. You will have to loop over multiple database connections in one way or another.

Plan carefully and enjoy your ride! Building a new product from scratch is always an exciting experience.

If you found some of these tips useful, or have any questions about our experience with these technologies at Hysolate, feel free to leave a comment or get in touch via Twitter (@alonkol).