Designing Web-Scale Storage and Well-Architected Framework (Part 1)

Part1, Part2

Introduction

In real world, availability of a web application is very important. The performance of a web application can result in high page views, better customer experience and higher conversion rates. Therefore, this article will talk about how to build a scalable and well-architected framework.

Web Storage

If possible, please store all the static assets in AWS S3. In case you just directly store the assets in web server, you probably have trouble of delay. On the other hand, AWS S3 can have multiple copy in multiple regions, that means user can access the closest one and reduce the delay.

S3 will automatically partitions our buckets according to the prefixes of the files. Thus, we prefer to add a hex hash prefix to the key to reduce the change that a request reads from the same partition multiple times at once.

In addition, apart from the prefixes, we prefer to maintain a secondary index and hash all key names to more easily retrieve our files in useful ways.

CloudFront

Even though we can set S3 as cross region. However, when users only need to get the same content from S3, it is a waste to retrieve through S3. In the middle between users and S3, we can insert a CloudFront to cache the frequently access content to minimize the redundant data retrieval operations.

CloudFront actually is a content delivery network service. It is independent of region so it has lower latency. We prefer to have static content cached while dynamic content straight from origin.

Database

We have to match the technology to the workload, not the other way around. In some cases, we may have to consider to use NoSQL. For example, leaderboards and scoring, rapid ingest of clickstream or log data, temporary data needs, hot tables, metadata or lookup tables and session data.

AWS recommends us to use DynamoDB if possible since it is a fully managed NoSQL database service without any administration.

In DynamoDB, we can specify at the time of the read request, whether a read should be eventually or strongly consistent.

Conclusion

We have introduced how important of a stable web application. Also, we recommend to store static assets in S3. If the object is frequently accessible, we may consider to use CloudFront. When it is suitable, we prefer to use DynamoDB due to the zero administration.

Leave a Reply