Learn Cloud Computing - Non-Relational (NoSQL) & In-Memory Databases

#Introduction

In episode 9 we discussed relational databases: managed RDBMS, multi-AZ high availability, and distributed SQL like Aurora and Spanner. Relational excels because of strict schemas, ACID transactions, and a universal query language. But more and more modern applications hold data that's uncomfortable being forced into tables with fixed columns — data whose shape keeps changing, that must be read with very low latency, or that has to serve millions of requests per second.

Episode 10 covers the opposite: NoSQL and in-memory databases. You'll learn when relational starts to be insufficient, the four NoSQL data models, the role of in-memory caches, the managed services on AWS, GCP, and Azure, and the practice of writing JSON items and storing them in DynamoDB via the CLI.

#When Relational Starts to Be Insufficient

There's no bad database — there are only misplaced databases. Relational is great for structured, related data that needs strict consistency, like financial transactions. It struggles in three conditions:

• Large scale: millions of items with queries that must respond within milliseconds.
• Dynamic schemas: new attributes appear at any time, like user profiles that can gain fields whenever needed.
• Extreme read volume: the same data read thousands of times — this is where a cache, not repeated queries, is the answer.

#Four NoSQL Data Models

The word "NoSQL" unites four very different families. Treating them all the same is a beginner's mistake — your choice must depend on your application's access patterns.

#Document

Data is stored as documents in JSON or BSON format, like a dictionary entry that can nest. One document can hold a name, address, and order list all at once — no joins needed. Suitable for user profiles, product catalogs, and CMS content. Representatives: MongoDB, Firestore, Cosmos DB.

#Key-Value

The simplest model: one unique key mapped to one value, like a giant dictionary. Reads always go through the key — very fast and very scalable, but with no complex query capability. Suitable for user sessions, feature flags, and small reference data. Representatives: DynamoDB in simple mode, Redis, Memcached.

#Wide-Column

Data is stored per row, but each row may have different columns — a hybrid of tables and documents. Suitable for large analytics and time-series data, where one row holds millions of timestamped values. Representatives: Cassandra, Bigtable.

#In-Memory

Data is stored entirely in RAM, not on disk. The result: reads in microseconds to milliseconds. This family is fundamentally different — its purpose isn't storing primary data, but accelerating frequently-read data. Representatives: Redis, Memcached.

#In-Memory Cache: Why and When

An in-memory cache exists for one simple reason: many database queries repeatedly answer the same question. Every time an application reads the same profile a hundred times per second, the database repeats work whose results are identical.

The classic pattern is called cache-aside: the application checks the cache first; if it's there, use it directly (cache hit); if not, fetch from the database, store it in the cache, then return it (cache miss). Each item gets a TTL (time to live) — an expiration — so data doesn't go stale forever.

Imagine a coffee shop that's often asked "what's on the menu today?" — answering that question from the menu board (cache) is far faster than opening the fridge every time (database). That's the essence of caching.

#Managed NoSQL and Cache Services in the Cloud

All three major clouds provide every family above as a managed service — you install nothing, just create resources and call their APIs:

• AWS: DynamoDB (key-value and document), ElastiCache (Redis and Memcached).
• GCP: Firestore (document), Bigtable (wide-column), Memorystore (Redis and Memcached).
• Azure: Cosmos DB (multi-model: document, key-value, wide-column, graph), Azure Cache for Redis.

#Practice: DynamoDB JSON Items and CLI Commands

DynamoDB stores data as items, and each item is a collection of attributes. Its strength lies in the flexible schema: the following item has different attributes within the same table — something impossible in a relational table.

{
  "pk": { "S": "user#42" },
  "sk": { "S": "profile" },
  "name": { "S": "Arman" },
  "age": { "N": "29" },
  "active": { "BOOL": true },
  "skills": { "SS": ["cloud", "devops"] },
  "address": {
    "M": {
      "city": { "S": "Jakarta" },
      "zip": { "S": "12345" }
    }
  }
}

Each attribute is written with its data type: S for string, N for number, BOOL for boolean, SS for set of strings, and M for a nested map. Another item in the same table could contain just two attributes — there are no schema constraints at the table level.

Storing this item in the users table is done with aws dynamodb put-item:

aws dynamodb put-item \
  --table-name users \
  --item '{
    "pk": { "S": "user#42" },
    "sk": { "S": "profile" },
    "name": { "S": "Arman" },
    "age": { "N": "29" },
    "skills": { "SS": ["cloud", "devops"] }
  }'

Notice the recommended DynamoDB pattern: the pk and sk pair are the partition key and sort key — this is the item's "address" in the table. A good key design mirrors how the application accesses data: if the application always reads profiles by user ID, then pk holds that ID.

#Comparing the Big 3 NoSQL and Cache Services

The concepts behind all these products are identical: flexible schemas for NoSQL, and storage in RAM for caches. Moving between providers means learning new APIs and names, not relearning concepts. The most valuable skill in the job market is the ability to judge when to use document, key-value, wide-column, or cache — not memorizing buttons in one console.

#Conclusion

In this episode 10 you understood when NoSQL is needed — large scale, dynamic schemas, and extreme read patterns — along with its four data models: document, key-value, wide-column, and in-memory. You also got to know the managed services on the three clouds — DynamoDB, Firestore, Bigtable, Cosmos DB, ElastiCache, Memorystore, and Azure Cache for Redis — plus the practice of writing JSON items and storing them via aws dynamodb put-item.

The keys to take away:

• Choose based on access patterns, not product names — the four NoSQL data models answer four different needs.
• A cache accelerates, it doesn't store — an in-memory cache protects the database, not replaces it.
• NoSQL isn't a relational replacement — it's another option for a different context, and a good engineer masters both.

Everything we've discussed so far still requires you to think about servers — how much capacity, how many instances, how many nodes. Episode 11 will flip that logic entirely: Serverless & Function as a Service (FaaS) — applications that run only when an event occurs, scale from zero to millions, and bill per execution in milliseconds.