My AWS DevOps and SA associate notes – Part 5 – SNS, CloudFormation, SQS, DynamoDB

This section of the notes contains SNS, CloudFormation, SQS and DynamoDB.

Links to other sections:

Part 1 – Overview

Part 2 – IAM, Security Groups, EBS, EFS

Part 3 – ELB, CLI commands, EC2, Lambda

Part 4 – VPC, Elastic Beanstalk, S3

SNS

Web notification service to publish messages to users/apps

PUSH not PULL.

SNS can be used to send messages to SQS, email, SMS and HTTP.

When publishers have info or updates to notify subscribers about when a message is published which immediately triggers SNS to PUSH messages to all subscribers.

SQS – message queue service used by distributed applications to exchange messages via a polling model.

SNS is commonly used to publish messages to SQS.

If using messaging with existing applications, use MQ. Supports industry standard APIs and protocols so you can switch from any message broker to MQ without rewriting messaging code.

If using messaging with new applications, use SNS and SQS.

CloudTrail for auditing of messages for accounts

Topic names are limited to 256 characters. 100k per account.

CloudFormation

Free on its own. Pay for the resources it uses.

Converts traditional hardware infra and converts it into code.

Gives devs and admins an easy way to create a collection of related AWS resources.

This has two bits to it:

– CF template is an architectural diagram (yaml or json format)
– CF stack is the end result of that diagram

Function called Fn:GetAtt is the best way to output data to CF.

CF templates can show up inside CF templates.

Automatic rollback on error is disabled.

Errors are charged.

If customers are looking at new to AWS and like to just upload code and get on with it – EB.

If customers are looking at turning their infrastructure into code – CF.

Limitless number of templates in CloudFormation.

Rollback_in_Progress – errors in CloudFormation.

SQS

14 days – max retention for SQS messages.

Delivered ONCE and FIFO.

Messages have a visibility timeout of 30s. ChangeMessageVisibilityTimeout to change visibility timeout.

Visibilitytimeout = 0 to make the message instantly available to other things.

Dead letter queues for unsuccessfully processed messages.

There’s a cost associated with polling. Long and short = same cost.

Does not support TLS 1.3.

Messages hang around in SQS queues for max 14 days. default 4 days.

DynamoDB

Stored on SSD with1 location written to, then sync replicated to the other 2.

DynamoDB uses eventually consistent reads by default, unless specified. To use strongly consisten reads, set the ConsistentRead parameter.

Eventual Consistency Read. Data read from closest location after a write. Best for apps (or for RPO !=0)

Strongly Consistent Read. App that needs consistency and can wait for those acks from 2 replicas before proceeding. RPO=0. USES MORE THROUGHPUT.

BatchGetItem can retrieve 100 items total size 16MB in ONE operation.

Scan operaiton can retrieve 1MB in ONE operation.

Two types of primary keys available:

– Single attribute (unique ID)

– Composite (unique ID and a date like when someone signed up)

Write capacity of a table in Dynamo DB is 1kb writes per second.

Authorization is handled by IAM. Also allows for fine grained policies.

Updating a table does not consume capacity units.

Removes the management of the databases.

Provisioned ThroughputExceededException – main hash key or index key is out of bounds.

Atomic in-place updates.

Limits on TABLES PER ACCOUNT AND the NUMBER OF PROVISIONED THROUGHPUT UNITS PER ACCOUNT. Can be eased by contacting AWS Support.

Lives on SSD.

For large amounts of infrequently accessed data, use S3 instead.

GET and PUT operation using a user defined primary key.

The primary key is the only required attribute when creating a table.

If huge scaling requirements are there, DynamoDB over RDS/EC2.

Larger files, unstructured blobs in S3.
Smaller files in DynamoDB such as file pointers, customer data.

Each table MUST have a primary key.

A Primary Key can be a single attribute key or a composite key.

NO limit on number of attributed within item, but total < 400KB.

Query vs Scan:

– query fets one or more items using the primary key.
– scan gets all items by performing a full scan, limited by specifying filters against one or more attributes.

Max 10k reads or writes before needing to contact Support.

Autoscaling suited for DynamoDB tables which have sustained operations on them from mins to hours.

FGAC is used by DynamoDB to indicate which items or a table can be accessed by whom and what actions are permitted. Uses IAM to auth the request and determine the capabilities of user. Best to allow access to certain attributes only or use DENY.

Optimistic Locking Model.

Does not do cross table joins.

5 local secondary indexes.

Write and Read Capacity.

One unit of Write Capacity ellows to perform ONE write/s – 1KB.
One unit of Read Capacity allows ONE strongly/TWO eventually consistent reads- 4KB.

Units required for Writes = Number of items per second X item size in 1KB blocks.
Units required for Reads = Number of items per second X item size in 4KB blocks.

Multiple copies of a single master table are permitted, unlimited copies.

CORS for DynamoDB – replication to an EC2 instance. This instance is not auto-scaled. If this instance falls behind or fails, the replication shifts to another EC2 instance in the auto-scaling group.

50 tags per DynamoDB table. Tags are deleted automatically when table is deleted.

Tags can be null.

TTL can be specified on specific items. Allows for removal of stale data and to keep data for archiving/auditing purposes.

256 tables per region.

Leave a Comment

Your email address will not be published.