My AWS DevOps and SA associate notes – Part 4 – VPC, Elastic Beanstalk, S3

This section contains notes for VPC, Elastic Beanstalk and S3.

Links to other sections:

Part 1 – Overview

Part 2 – IAM, Security Groups, EBS, EFS

Part 3 – ELB, CLI commands, EC2, Lambda

Part 5 – SNS, CloudFormation, SQS, DynamoDB

VPC

Allows for the logical isolation of the AWS cloud where resources are launched in a virtual network. Complete control over networking inside that bubble, including IP addressing, routing and gateways.

Possible to create a hardware VPN between the on-prem DC to the AWS cloud to extend the datacenter.

Security groups can span subnets.

ONLY ONE internet gateway per VPC. Highly available. Spread across availability zones.

All subnets inside the default VPC have internet access.

VPCs can be peered to each other using private IPs.

VPCs across different accounts can be peered too.

Hub and spoke configuration for VPC. No transitive peering.

Central VPC talks in and out to other VPCs.

VPC components:

– internet/private gateways
– route tables
– NACLs
– security groups

.1, .2, .3, .255 – in any subnet are reserved.

Egress only internet gateways are IPv6.

Disable Source/Destination checks with NAT instance during creation. No need to do this with the NAT Gateway.

NAT gateways

– autoscaling upto 10Gbps
– no patching
– auto assigned a public IP address
– must create additional NAT gateways in different multiple AZs for redundancy

NACLs start off as default deny in and outbound.

DEFAULT VPC CREATES A DEFAULT NACL WHICH IS ALLOW ALL.

A NACL can be associated with multiple subnets. But one subnet can be associated with a single NACL only.

When a subnet is associated with a new NACL, the previous assoication is removed.

BLOCK OUT USING NACLS ONLY, NOT USING SECURITY GROUPS.

Extreme performance and static IP addressing for applications = Network Load Balancer. Otherwise the Application Load Balancer works.

VPC Flow Logs – a feature to monitors traffic in and out of VPC.

Flow logs data is stored using CloudWatch logs, retrieved and viewed using these Cloudwatch logs.

Once created a flow log cannot be modified, create a new one for changes.

Windows activation/licensing lookups or Amazon DNS is not monitored using Amazon Flow Logs. metadata data traffis is not monitored.

NAT is for internet traffic connectivity. Bastions are jump boxes for secure administration of various EC2 instances.

NAT gateways will get traffic going out on public internet.
Endpoint gateways keeps it all within private addressing.

Lowest numbered rule evaluated first.

NACLs are stateless, different inbound and outbound rules possible.

Enable Flow Logs for VPCs within the ONE AWS account ONLY.
Flowlogs cannot be tagged.
Config of a flowlog cannot be changed.

Elastic Beanstalk (EB)

Deploy, monitor and scale an app quickly.

provision application infrastructure transparently.

Devs just upload their code and beanstalk just provisions the resources required to spin it up.

Focusing on components and performance, not config and specs.

CloudFormation is CLI, Beanstalk is GUI.

Entire application stack can be one EB or multiple EBs.

Apps are uploaded to EB as a zip file.

Unlimited versions of apps within EB.

Platforms supported on EB:

– Python (Apache HTTP)
– Docker
– node.js (Nginx or Apache HTTP)
– Java SE (Apache HTTP)
– Go
– .Net 7.5 8.0 8.5
– Tomcat
– Ruby (Passenger or Puma)
– PHP (Apache HTTP)
– Packer
– GlassFish
– Multicontainer Docker

Rolling updates – all at once, immutable, rolling, with batch. With batch – create new instances of the app to make sure there are enough instances in service.

Security using service roles and a new key pair.

Two options for the database side of things – either let EB do it or create a DB and hook it up with EB (recommended by AWS).

DB instances created by EB are deleted upon EB instance termination.

DB instances ONLY connected to EB are NOT deleted upon EB instance termination.

EB is free on its own, just gotta pay for the resources it consumes.

S3

– Object based storage – upload files.
– Not for installing an OS or DB, for that EBS is used.
– file size < 5TB.
– Unlimited storage.
– Files are stored in buckets (folders).
– Multiple buckets easily possible in S3.
– S3 is a universal namespace.
– https://s3-<region>/<bucket name>/
– Read after write consistency for puts of new objects.
– Delete can take some time – eventual consistency.
– 3 9’s SLA.
– 11 9’s durability (wont lose a file).
– Tiered storage options.
– Data security – ACLs and bucket policies.
– S3 can sustain the loss of 2 facilities concurrently.

S3 types of storage
– Normal. For everything.
– IA – infrequently accessed data. Pay slips for instance.
– RRS – Reduced Redundancy Storage. Regenerate-able data. 1 concurrent facility storage.
– Glacier – very cheap, retrieval can take 3-5 hours.

S3 transfer acceleration enabled fast and easy transfer of data of files over long distances. Data arrives at the edge location, it’s then routed to S3 over a data optimized network path.

CLOUDFRONT – EDGE LOCATIONS, CACHING.

S3 versioning cannot be disabled once enabled, only suspended. By default GET operation returns the most recent written version. Oldver versions of overwritten and deleted objects can be retrieved by specifying a version in the request.

MFA permitted with versioning.

Source and destination buckets both need versioning turned on for cross region replication.

All versions of an object will be permanently stored in the S3 bucket, even deleted versions can be restored. They have to be deleted permanently. Or set Lifecycle Rules to manage the lifetime of objects.

Lifecycle Management – S3 will automatically move data to less expensive tiers of storage and/or delete objects.

Standard (30 days) > IA (30 days) > Glacier.

Edge Location – caching. At least 50 edge locations.

Web distribution – website.
RTMP – media streaming.

Edge locations can be written to too.

Objects are cached for 24 hours by default.

Cached objects can be cleared but charge is incurred. Use it when you dont want to wait for the TTL to expire.

All buckets and objects are private by default.

ACLs apply to individual objects. Policies apply to buckets.

Encryption –

In Transit – SSL/TLS
At Rest –
– Server side (S3 Managed Keys).
– AWS KMS (audit trail).
– Server side encryption with customer provided keys.
– Client Side encryption.
– on your own (by customers).

File Gateway – flat files. Stored on S3.
Volume Gateway – Stored Volumes (entire dataset is stored on site).

Gateway VTLs – used for popular backup applications – Netbackup/Backup Exec, Veeam.

Snowball:

+Edge – storage plus compute
Snowmobile – on a truck
Import and export to S3

Storage acceleration to rapidly send data elsewhere.

STATIC WEBSITES ONLY on S3.

CORS – Cross Origin Resource Sharing – diffenet resources in different buckets. Enable on the buckets and point out the URL for the required resource.

FORMATS:

IF IT’S A BUCKET – THEN THE NAME OF THE BUCKET IS AT THE END.

IF IT’S A WEBSITE – THEN THE BUCKET IS UP FRONT > NAME OF SITE.

FAST UPLOAD – MULTIPART UPLOAD.

Amazon S3 Mgmt Console, AWS SDKs or the Amazon S3 APIs for Lifecycle Management.

Amazon Glacier APIs cannot access Glacier storage data.

Expedited 1->5 mins.
Standard 3->5 hours.
Bulk 5->12 hours.

S3 Transfer Acceleration between client and bucket. Data arrives at closest Edge location and then routed to S3 bucket via the most network optimized path.

Two types of transfers – s3-acclerate and s3-acclerate.dualstack. Point the PUT and GET requests to either of the transfer endpoints.

Use case for transfer acceleration – uploading from different geo locations to central bucket and if this happens regularly.

Multipart uploads also supported with S3 Transfer Acceleration.

S3 object tags are key value pairs applied to S3 objects.

Cross Region Replication automatically replicates data across AWS regions. With CRR, every object uploaded to an S3 bucket is automatically replicated to a destination bucket in a different AWS region you choose. Low latency to geographically dispersed locations or to satisfy auditing compliance requirements. This is enabled at the bucket level and specifying the destination bucket. CRITICAL TO ENABLE VERSIONING AT BOTH SOURCE AND DESTINATION BUCKETS. Can also be used to protect against unintentional and malicious deletion by granting ownership overwtite to maintain a distinct ownership stack between the two sites.

Amazon SSE-S3 managed keys or Amazon SSE-KMS managed keys for srever side encryption of data in buckets. Encrypts before saving to disk, decrypts when objects are downloaded.

S3 buckets contain Resources, Actions, Effects and Principal.

403 Forbidden – AccessDenied.

Anything larger than 100MB can use multi part upload.

Multipart upload – improved throughout, quick recovery, pause/resume, begin an upload as the object is being created.

Leave a Comment

Your email address will not be published.