Skip to content

Latest commit

 

History

History
160 lines (145 loc) · 12.2 KB

rds.md

File metadata and controls

160 lines (145 loc) · 12.2 KB

RDS - Relational Database Service

  • RDS is often described as a Database-as-a-service (DBaaS) but this is not accurate. It should be named Database Server as a Service (DBSaaS) product
  • RDS provides managed database instances, which can themselves hold one or more databases
  • Benefits of RDS are the we don't need to manage the physical hardware, the server operating system or the database system itself
  • RDS supports MySQL, MariaDB, PostgreSQL, Oracle, Microsoft SQL Server
  • Amazon Aurora: it is a db engine created by AWS and we can select it as well for usage
  • RDS Subnet Group: list of subnets which an RDS database can use. Generally it is best practice to have on Subnet Group per database deployment

RDS Database Instance

  • Runs one of the few types of db engine mentioned above
  • Can contain multiple user created databases
  • A database instance after creation can be accessed using its hostname (CNAME)
  • RDS instances come in various types, share many of features of EC2. Example of instances: db.m5, db.r5, db.t3
  • RDS instances can be single AZ or multi AZ (active-passive failover)
  • When an instance is provisioned, it will have a dedicated storage allocated as well (usually EBS)
  • Storage allocated can be based on SSD storage (IO1, GP2) or magnetic (mainly for compatibility)
  • Billing for RDS:
    • We are billed based on instance size on a hourly rate
    • We are billed for additional instances used for Multi AZ deployments
    • We are also billed per storage (GB/month) + extra per iops in case of provisioned iops (IO1)
    • Data transfer is also billed if data is coming from/goes to the internet/other regions
    • Backups and snapshots are also billed (per GB per month)
    • Licensing is applicable is also billed

RDS Multi AZ

  • They are 2 types of Multi AZ deployments:
    • Multi AZ Instance (historically called Multi AZ)
    • Multi AZ Cluster
  • Multi AZ Instance:
    • Used to add resilience to an RDS instance
    • Replication happens at the storage level
    • Enables synchronous replication between primary and standby instances
    • Multi AZ is an option which can be enabled on an RDS instance, when enabled secondary hardware is allocated in another AZ (standby replica)
    • RDS is accessed via provided endpoint address (CNAME)
    • With a single instance the endpoint address points the instance itself, with multi AZ, by default the endpoint points to the primary instance
    • We can not directly access the standby instance
    • If an error occurs with the primary instance, RDS automatically changes the endpoint to point to the standby replica. This failover occurs in around 60-120 seconds
    • Multi AZ is not available in the Free-tier (generally costs double as it would the single AZ)
    • Backups are taken from the standby instance (removes performance impact)
    • In case of a failover the DNS name will be updated to point to the standby replica instance. Since this is a DNS change, for the update it generally takes between 60-120 seconds to occur. This can be lessened by removing DNS caching in the application
    • With Multi AZ instance we have ONE standby replica. This replica cannot be used for read and writes. It waits for a failover to happen and then it can be used
    • Backups can be taken from the standby instance to improve performance
    • Failovers can happen if:
      • AZ outage
      • Primary instance failure
      • Manual failover
      • Instance type change
      • Software patching
  • Multi AZ Cluster:
    • RDS is capable of having one writer replicate to two reader instances. We can have 2 readers only!
    • These readers are in different AZs compared to the writer
    • Compared two Aurora cluster mode, Multi AZ cluster can have 2 readers only, while Aurora Cluster can have more
    • In case of Multi AZ cluster the instances to which data is replicated are usable, compared to Multi AZ instance mode when they are not
    • In terms of replication the data is viewed as committed when one of the readers confirms that it was written
    • Other comparisons two Aurora Cluster:
      • In RDS multi AZ cluster each instance has its own storage, in case of Aurora this is not the case
      • Like Aurora, the cluster can be accessed with multiple endpoints:
        • Cluster endpoint: database CNAME, points to the writer, can be used for reads/writes and administration
        • Reader endpoint: points to any available endpoint for reads (it can point to the writer instance in certain cases). Generally it points to the dedicated reader instances
        • Instance endpoints: each instance gets one endpoint, generally not recommended to be used
    • Generally Multi AZ Cluster runs on faster hardware: Graviton + local NVME SSD storage. Any writes are written to local super fast storage, after that they are flushed to the EBS
    • Replications are done via transaction logs => much more efficient then Multi AZ instance. This also allows faster failover: ~35 seconds + any time required to apply the transaction logs

RDS Backups and Restores

  • RPO (Recovery Point Objective): time between the last working backup and the failure. Lower the RPO value, usually the more expensive the solution
  • RTO (Recovery Time Objective): time between the failure and system being fully recovered. Can be reduced with spare hardware, predefined processes, etc. Lower the RTO value, the system is usually more expensive
  • RDS backup types:
    • Manual snapshots:
      • Have to be run manually, or via a script
      • First snapshot is full content of the DB, incremental onward
      • When any snapshot occurs, there is brief interruption in the flowing of data between the compute resource and the storage (no noticeable effect in case of Multi AZ, since the backup is taken from the standby instance)
      • Manual snapshots do not expire
      • When we delete an RDS instance, AWS offers to make one final snapshot
    • Automatic backups:
      • They occur once per day (backup window is defined on the instance)
      • Snapshots which occur automatically, first being full snapshot, incremental afterwards
      • In addition to the automated snapshots, every 5 minute transaction logs are written to S3
      • Automatic backups are not retained, we can set the retention period between 0 and 35 days
      • Automatic backups can be retained after a DB is deleted, but they still expire after the retention period
      • We can replicate backups to another region: both snapshots and transaction logs can be replicated. Charges apply to cross-region data copy and any storage used in the destination region
      • Cross-region replication has to be explicitly configured within automated backups
  • Backups are stored in AWS manages S3 buckets (backups are not visible to us directly in S3) => any data in S3 is regionally resilient
  • RDS backups are taken from the standby instance in case Multi AZ is enabled
  • RDS Restores:
    • RDS creates a new RDS instance when we restore an automated backup or a manual snapshot => new address will be created for the DB
    • When we restore a snapshot, we restore our DB to a single point in time, when the creation time of the snapshots happened
    • With automated backups we can chose a point-in-time to where we want to restore (any 5 minute point-in-time)
    • Restoring snapshots is not a fast procedure (important for RTO)

RDS Read-Replicas

  • Provide 2 main benefits: performance and availability
  • Read replicas are read-only replicas of an RDS instance
  • Read replicas can be used for reading only data
  • Multi AZ Cluster mode is a similar to how read replicas work, but for read replicas we have to think of read replicas as separate things:
    • They are not part of the main database instance
    • They have their own endpoint address
    • Require application support
    • There is no automatic failover to a read replica
  • The primary instance and read replica is kept sync using asynchronous replication
  • There can be a small amount of lag in case of replication
  • Read replicas can be created in a different AZ or different region (CRR - Cross-Region Replication)
  • We can 5 direct read-replicas per DB instance
  • Each read-replica provides an additional instance of read performance
  • Read-replicas can also have read-replicas, but lag starts to be a problem in this case
  • Read-replicas can provide global performance improvements
  • Snapshots and backups improve RPO but not RTO. Read-replicas offer near 0 RPO
  • Read-replicas can be promoted to primary in case of a failure. This offers low RTO as well (lags of minutes)
  • Read-replicas can replicate data corruption

Data Security

  • With all the RDS engines we can use encryption in transit (SSL/TLS). This can be set to be mandatory on a per user bases
  • For encryption at rest RDS supports EBS volume encryption using KMS which is handled by the host EBS and it is invisible for the database engine
  • We can use customer managed or AWS generated CMK data keys for encryption at rest
  • Storage, logs and snapshots will be encrypted with the same customer master key
  • Encryption can not be removed after it is activated
  • In addition to encryption at rest MSSQL and Oracle support TDE (Transparent Data Encryption) - encryption at the database engine level
  • Oracle supports TDE with CloudHSM, offering much stronger encryption
  • IAM authentication with RDS:
    • Normally login is controlled with local database users (username/password)
    • We can configure RDS to allow IAM authentication (only authentication, not authorization, authorization is handled internally!): ASG Lifecycle Hooks

RDS Proxy

  • Opening and closing connections consumes resources and takes time => in case we only want to read/write a tiny amount of data the overhead of establishing a connection creates a significant latency
  • Handling failure of databases instances is hard, this adds significant overhead and risks to our application
  • DB proxies can help, but managing them is not always trivial (scaling, resilience)
  • In case of an RDS proxy our application connects to the proxy, which handles connection polling and connectivity to the database
  • RDS proxies provide multiplexing: a smaller number of connections can be used to connect to the database while having a larger number of applications using the database through the proxy. This helps to reduce the load on the database
  • RDS Proxy can help with database failover events abstracting this from the applications. The proxy can wait until a healthy database instance is in place and can automatically connect to it
  • When to use RDS proxy?
    • In case we have errors such as Too many connections. An RDS proxy can reduce the number of connections to the dabase while being able to handle many more connections from the applications to itself
    • Useful when using AWS Lambda, we won't need to invoke a new connection after each invocation of our function. Saves time by connection reuse and IAM auth
    • Useful for long running applications (SAAS apps) by reducing latency
  • RDS Proxy key facts:
    • Fully managed by RDS/Aurora
    • By default provides auto scaling, HA
    • Provides connections pooling, which reduces DB load
    • Only accessible from a VPC, not accessible from the public internet
    • Accessed via Proxy Endpoint
    • Can enforce SSL/TLS connection
    • Can reduce failover time by over 60% in case of Aurora
    • Abstracts the failure of a database away for our application

RDS Custom

  • Fills the gap between the main RDS product and EC2 running a DB engine
  • The main RDS is fully managed database service => OS/Engine access is limited
  • In contrast databases running on EC2 are self managed, this can have significant management overhead
  • RDS custom bridges this gap, we can utilize RDS but still get access to customization we would have when running a DB instance on EC2
  • Currently RDS custom works from MSSQL or Oracle
  • We can connect to the underlying OS using SSH, RDP or Session Manager
  • RDS custom will run withing our AWS account. Classic RDS will run in an AWS managed environment
  • If we need to perform RDS customization for RDS Custom, we need to look inside the Database Automation settings to make sure we wont have any disruption caused by the Database Automation. We need to pause Database Automation for this period