Simple Queue Service

  Amazon Web Services (AWS), Application Services

Video: https://www.udemy.com/aws-certified-solutions-architect-associate/learn/v4/t/lecture/2050708?start=0
FAQ: https://aws.amazon.com/sqs/faqs/

 

SQS Fact:

SQS is the OLDEST Amazon service that was available.

SQS is a web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them.

SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed be another component.  A queue is a temporary repository for messages that are waiting for processing.

Example: Meme Generator

  • Upload photo to website.
  • Website puts a ‘job’ into the SQS queue.
  • Application instances then poll the queue looking for work.
  • Once the work is complete, it goes back to SQS looking for the next job.

Example #2: Travel Website

  • User enters location and dates they want to fly.
  • Website uploads this ‘job’ to SNS
  • Application servers poll SQS for the job and find the lowest flight
  • Data is fed back to the web server & relayed back to the user.

Notes:

  • Auto-scaling can be implemented for the application instances.
    • If SQS queue exceeds ‘x’ jobs, provision additional processing instances.
    • If SQS queue falls below ‘y’ jobs, remove some instances.
  • SQS is a PULLING service, not a ‘Pushing’ (See SNS)
  • Messages remain in the queue until removed.
    • Messages in process are marked ‘invisible’ until timed out, when they become visible again.

What is SQS

Using SQS, you can decouple the components of an application so they run independently with SQS easing message management between components.  Any component of a distributed application can store messages in a fail-safe queue.  Messages can contain up to 256 KB of text in any format (JSON, XML, plain text, etc.).  Any component can later retrieve the messages programmatically using the SQS API.

The queue acts as a buffer between the components producing and saving data, and the component receiving the data for processing.  This means the queue resolves issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer are only intermittently connected to the network.

Types of Queues

Standard Queues (Default)

  • A Standard queue lets you have a nearly unlimited number of transactions per second.  Standard queues guarantee that a message is delivered at least once.  However, occasionally because of the highly-distributed architecture that allows high throughput, more than one copy of a message might be delivered out of order.  Standard queues provide best-effort ordering which ensures that messages are generally delivered in the same order as they are sent.

 

Fifo Queues (First In / First Out)

  • The FIFO queue complements the standards queue.  The most important features of this queue type are FIFO (first in – first out) deliver and exactly once processing.  The order in which the messages are sent and received is strictly preserved and a message is delivered once and remains available until a consumer processes and deletes it.  Duplicates are not introduced into the queue.  FIFO queues also support message groups that allow multiple ordered message groups within a single queue.  FIFO queues are limited to 300 transactions per second (TPS) but have all the capabilities of standard queues.

Key Facts

  • SQS is Pull based and not Pushed based.
  • Messages are limited to 256 KB in size.
  • Messages can be kept in the queue from 1 minute to 14 days.  4 days is the default.
  • The Visibility Time Out is the amount of time that a message is invisible in the queue after a read picks up that message.
    • If the message is processed before the visibility time out, the message is deleted from the queue.
    • If the message is not processed within that time, the message become visible again and another reader (aka consumer) will process it.  This can result in the same message being delivered twice.
  • SQS guarantees that your messages will be processed at least once.
  • SQS Long Polling is a way to retrieve messages from your SQS queues.
    • Short Polling returns immediately, even if the message queue is empty.
    • Long Polling doesn’t return a response until
      • a message arrives in the queue, or
      • the long poll times out.
      • Long Polling makes a single request until a message is returned, vs. several Short Polls that may return empty.  This may make Long Polling more cost advantageous over Short Polling.

LEAVE A COMMENT