Join our Discord Server
Ajeet Raina Ajeet Singh Raina is a former Docker Captain, Community Leader and Distinguished Arm Ambassador. He is a founder of Collabnix blogging site and has authored more than 700+ blogs on Docker, Kubernetes and Cloud-Native Technology. He runs a community Slack of 9800+ members and discord server close to 2600+ members. You can follow him on Twitter(@ajeetsraina).

Setting up AWS S3 bucket locally using Localstack and Docker

2 min read

With over 52,000 stars and 520+ contributors, LocalStack is an open-source tool that emulates a wide range of AWS services on a local machine. It’s primarily used for testing and development purposes, allowing developers to run applications locally without needing to interact with the actual AWS cloud.

LocalStack is a cloud development platform that solves the challenge of costly and slow cloud resource usage during development and testing of serverless applications. By providing a local environment that replicates AWS services, LocalStack helps developers ensure that their applications work correctly before deploying them to a live environment.

Key features of LocalStack include:

  • Service Emulation: It emulates various AWS services such as S3, DynamoDB, Lambda, API Gateway, SQS, SNS, CloudFormation, and more.
  • Local Development: Developers can run AWS services locally, which speeds up development cycles and reduces costs associated with using AWS resources.
  • Consistency: Ensures that the development environment closely mirrors the production environment on AWS, reducing discrepancies and potential issues during deployment.
  • Automation and CI/CD Integration: LocalStack can be integrated into CI/CD pipelines, enabling automated testing of AWS-dependent code. LocalStack can be run using Docker, making it easy to set up and use on various platforms.

Benefits of using Localstack:

  • Reduced Costs: By emulating AWS services locally, LocalStack eliminates the need to constantly deploy to the cloud, saving you money on cloud charges.
  • Faster Development: LocalStack lets you spin up services instantly on your machine, significantly speeding up development cycles compared to waiting for cloud deployments.

Getting Started

You can directly start the LocalStack container using the Docker CLI. This method requires more manual steps and configuration but gives you more control over the container settings.

The first time you start localstack, it requires the Docker daemon to be up and running on your system.

localstack start -d

     __                     _______ __             __
    / /   ____  _________ _/ / ___// /_____ ______/ /__
   / /   / __ \/ ___/ __ `/ /\__ \/ __/ __ `/ ___/ //_/
  / /___/ /_/ / /__/ /_/ / /___/ / /_/ /_/ / /__/ ,<
 /_____/\____/\___/\__,_/_//____/\__/\__,_/\___/_/|_|

 πŸ’» LocalStack CLI 3.5.0
 πŸ‘€ Profile: default

[09:22:54] starting LocalStack in Docker mode 🐳                                                                                                    localstack.py:503
           preparing environment                                                                                                                    bootstrap.py:1283
ERROR: '['docker', 'ps']': exit code 1; output: b'Cannot connect to the Docker daemon at unix:///Users/ajeetsraina/.docker/run/docker.sock. Is the docker daemon running?\n'
❌ Error: Docker could not be found on the system.
Please make sure that you have a working docker environment on your machine.

Localstack expects the Docker environment up and running before proceeding. Go ahead and install Docker Engine or Docker Desktop on your system.

localstack start -d

     __                     _______ __             __
    / /   ____  _________ _/ / ___// /_____ ______/ /__
   / /   / __ \/ ___/ __ `/ /\__ \/ __/ __ `/ ___/ //_/
  / /___/ /_/ / /__/ /_/ / /___/ / /_/ /_/ / /__/ ,<
 /_____/\____/\___/\__,_/_//____/\__/\__,_/\___/_/|_|

 πŸ’» LocalStack CLI 3.5.0
 πŸ‘€ Profile: default

[09:23:54] starting LocalStack in Docker mode 🐳                                                                                                    localstack.py:503
           preparing environment                                                                                                                    bootstrap.py:1283
           configuring container                                                                                                                    bootstrap.py:1291
           starting container                                                                                                                       bootstrap.py:1301
[09:23:55] detaching                                                                                                                                bootstrap.py:1305

You can also start the Docker container simply by executing the following docker run command:

$ docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 localstack/localstack

List of AWS services that Localstack support

localstack status services
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Service                  ┃ Status      ┃
┑━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
β”‚ acm                      β”‚ βœ” available β”‚
β”‚ apigateway               β”‚ βœ” available β”‚
β”‚ cloudformation           β”‚ βœ” available β”‚
β”‚ cloudwatch               β”‚ βœ” available β”‚
β”‚ config                   β”‚ βœ” available β”‚
β”‚ dynamodb                 β”‚ βœ” available β”‚
β”‚ dynamodbstreams          β”‚ βœ” available β”‚
β”‚ ec2                      β”‚ βœ” available β”‚
β”‚ es                       β”‚ βœ” available β”‚
β”‚ events                   β”‚ βœ” available β”‚
β”‚ firehose                 β”‚ βœ” available β”‚
β”‚ iam                      β”‚ βœ” available β”‚
β”‚ kinesis                  β”‚ βœ” available β”‚
β”‚ kms                      β”‚ βœ” available β”‚
β”‚ lambda                   β”‚ βœ” available β”‚
β”‚ logs                     β”‚ βœ” available β”‚
β”‚ opensearch               β”‚ βœ” available β”‚
β”‚ redshift                 β”‚ βœ” available β”‚
β”‚ resource-groups          β”‚ βœ” available β”‚
β”‚ resourcegroupstaggingapi β”‚ βœ” available β”‚
β”‚ route53                  β”‚ βœ” available β”‚
β”‚ route53resolver          β”‚ βœ” available β”‚
β”‚ s3                       β”‚ βœ” available β”‚
β”‚ s3control                β”‚ βœ” available β”‚
β”‚ scheduler                β”‚ βœ” available β”‚
β”‚ secretsmanager           β”‚ βœ” available β”‚
β”‚ ses                      β”‚ βœ” available β”‚
β”‚ sns                      β”‚ βœ” available β”‚
β”‚ sqs                      β”‚ βœ” available β”‚
β”‚ ssm                      β”‚ βœ” available β”‚
β”‚ stepfunctions            β”‚ βœ” available β”‚
β”‚ sts                      β”‚ βœ” available β”‚
β”‚ support                  β”‚ βœ” available β”‚
β”‚ swf                      β”‚ βœ” available β”‚
β”‚ transcribe               β”‚ βœ” available β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Creating a AWS S3 Bucket

Create an s3 bucket with LocalStack’s awslocal CLI:

awslocal s3api create-bucket --bucket sample-bucket
{
    "Location": "/sample-bucket"
}

Listing the S3 Bucket

awslocal s3api list-buckets
{
    "Buckets": [
        {
            "Name": "sample-bucket",
            "CreationDate": "2024-06-29T17:56:46.000Z"
        }
    ],
    "Owner": {
        "DisplayName": "webfile",
        "ID": "75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a"
    }
}

Listing items inside the bucket:

awslocal s3api list-objects --bucket sample-bucket
{
    "RequestCharged": null
}

Running Localstack using Docker Compose

services:
  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}"
    image: localstack/localstack
    ports:
      - "127.0.0.1:4566:4566"            # LocalStack Gateway
      - "127.0.0.1:4510-4559:4510-4559"  # external services port range
    environment:
      # LocalStack configuration: https://docs.localstack.cloud/references/configuration/
      - DEBUG=${DEBUG:-0}
    volumes:
      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

Starting the container

docker compose up
[+] Running 1/0
 βœ” Container localstack_demo  Recreated                                                                                                                0.1s
Attaching to localstack-main
localstack-main  |
localstack-main  | LocalStack version: 3.5.1.dev
localstack-main  | LocalStack build date: 2024-06-24
localstack-main  | LocalStack build git hash: 9a3d238ac
localstack-main  |
localstack-main  | Ready.

Creating a queue using SQS with LocalStack’s awslocal CLI:

$ awslocal sqs create-queue --queue-name test-queue

Result:

{
    "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-queue"
}

Listing the queue

$ awslocal sqs list-queues

Result:

{
    "QueueUrls": [
        "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-queue"
    ]
}

Conclusion

LocalStack has emerged as a valuable tool for developers working with cloud-native applications. By simulating a complete AWS environment on your local machine, it eliminates the need for expensive cloud resources and complex setups during development and testing. This allows for faster iteration cycles, improved developer experience, and reduced costs associated with cloud infrastructure.

References:

Have Queries? Join https://launchpass.com/collabnix

Ajeet Raina Ajeet Singh Raina is a former Docker Captain, Community Leader and Distinguished Arm Ambassador. He is a founder of Collabnix blogging site and has authored more than 700+ blogs on Docker, Kubernetes and Cloud-Native Technology. He runs a community Slack of 9800+ members and discord server close to 2600+ members. You can follow him on Twitter(@ajeetsraina).
Join our Discord Server
Index