|
With S3 Object Lambda, you’ll be able to use your individual code to course of information retrieved from Amazon S3 as it’s returned to an software. Over time, we added new capabilities to S3 Object Lambda, like the power to add your individual code to S3 HEAD and LIST API requests, along with the help for S3 GET requests that was obtainable at launch.
Today, we’re launching aliases for S3 Object Lambda Access Points. Aliases at the moment are mechanically generated when S3 Object Lambda Access Points are created and are interchangeable with bucket names wherever you employ a bucket title to entry information saved in Amazon S3. Therefore, your purposes don’t must find out about S3 Object Lambda and may take into account the alias to be a bucket title.
You can now use an S3 Object Lambda Access Point alias as an origin on your Amazon CloudFront distribution to tailor or customise information for finish customers. You can use this to implement automated picture resizing or to tag or annotate content material as it’s downloaded. Many photographs nonetheless use older codecs like JPEG or PNG, and you should use a transcoding perform to ship photographs in additional environment friendly codecs like WebP, BPG, or HEIC. Digital photographs include metadata, and you may implement a perform that strips metadata to assist fulfill information privateness necessities.
Let’s see how this works in follow. First, I’ll present a easy instance utilizing textual content that you would be able to observe alongside by simply utilizing the AWS Management Console. After that, I’ll implement a extra superior use case processing photographs.
Using an S3 Object Lambda Access Point because the Origin of a CloudFront Distribution
For simplicity, I’m utilizing the similar software within the launch publish that adjustments all textual content within the authentic file to uppercase. This time, I take advantage of the S3 Object Lambda Access Point alias to arrange a public distribution with CloudFront.
I observe the identical steps as within the launch publish to create the S3 Object Lambda Access Point and the Lambda perform. Because the Lambda runtimes for Python 3.8 and later don’t embody the requests module, I replace the perform code to make use of urlopen from the Python Standard Library:
import boto3
from urllib.request import urlopen
s3 = boto3.consumer('s3')
def lambda_handler(occasion, context):
print(occasion)
object_get_context = occasion['getObjectContext']
request_route = object_get_context['outputRoute']
request_token = object_get_context['outputToken']
s3_url = object_get_context['inputS3Url']
# Get object from S3
response = urlopen(s3_url)
original_object = response.learn().decode('utf-8')
# Transform object
transformed_object = original_object.higher()
# Write object again to S3 Object Lambda
s3.write_get_object_response(
Body=transformed_object,
RequestRoute=request_route,
RequestToken=request_token)
returnTo check that that is working, I open the identical file from the bucket and thru the S3 Object Lambda Access Point. In the S3 console, I choose the bucket and a pattern file (known as s3.txt) that I uploaded earlier and select Open.
A brand new browser tab is opened (you may must disable the pop-up blocker in your browser), and its content material is the unique file with mixed-case textual content:
Amazon Simple Storage Service (Amazon S3) is an object storage service that gives...
I select Object Lambda Access Points from the navigation pane and choose the AWS Region I used earlier than from the dropdown. Then, I seek for the S3 Object Lambda Access Point that I simply created. I choose the identical file as earlier than and select Open.
In the brand new tab, the textual content has been processed by the Lambda perform and is now all in uppercase:
AMAZON SIMPLE STORAGE SERVICE (AMAZON S3) IS AN OBJECT STORAGE SERVICE THAT OFFERS...
Now that the S3 Object Lambda Access Point is appropriately configured, I can create the CloudFront distribution. Before I try this, within the record of S3 Object Lambda Access Points within the S3 console, I copy the Object Lambda Access Point alias that has been mechanically created:
In the CloudFront console, I select Distributions within the navigation pane after which Create distribution. In the Origin area, I take advantage of the S3 Object Lambda Access Point alias and the Region. The full syntax of the area is:
ALIAS.s3.REGION.amazonaws.com
S3 Object Lambda Access Points can’t be public, and I take advantage of CloudFront origin entry management (OAC) to authenticate requests to the origin. For Origin entry, I choose Origin entry management settings and select Create management setting. I write a reputation for the management setting and choose Sign requests and S3 within the Origin kind dropdown.
Now, my Origin entry management settings use the configuration I simply created.
To scale back the variety of requests going via S3 Object Lambda, I allow Origin Shield and select the closest Origin Shield Region to the Region I’m utilizing. Then, I choose the CachingOptimized cache coverage and create the distribution. As the distribution is being deployed, I replace permissions for the assets utilized by the distribution.
Setting Up Permissions to Use an S3 Object Lambda Access Point because the Origin of a CloudFront Distribution
First, the S3 Object Lambda Access Point wants to provide entry to the CloudFront distribution. In the S3 console, I choose the S3 Object Lambda Access Point and, within the Permissions tab, I replace the coverage with the next:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "cloudfront.amazonaws.com"
},
"Action": "s3-object-lambda:Get*",
"Resource": "arn:aws:s3-object-lambda:REGION:ACCOUNT:accesspoint/NAME",
"Condition": {
"StringEquals": {
"aws:SourceArn": "arn:aws:cloudfront::ACCOUNT:distribution/DISTRIBUTION-ID"
}
}
}
]
}The supporting entry level additionally wants to permit entry to CloudFront when known as by way of S3 Object Lambda. I choose the entry level and replace the coverage within the Permissions tab:
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "s3objlambda",
"Effect": "Allow",
"Principal": {
"Service": "cloudfront.amazonaws.com"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:REGION:ACCOUNT:accesspoint/NAME",
"arn:aws:s3:REGION:ACCOUNT:accesspoint/NAME/object/*"
],
"Condition": {
"ForAnyValue:StringEquals": {
"aws:CalledVia": "s3-object-lambda.amazonaws.com"
}
}
}
]
}The S3 bucket wants to permit entry to the supporting entry level. I choose the bucket and replace the coverage within the Permissions tab:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "*",
"Resource": [
"arn:aws:s3:::BUCKET",
"arn:aws:s3:::BUCKET/*"
],
"Condition": {
"StringEquals": {
"s3:KnowledgeAccessPointAccount": "ACCOUNT"
}
}
}
]
}Finally, CloudFront wants to have the ability to invoke the Lambda perform. In the Lambda console, I select the Lambda perform utilized by S3 Object Lambda, after which, within the Configuration tab, I select Permissions. In the Resource-based coverage statements part, I select Add permissions and choose AWS Account. I enter a singular Statement ID. Then, I enter cloudfront.amazonaws.com as Principal and choose lambda:InvokeFunction from the Action dropdown and Save. We are working to simplify this step sooner or later. I’ll replace this publish when that’s obtainable.
Testing the CloudFront Distribution
When the distribution has been deployed, I check that the setup is working with the identical pattern file I used earlier than. In the CloudFront console, I choose the distribution and replica the Distribution area title. I can use the browser and enter https://DISTRIBUTION_DOMAIN_NAME/s3.txt within the navigation bar to ship a request to CloudFront and get the file processed by S3 Object Lambda. To shortly get all the information, I take advantage of curl with the -i choice to see the HTTP standing and the headers within the response:
It works! As anticipated, the content material processed by the Lambda perform is all uppercase. Because that is the primary invocation for the distribution, it has not been returned from the cache (x-cache: Miss from cloudfront). The request went via S3 Object Lambda to course of the file utilizing the Lambda perform I offered.
Let’s attempt the identical request once more:
This time the content material is returned from the CloudFront cache (x-cache: Hit from cloudfront), and there was no additional processing by S3 Object Lambda. By utilizing S3 Object Lambda because the origin, the CloudFront distribution serves content material that has been processed by a Lambda perform and could be cached to cut back latency and optimize prices.
Resizing Images Using S3 Object Lambda and CloudFront
As I discussed in the beginning of this publish, one of many use instances that may be carried out utilizing S3 Object Lambda and CloudFront is picture transformation. Let’s create a CloudFront distribution that may dynamically resize a picture by passing the specified width and top as question parameters (w and h respectively). For instance:
For this setup to work, I must make two adjustments to the CloudFront distribution. First, I create a brand new cache coverage to incorporate question parameters within the cache key. In the CloudFront console, I select Policies within the navigation pane. In the Cache tab, I select Create cache coverage. Then, I enter a reputation for the cache coverage.
In the Query settings of the Cache key settings, I choose the choice to Include the next question parameters and add w (for the width) and h (for the peak).
Then, within the Behaviors tab of the distribution, I choose the default habits and select Edit.
There, I replace the Cache key and origin requests part:
- In the Cache coverage, I take advantage of the brand new cache coverage to incorporate the
wandhquestion parameters within the cache key. - In the Origin request coverage, use the
AllViewerExceptHostHeadermanaged coverage to ahead question parameters to the origin.
Now I can replace the Lambda perform code. To resize photographs, this perform makes use of the Pillow module that must be packaged with the perform when it’s uploaded to Lambda. You can deploy the perform utilizing a software just like the AWS SAM CLI or the AWS CDK. Compared to the earlier instance, this perform additionally handles and returns HTTP errors, comparable to when content material isn’t discovered within the bucket.
import io
import boto3
from urllib.request import urlopen, HTTPError
from PIL import Image
from urllib.parse import urlparse, parse_qs
s3 = boto3.consumer('s3')
def lambda_handler(occasion, context):
print(occasion)
object_get_context = occasion['getObjectContext']
request_route = object_get_context['outputRoute']
request_token = object_get_context['outputToken']
s3_url = object_get_context['inputS3Url']
# Get object from S3
attempt:
original_image = Image.open(urlopen(s3_url))
besides HTTPError as err:
s3.write_get_object_response(
StatusCode=err.code,
ErrorCode="HTTPError",
ErrorMessage=err.cause,
RequestRoute=request_route,
RequestToken=request_token)
return
# Get width and top from question parameters
user_request = occasion['userRequest']
url = user_request['url']
parsed_url = urlparse(url)
query_parameters = parse_qs(parsed_url.question)
attempt:
width, top = int(query_parameters['w'][0]), int(query_parameters['h'][0])
besides (KeyError, ValueError):
width, top = 0, 0
# Transform object
if width > 0 and top > 0:
transformed_image = original_image.resize((width, top), Image.ANTIALIAS)
else:
transformed_image = original_image
transformed_bytes = io.BytesIO()
transformed_image.save(transformed_bytes, format="JPEG")
# Write object again to S3 Object Lambda
s3.write_get_object_response(
Body=transformed_bytes.getvalue(),
RequestRoute=request_route,
RequestToken=request_token)
returnI add an image I took of the Trevi Fountain within the supply bucket. To begin, I generate a small thumbnail (200 by 150 pixels).
https://DISTRIBUTION_DOMAIN_NAME/trevi-fountain.jpeg?w=200&h=150
Now, I ask for a barely bigger model (400 by 300 pixels):
It works as anticipated. The first invocation with a selected measurement is processed by the Lambda perform. Further requests with the identical width and top are served from the CloudFront cache.
Availability and Pricing
Aliases for S3 Object Lambda Access Points can be found in the present day in all business AWS Regions. There is not any extra price for aliases. With S3 Object Lambda, you pay for the Lambda compute and request fees required to course of the info, and for the info S3 Object Lambda returns to your software. You additionally pay for the S3 requests which are invoked by your Lambda perform. For extra data, see Amazon S3 Pricing.
Aliases at the moment are mechanically generated when an S3 Object Lambda Access Point is created. For current S3 Object Lambda Access Points, aliases are mechanically assigned and prepared to be used.
It’s now simpler to make use of S3 Object Lambda with current purposes, and aliases open many new potentialities. For instance, you should use aliases with CloudFront to create a web site that converts content material in Markdown to HTML, resizes and watermarks photographs, or masks personally identifiable data (PII) from textual content, photographs, and paperwork.
Customize content material on your finish customers utilizing S3 Object Lambda with CloudFront.
— Danilo