|
This April, we introduced Amazon Bedrock as a part of a set of recent instruments for constructing with generative AI on AWS. Amazon Bedrock is a completely managed service that gives a alternative of high-performing basis fashions (FMs) from main AI corporations, together with AI21 Labs, Anthropic, Cohere, Stability AI, and Amazon, together with a broad set of capabilities to construct generative AI purposes, simplifying the event whereas sustaining privateness and safety.
Today, I’m joyful to announce that Amazon Bedrock is now typically accessible! I’m additionally excited to share that Meta’s Llama 2 13B and 70B parameter fashions will quickly be accessible on Amazon Bedrock.
Amazon Bedrock’s complete capabilities aid you experiment with a wide range of high FMs, customise them privately along with your information utilizing methods similar to fine-tuning and retrieval-augmented era (RAG), and create managed brokers that carry out advanced enterprise duties—all with out writing any code. Check out my earlier posts to be taught extra about brokers for Amazon Bedrock and the best way to join FMs to your organization’s information sources.
Note that some capabilities, similar to brokers for Amazon Bedrock, together with data bases, proceed to be accessible in preview. I’ll share extra particulars on what capabilities proceed to be accessible in preview in direction of the top of this weblog put up.
Since Amazon Bedrock is serverless, you don’t should handle any infrastructure, and you may securely combine and deploy generative AI capabilities into your purposes utilizing the AWS companies you’re already accustomed to.
Amazon Bedrock is built-in with Amazon CloudWatch and AWS CloudTrail to assist your monitoring and governance wants. You can use CloudWatch to trace utilization metrics and construct personalized dashboards for audit functions. With CloudTrail, you’ll be able to monitor API exercise and troubleshoot points as you combine different methods into your generative AI purposes. Amazon Bedrock additionally permits you to construct purposes which are in compliance with the GDPR and you should use Amazon Bedrock to run delicate workloads regulated underneath the U.S. Health Insurance Portability and Accountability Act (HIPAA).
Get Started with Amazon Bedrock
You can entry accessible FMs in Amazon Bedrock via the AWS Management Console, AWS SDKs, and open-source frameworks similar to LangChain.
In the Amazon Bedrock console, you’ll be able to browse FMs and discover and cargo instance use instances and prompts for every mannequin. First, that you must allow entry to the fashions. In the console, choose Model entry within the left navigation pane and allow the fashions you want to entry. Once mannequin entry is enabled, you’ll be able to check out totally different fashions and inference configuration settings to discover a mannequin that matches your use case.
For instance, right here’s a contract entity extraction use case instance utilizing Cohere’s Command mannequin:
The instance reveals a immediate with a pattern response, the inference configuration parameter settings for the instance, and the API request that runs the instance. If you choose Open in Playground, you’ll be able to discover the mannequin and use case additional in an interactive console expertise.
Amazon Bedrock gives chat, textual content, and picture mannequin playgrounds. In the chat playground, you’ll be able to experiment with varied FMs utilizing a conversational chat interface. The following instance makes use of Anthropic’s Claude mannequin:
As you consider totally different fashions, it’s best to attempt varied immediate engineering methods and inference configuration parameters. Prompt engineering is a brand new and thrilling talent targeted on the best way to higher perceive and apply FMs to your duties and use instances. Effective immediate engineering is about crafting the proper question to get essentially the most out of FMs and acquire correct and exact responses. In basic, prompts ought to be easy, simple, and keep away from ambiguity. You may present examples within the immediate or encourage the mannequin to cause via extra advanced duties.
Inference configuration parameters affect the response generated by the mannequin. Parameters similar to Temperature, Top P, and Top Ok provide you with management over the randomness and variety, and Maximum Length or Max Tokens management the size of mannequin responses. Note that every mannequin exposes a distinct however typically overlapping set of inference parameters. These parameters are both named the identical between fashions or related sufficient to cause via if you check out totally different fashions.
We talk about efficient immediate engineering methods and inference configuration parameters in additional element in week 1 of the Generative AI with Large Language Models on-demand course, developed by AWS in collaboration with DeepLearning.AI. You may examine the Amazon Bedrock documentation and the mannequin supplier’s respective documentation for added suggestions.
Next, let’s see how one can work together with Amazon Bedrock by way of APIs.
Using the Amazon Bedrock API
Working with Amazon Bedrock is so simple as deciding on an FM in your use case after which making just a few API calls. In the next code examples, I’ll use the AWS SDK for Python (Boto3) to work together with Amazon Bedrock.
List Available Foundation Models
First, let’s arrange the boto3 shopper after which use list_foundation_models() to see essentially the most up-to-date checklist of obtainable FMs:
import boto3
import json
bedrock = boto3.shopper(
service_name="bedrock",
region_name="us-east-1"
)
bedrock.list_foundation_models()Run Inference Using Amazon Bedrock’s InvokeModel API
Next, let’s carry out an inference request utilizing Amazon Bedrock’s InvokeModel API and boto3 runtime shopper. The runtime shopper manages the information aircraft APIs, together with the InvokeModel API.
The InvokeModel API expects the next parameters:
The modelId parameter identifies the FM you need to use. The request physique is a JSON string containing the immediate in your process, along with any inference configuration parameters. Note that the immediate format will differ based mostly on the chosen mannequin supplier and FM. The contentType and settle for parameters outline the MIME kind of the information within the request physique and response and default to software/json. For extra data on the most recent fashions, InvokeModel API parameters, and immediate codecs, see the Amazon Bedrock documentation.
Example: Text Generation Using AI21 Lab’s Jurassic-2 Model
Here is a textual content era instance utilizing AI21 Lab’s Jurassic-2 Ultra mannequin. I’ll ask the mannequin to inform me a knock-knock joke—my model of a Hello World.
bedrock_runtime = boto3.shopper(
service_name="bedrock-runtime",
region_name="us-east-1"
)
modelId = 'ai21.j2-ultra-v1'
settle for="software/json"
contentType="software/json"
physique = json.dumps(
{"immediate": "Knock, knock!",
"maxTokens": 200,
"temperature": 0.7,
"topP": 1,
}
)
response = bedrock_runtime.invoke_model(
physique=physique,
modelId=modelId,
settle for=settle for,
contentType=contentType
)
response_body = json.hundreds(response.get('physique').learn())Here’s the response:
You may use the InvokeModel API to work together with embedding fashions.
Example: Create Text Embeddings Using Amazon’s Titan Embeddings Model
Text embedding fashions translate textual content inputs, similar to phrases, phrases, or presumably giant items of textual content, into numerical representations, referred to as embedding vectors. Embedding vectors seize the semantic which means of the textual content in a high-dimension vector house and are helpful for purposes similar to personalization or search. In the next instance, I’m utilizing the Amazon Titan Embeddings mannequin to create an embedding vector.
immediate = "Knock-knock jokes are hilarious."
physique = json.dumps({
"enterText": immediate,
})
model_id = 'amazon.titan-embed-text-v1'
settle for="software/json"
content_type="software/json"
response = bedrock_runtime.invoke_model(
physique=physique,
modelId=model_id,
settle for=settle for,
contentType=content_type
)
response_body = json.hundreds(response['body'].learn())
embedding = response_body.get('embedding')The embedding vector (shortened) will look much like this:
[0.82421875, -0.6953125, -0.115722656, 0.87890625, 0.05883789, -0.020385742, 0.32421875, -0.00078201294, -0.40234375, 0.44140625, ...]
Note that Amazon Titan Embeddings is on the market at this time. The Amazon Titan Text household of fashions for textual content era continues to be accessible in restricted preview.
Run Inference Using Amazon Bedrock’s InvokeModelWithResponseStream API
The InvokeModel API request is synchronous and waits for your entire output to be generated by the mannequin. For fashions that assist streaming responses, Bedrock additionally gives an InvokeModelWithResponseStream API that allows you to invoke the desired mannequin to run inference utilizing the supplied enter however streams the response because the mannequin generates the output.
Streaming responses are notably helpful for responsive chat interfaces to maintain the consumer engaged in an interactive software. Here is a Python code instance utilizing Amazon Bedrock’s InvokeModelWithResponseStream API:
response = bedrock_runtime.invoke_model_with_response_stream(
modelId=modelId,
physique=physique)
stream = response.get('physique')
if stream:
for occasion in stream:
chunk=occasion.get('chunk')
if chunk:
print(json.hundreds(chunk.get('bytes').decode))Data Privacy and Network Security
With Amazon Bedrock, you’re in command of your information, and all of your inputs and customizations stay non-public to your AWS account. Your information, similar to prompts, completions, and fine-tuned fashions, just isn’t used for service enchancment. Also, the information isn’t shared with third-party mannequin suppliers.
Your information stays within the Region the place the API name is processed. All information is encrypted in transit with a minimal of TLS 1.2 encryption. Data at relaxation is encrypted with AES-256 utilizing AWS KMS managed information encryption keys. You may use your individual keys (buyer managed keys) to encrypt the information.
You can configure your AWS account and digital non-public cloud (VPC) to make use of Amazon VPC endpoints (constructed on AWS PrivateLink) to securely connect with Amazon Bedrock over the AWS community. This permits for safe and personal connectivity between your purposes operating in a VPC and Amazon Bedrock.
Governance and Monitoring
Amazon Bedrock integrates with IAM that can assist you handle permissions for Amazon Bedrock. Such permissions embrace entry to particular fashions, playground, or options inside Amazon Bedrock. All AWS-managed service API exercise, together with Amazon Bedrock exercise, is logged to CloudTrail inside your account.
Amazon Bedrock emits information factors to CloudWatch utilizing the AWS/Bedrock namespace to trace frequent metrics similar to InputTokenCount, OutputTokenCount, InvocationLatency, and (variety of) Invocations. You can filter outcomes and get statistics for a selected mannequin by specifying the mannequin ID dimension if you seek for metrics. This close to real-time perception helps you monitor utilization and value (enter and output token rely) and troubleshoot efficiency points (invocation latency and variety of invocations) as you begin constructing generative AI purposes with Amazon Bedrock.
Billing and Pricing Models
Here are a few issues round billing and pricing fashions to remember when utilizing Amazon Bedrock:
Billing – Text era fashions are billed per processed enter tokens and per generated output tokens. Text embedding fashions are billed per processed enter tokens. Image era fashions are billed per generated picture.
Pricing Models – Amazon Bedrock offers two pricing fashions, on-demand and provisioned throughput. On-demand pricing permits you to use FMs on a pay-as-you-go foundation with out having to make any time-based time period commitments. Provisioned throughput is primarily designed for big, constant inference workloads that want assured throughput in alternate for a time period dedication. Here, you specify the variety of mannequin items of a selected FM to fulfill your software’s efficiency necessities as defined by the utmost variety of enter and output tokens processed per minute. For detailed pricing data, see Amazon Bedrock Pricing.
Now Available
Amazon Bedrock is on the market at this time in AWS Regions US East (N. Virginia) and US West (Oregon). To be taught extra, go to Amazon Bedrock, examine the Amazon Bedrock documentation, discover the generative AI house at neighborhood.aws, and get hands-on with the Amazon Bedrock workshop. You can ship suggestions to AWS re:Post for Amazon Bedrock or via your traditional AWS contacts.
(Available in Preview) The Amazon Titan Text household of textual content era fashions, Stability AI’s Stable Diffusion XL picture era mannequin, and brokers for Amazon Bedrock, together with data bases, proceed to be accessible in preview. Reach out via your traditional AWS contacts for those who’d like entry.
(Coming Soon) The Llama 2 13B and 70B parameter fashions by Meta will quickly be accessible by way of Amazon Bedrock’s absolutely managed API for inference and fine-tuning.
Start constructing generative AI purposes with Amazon Bedrock, at this time!
— Antje