New – AWS CloudPath Lake Supports Ingesting Activity Events From Non-AWS Sources

In November 2013, we introduced AWS CloudPath to trace consumer exercise and API utilization. AWS CloudPath allows auditing, safety monitoring, and operational troubleshooting. CloudPath information consumer exercise and API calls throughout AWS companies as occasions. CloudPath occasions allow you to reply the questions of “who did what, where, and when?”.

Recently we’ve improved the flexibility so that you can simplify your auditing and safety evaluation through the use of AWS CloudPath Lake. CloudPath Lake is a managed knowledge lake for capturing, storing, accessing, and analyzing consumer and API exercise on AWS for audit, safety, and operational functions. You can mixture and immutably retailer your exercise occasions, and run SQL-based queries for search and evaluation.

We have heard your suggestions that aggregating exercise data from numerous functions throughout hybrid environments is advanced and dear, however essential for a complete image of your group’s safety and compliance posture.

Today we’re saying help of ingestion for exercise occasions from non-AWS sources utilizing CloudPath Lake, making it a single location of immutable consumer and API exercise occasions for auditing and safety investigations. Now you’ll be able to consolidate, immutably retailer, search, and analyze exercise occasions from AWS and non-AWS sources, equivalent to in-house or SaaS functions, in a single place.

Using the brand new PutAuditEvents API in CloudPath Lake, you’ll be able to centralize consumer exercise data from disparate sources into CloudPath Lake, enabling you to research, troubleshoot and diagnose points utilizing this knowledge. CloudPath Lake information all occasions in standardized schema, making it simpler for customers to eat this data to comprehensively and shortly reply to safety incidents or audit requests.

CloudPath Lake can also be built-in with chosen AWS Partners, equivalent to Cloud Storage Security, Clumio, CrowdStrike, CyberArk, GitHub, Kong Inc, LaunchDarkly, MontyCloud, Netskope, Nordcloud, Okta, One Identity, Shoreline.io, Snyk, and Wiz, permitting you to simply allow audit logging by way of the CloudPath console.

Getting Started to Integrate External Sources
You can begin to ingest exercise occasions from your personal knowledge sources or accomplice functions by selecting Integrations beneath the Lake menu within the AWS CloudPath console.

To create a brand new integration, select Add integration and enter your channel identify. You can select the accomplice software supply from which you wish to get occasions. If you’re integrating with occasions from your personal functions hosted on-premises or within the cloud, select My customized integration.

For Event supply location, you’ll be able to select locations to your occasions from this integration. This permits your software or companions to ship occasions to your occasion knowledge retailer of CloudPath Lake. An occasion knowledge retailer can retain your exercise occasions for every week to as much as seven years. Then you’ll be able to run queries on the occasion knowledge retailer.

Choose both Use current occasion knowledge shops or Create new occasion knowledge retailer—to obtain occasions from integrations. To be taught extra about occasion knowledge retailer, see Create an occasion knowledge retailer within the AWS documentation.

You may also arrange the permissions coverage for the channel useful resource created with this integration. The data required for the coverage depends on the mixing sort of every accomplice functions.

There are two sorts of integrations: direct and resolution. With direct integrations, the accomplice calls the PutAuditEvents API to ship occasions to the occasion knowledge retailer to your AWS account. In this case, it’s worthwhile to present External ID, the distinctive account identifier supplied by the accomplice. You can see a hyperlink to accomplice web site for the step-by-step information. With resolution integrations, the applying runs in your AWS account and the applying calls the PutAuditEvents API to ship occasions to the occasion knowledge retailer to your AWS account.

To discover the Integration sort to your accomplice, select the Available sources tab from the integrations web page.

After creating an integration, you’ll need to supply this Channel ARN to the supply or accomplice software. Until these steps are completed, the standing will stay as incomplete. Once CloudPath Lake begins receiving occasions for the built-in accomplice or software, the standing subject will probably be up to date to mirror the present state.

To ingest your software’s exercise occasions into your integration, name the PutAuditEvents API so as to add the payload of occasions. Be certain that there is no such thing as a delicate or personally figuring out data within the occasion payload earlier than ingesting it into CloudPath Lake.

You could make a JSON array of occasion objects, which features a required user-generated ID from the occasion, the required payload of the occasion as the worth of EventInformation, and an non-compulsory checksum to assist validate the integrity of the occasion after ingestion into CloudPath Lake.

{
  "AuditEvents": [
     {
      "Id": "event_ID",
      "EventData": "{event_payload}", "EventDataChecksum": "optional_checksum",
     },
   ... ]
}

The following instance reveals the right way to use the put-audit-events AWS CLI command.

$ aws cloudtrail-data put-audit-events 
--channel-arn $ChannelArn 
--external-id $DistinctiveExternalIDFromPartner 
--audit-events 
{
  "Id": "87f22433-0f1f-4a85-9664-d50a3545baef",
  "EventInformation":"{"eventVersion": .01","eventSource":"MyCustomLog2", ...}",
},
{
  "Id": "7e5966e7-a999-486d-b241-b33a1671aa74",
  "EventInformation":"{"eventVersion": .02","eventSource":"MyCustomLog1", ...}",
"EventDataChecksum":"848df986e7dd61f3eadb3ae278e61272xxxx",
}

On the Editor tab within the CloudPath Lake, write your personal queries for a brand new built-in occasion knowledge retailer to verify delivered occasions.

You could make your personal integration question, like getting all principals throughout AWS and exterior assets which have made API calls after a specific date:

SELECT consumerIdentity.principalId FROM $AWS_EVENT_DATA_STORE_ID 
WHERE eventTime > '2022-09-24 00:00:00'
UNION ALL
SELECT eventData.consumerIdentity.principalId FROM $PARTNER_EVENT_DATA_STORE_ID
WHRERE eventData.eventTime > '2022-09-24 00:00:00'

To be taught extra, see CloudTrail Lake occasion schema and pattern queries that can assist you get began.

Launch Partners
You can see the record of our launch companions to help a CloudPath Lake integration possibility within the Available functions tab. Here are weblog posts and bulletins from our companions who collaborated on this launch (some will probably be added within the subsequent few days).

Now Available
AWS CloudPath Lake now helps ingesting exercise occasions from exterior sources in all AWS Regions the place CloudPath Lake is out there right this moment. To be taught extra, see the AWS documentation and every accomplice’s getting began guides.

If you have an interest in turning into an AWS CloudPath Partner, you’ll be able to contact your ordinary accomplice contacts.

– Channy