Black Hat USA 2023 NOC: Network Assurance

The Black Hat Network Operations Center (NOC) gives a excessive safety, excessive availability community in some of the demanding environments on this planet – the Black Hat occasion.

The NOC companions are chosen by Black Hat, with Arista, Cisco, Corelight, Lumen, WebWitness and Palo Alto Networks delivering from Las Vegas this yr. We admire Iain Thompson of The Register, for taking time to attend a NOC presentation and tour the operations. Check out Iain’s article: ‘Inside the Black Hat network operations center, volunteers work in geek heaven.’

We additionally present built-in safety, visibility and automation: a SOC (Security Operations Center) contained in the NOC, with Grifter and Bart because the leaders.

Integration is vital to success within the NOC. At every convention, we’ve a hack-a-thon: to create, show, check, enhance and eventually put into manufacturing new or improved integrations. To be a NOC accomplice, you should be keen to collaborate, share API (Automated Programming Interface) keys and documentation, and are available collectively (at the same time as market opponents) to safe the convention, for the great of the attendees.

XDR (eXtended Detection and Response) Integrations

At Black Hat USA 2023, Cisco Secure was the official Mobile Device Management, DNS (Domain Name Service) and Malware Analysis Provider. We additionally deployed ThousandEyes for Network Assurance.

As the wants of Black Hat developed, so have the Cisco Secure Technologies within the NOC:

The Cisco XDR dashboard made it straightforward to see the standing of every of the related Cisco Secure applied sciences, and the standing of ThousandEyes brokers.

Below are the Cisco XDR integrations for Black Hat USA, empowering analysts to research Indicators of Compromise (IOC) in a short time, with one search. We admire alphaMountain.ai, Pulsedive and Recorded Future donating full licenses to the Black Hat USA 2023 NOC.

For instance, an IP tried AndroxGh0st Scanning Traffic towards the Registration server, blocked by Palo Alto Networks firewall.

Investigation of the IP confirmed it was identified malicious.

Also, the geo location in RU and identified affiliated domains. With this info, the NOC management accredited the shunning of the IP.

File Analysis and Teamwork within the NOC

Corelight and WebWitness extracted practically 29,000 recordsdata from the convention community stream, which have been despatched for evaluation in Cisco Secure Malware Analytics (Threat Grid).

It was humorous to see the variety of Windows replace recordsdata that have been downloaded at this premier cybersecurity convention. When file was convicted as malicious, we’d examine the context:

• Is it from a classroom, the place the subject is expounded to the conduct of the malware?
• Or, is from a briefing or a demo within the Business Hall?
• Is it propagating or confined to that single space?

The pattern above was submitted by Corelight and investigation confirmed a number of downloads within the coaching class Windows Reverse Engineering (+Rust) from Scratch (Zero Kernel & All Things In-between), a licensed exercise.

The ABCs of XDR within the NOC, by Ben Greenbaum

One of the numerous Cisco instruments in our Black Hat package was the newly introduced Cisco XDR. The highly effective, multi-faceted and dare I say it “extended” detection and response engine allowed us to simply meet the next objectives:

One of the much less public-facing advantages of this distinctive ecosystem is the flexibility for our engineers and product leaders to get face time with our friends at accomplice group, together with those who would usually – and rightfully – be thought-about our opponents. As at Black Hat occasions up to now, I acquired to take part in significant conversations concerning the intersection of utilization of Cisco and three^rd occasion merchandise, tweak our API plans and clearly categorical the wants we’ve from our accomplice applied sciences to higher serve our clients in widespread. This collaborative, cooperative mission permits all our groups to enhance the way in which our merchandise work, and the way in which they work collectively, for the betterment of our clients’ skills to satisfy their safety aims. Truly a singular state of affairs and one through which we’re grateful to take part.

Secure Cloud Analytics in XDR, by Adi Sankar

Secure Cloud Analytics (SCA) permits you to acquire the visibility and steady menace detection wanted to safe your public cloud, non-public community and hybrid atmosphere. SCA can detect early indicators of compromise within the cloud or on-premises, together with insider menace exercise and malware, in addition to coverage violations, misconfigured cloud belongings, and person misuse. These NDR (Network Detection and Response) capabilities have now change into native performance inside Cisco XDR. Cisco XDR was obtainable beginning July 31^st 2023, so it was a good time to place it by its paces on the Black Hat USA convention in August.

Cisco Telemetry Broker Deployment

Cisco Telemetry Broker (CTB) routes and replicates telemetry information from a supply location(s) to a vacation spot shopper(s). CTB transforms information protocols from the exporter to the patron’s protocol of selection and due to its flexibility CTB was chosen to pump information from the Black Hat community to SCA.

Typically, a CTB deployment requires a dealer node and a supervisor node. To cut back our on-prem foot print I proactively deployed a CTB supervisor node in AWS (Amazon Web Services) (though this deployment just isn’t obtainable for patrons but, cloud managed CTB is on the roadmap). Since the supervisor node was deployed already, we solely needed to deploy a dealer node on premise in ESXi.

With the 10G succesful dealer node deployed it was time to put in a particular plugin from engineering. This bundle just isn’t obtainable for patrons and remains to be in beta, however we’re fortunate sufficient to have engineering help to check out the newest and biggest expertise Cisco has to supply (Special shoutout to Junsong Zhao from engineering for his help). The plugin installs a circulation sensor inside a docker container. This permits CTB to ingest a SPAN from an Arista swap and rework it to IPFIX information. The circulation sensor plugin (previously Stealthwatch circulation sensor) makes use of a mixture of deep packet inspection and behavioral evaluation to determine anomalies and protocols in use throughout the community.

In addition to the SPAN, we requested that Palo Alto ship NetFlow from their Firewalls to CTB. This permits us to seize telemetry from the sting units’ egress interface giving us insights into visitors from the exterior web, inbound to the Blackhat community. In the CTB supervisor node I configured each inputs to be exported to our SCA tenant.

 

Private Network monitoring within the cloud

 

First, we have to configure SCA by turning on all of the NetFlow based mostly alerts. In this case it was already accomplished since we used the identical tenant for a Blackhat Singapore. However, this motion could be automated utilizing the API api/v3/alerts/publish_preferences/ by setting each “should_publish” and “auto_post_to_securex” to true within the payload. Next, we have to configure entity teams in SCA to correspond with inside Blackhat community. Since subnets can change convention to convention, I automated this configuration utilizing a workflow in XDR Automate.

The subnets are documented in a CSV file from which the workflow parses 3 fields: the CIDR of the subnet, a reputation and an outline. Using these fields to execute a POST name to the SCA /v3/entitygroups/entitygroups/ API creates the corresponding entity teams. Much quicker than manually configuring 111 entity teams!

Now that we’ve community telemetry information flowing to the cloud SCA can create detections in XDR. SCA begins with observations which flip into alerts that are then correlated into assault chains earlier than lastly creating an Incident. Once the incident is created it’s submitted for precedence scoring and enrichment. Enrichment queries the opposite built-in applied sciences similar to Umbrella, Netwitness and menace intelligence sources concerning the IOC’s from the incident, bringing in extra context.

SCA detected 289 alerts together with Suspected Port Abuse, Internal Port Scanner, New Unusual DNS Resolver,and Protocol Violation (Geographic). SCA correlated 9 assault chains together with one assault chain with a complete of 103 alerts and 91 hosts on the community. These assault chains have been seen as incidents inside the XDR console and investigated by menace hunters within the NOC.

Conclusion

Cisco XDR collects telemetry from a number of safety controls, conducts analytics on that telemetry to reach at a detection of maliciousness, and permits for an environment friendly and efficient response to these detections. We used Cisco XDR to its fullest within the NOC from automation workflows, to analyzing community telemetry, to aggregating menace intelligence, investigating incidents, retaining observe of managed units and way more!

Hunter summer season camp is again. Talos IR menace looking throughout Black Hat USA 2023, by Jerzy ‘Yuri’ Kramarz

This is the second yr Talos Incident Response is supporting Network Operations Centre (NOC) through the Black Hat USA convention, in a menace looking capability.

My goal was to make use of multi-vendor expertise stacks to detect and cease ongoing assaults on key infrastructure externally and internally and determine potential compromises to attendees’ programs. To accomplish this, the menace looking crew targeted on answering three key hypothesis-driven questions and matched that with information modeling throughout totally different expertise implementations deployed within the Black Hat NOC:

• Are there any attendees making an attempt to breach one another’s programs in or exterior of a classroom atmosphere?
• Are there any attendees making an attempt to subvert any NOC Systems?
• Are there any attendees compromised, and will we warn them?

Like final yr, evaluation began with understanding how the community structure is laid out, and what sort of information entry is granted to NOC from varied companions contributing to the occasion. This is one thing that modifications yearly.

Great many thanks go to our pals from WebWitness, Corelight, Palo Alto Networks, Arista and Mandiant and plenty of others, for sharing full entry to their applied sciences to make sure that looking wasn’t contained to only Cisco gear and that contextual intelligence might be gathered throughout totally different safety merchandise. In addition to expertise entry, I additionally acquired nice assist and collaboration from accomplice groups concerned in Black Hat. In a number of instances, a number of groups have been contributing technical experience to determine and confirm potential indicators of compromise.

For our personal expertise stack, Cisco provided entry to Cisco XDR, Meraki, Cisco Secure Malware Analytics, Thousands Eyes, Umbrella and Secure Cloud Analytics (previously often called StealthWatch).

The Hunt

Our day by day menace hunt began with gathering information and searching on the connections, packets and varied telemetry gathered throughout your entire community safety stack in Cisco applied sciences and different platforms, similar to Palo Alto Networks or WebWitness XDR. Given the infrastructure was an agglomeration of assorted applied sciences, it was crucial to develop a menace looking course of which supported every of the distributors. By combining entry to shut to 10 totally different applied sciences, our crew gained a better visibility into visitors, however we additionally recognized a number of attention-grabbing situations of various units compromised on the Black Hat community.

One such instance was an AsyncRat-compromised system discovered with WebWitness XDR, based mostly on a particular key phrase positioned within the SSL certificates. As seen within the screenshot under, the device permits for highly effective deep-packet-inspection evaluation.

After optimistic identification of the AsyncRat exercise, we used the Arista wi-fi API to trace the person to a particular coaching room and notified them about the truth that their machine seemed to be compromised. Sometimes some of these actions could be a part of a Black Hat coaching lessons, however on this case, it appeared evident that the person was unaware of the legit compromise. This little snippet of code helped us discover out the place attendees have been within the lecture rooms, based mostly on Wireless AP connection, so we might notify them about their compromised programs.

Throughout our evaluation we additionally recognized one other occasion of direct malware compromise and associated community communication which matched the exercise of an AutoIT.F trojan speaking over a command and management (C2) to a well-know malicious IP [link to a JoeBox report]. The C2 the adversary used was checking on TCP ports 2842 and 9999. The instance of AutoIT.F trojan request, noticed on the community could be discovered under.

Above visitors pattern was decoded, to extract C2 visitors file and the next decoded strings seemed to be the ultimate payload. Notice that the payload included {hardware} specification, construct particulars and system title together with different particulars.

Likewise, on this case, we managed to trace the compromised system by the Wi-Fi connection and notifiy the person that their system seemed to be compromised.

Clear Text authentication nonetheless exists in 2023

Although in a roundabout way associated to malware an infection, we did uncover a number of different attention-grabbing findings throughout our menace hunt, together with quite a few examples of clear textual content visitors disclosing e mail credentials or authentication session cookies for number of purposes. In some situations, it was attainable to watch clear-text LDAP bind makes an attempt which disclosed which group the machine belonged to or direct publicity of the username and password mixture by protocols similar to POP3, LDAP, HTTP (Hyper Text Transfer Protocol) or FTP. All these protocols could be simply subverted by man-in-the-middle (MitM) assaults, permitting an adversary to authenticate towards companies similar to e mail. Below is an instance of the plain textual content authentication credentials and different particulars noticed by varied platforms obtainable at Black Hat.

Other examples of clear textual content disclosure have been noticed by way of fundamental authentication which merely used base64 to encode the credentials transmitted over clear textual content. An instance of this was observed with an Urban VPN (Virtual Private Network) supplier which seems to seize configuration recordsdata in clear textual content with fundamental authentication.

Just a few different situations of assorted clear textual content protocols similar to IMAP have been additionally recognized on the community, which we have been stunned to nonetheless be use in 2023.

What was attention-grabbing to see is that a number of trendy cellular purposes, similar to iPhone Mail, are glad to simply accept poorly configured e mail servers and use insecure companies to serve fundamental functionalities, similar to e mail studying and writing. This resulted in quite a few emails being current on the community, as seen under:

This yr, we additionally recognized a number of cellular purposes that not solely supported insecure protocols similar to IMAP, but in addition carried out direct communication in clear textual content, speaking every thing in clear textual content, together with person photos, as famous under:

In a number of situations, the cellular software additionally transmitted an authentication token in clear textual content:

Even extra attention-grabbing was the truth that we’ve recognized a number of distributors making an attempt to obtain hyperlinks to patches over HTTP, as nicely. In some situations, we’ve seen unique requests despatched over HTTP protocol with the “Location” header response in clear textual content pointing to an HTTPS location. Although I might count on these patches to be signed, speaking over HTTP makes it fairly straightforward to change the visitors in MitM situation to redirect downloads to separate places.

There have been quite a few different examples of HTTP protocol used to carry out operations similar to studying emails by webmail portals or downloading PAC recordsdata which disclose inside community particulars as famous on the screenshots under.

Cisco XDR expertise in motion

In addition to the standard expertise portfolio provided by Cisco and its companions, this yr was additionally the primary yr I had the pleasure of working with Cisco XDR console, which is a brand new Cisco product. The concept behind XDR is to provide a single “pane of glass” overview of all of the totally different alerts and applied sciences that work collectively to safe the atmosphere. Some of Cisco’s safety merchandise similar to Cisco Secure Endpoint for iOS and Umbrella have been related to by way of XDR platform and shared their alerts, so we might use these to realize a fast understanding of every thing that’s occurring on community from totally different applied sciences. From the menace looking perspective, this permits us to rapidly see the state of the community and what different units and applied sciences is likely to be compromised or execute suspicious actions.

While inside visitors, we additionally discovered and plotted fairly a number of totally different port scans working throughout the interior and exterior community. While we’d not cease these except they have been sustained and egregious, it was attention-grabbing to see totally different makes an attempt by college students to search out ports and units throughout networks. Good factor that community isolation was in place to forestall that.

The instance under exhibits fast exterior investigation utilizing XDR, which resulted in profitable identification of any such exercise. What triggered the alert was a sequence of occasions which recognized scanning and the truth that suspected IP additionally had relationships with a number of malicious recordsdata seen in VirusTotal:

Based on this evaluation, we rapidly confirmed that port scanning is certainly legitimate and decided which units have been impacted, as seen under. This, mixed with visibility from different instruments similar to Palo Alto Networks boundary firewalls, gave us stronger confidence in our raised alerts. The further contextual info associated to malicious recordsdata additionally allowed us to substantiate that we’re coping with a suspicious IP.

Throughout the Black Hat convention, we noticed many various assaults spanning throughout totally different endpoints. It was useful to have the ability to filter on these assaults rapidly to search out the place the assault originated and whether or not it was a real optimistic.

Using the above view, it was additionally attainable to instantly observe what contributed to the calculation of malicious rating and what sources of menace intelligence might be used to determine how was the malicious rating calculated for every of the elements that made up the general alert.

It’s not nearly inside networks

In phrases of the exterior assaults, Log4J, SQL injections, OGLN exploitation makes an attempt, and every kind of enumeration have been a day by day incidence on the infrastructure and the purposes used for attendee registration, together with different typical web-based assaults similar to path traversals. The following desk summarizes among the noticed among the efficiently blocked assaults the place we’ve seen the most important quantity. Again, our due to Palo Alto Networks for giving us entry to their Panorama platform, so we are able to observe varied assaults towards the Black Hat infrastructure.

Overall, we noticed a sizeable variety of port scans, floods, probes and every kind of internet software exploitation makes an attempt exhibiting up day by day at varied peak hours. Fortunately, all of them have been efficiently recognized for context (is that this a part of a coaching class or demonstration?) and contained (if applicable) earlier than inflicting any hurt to exterior programs. We even had a suspected Cobalt Strike server (179.43.189[.]250) [link to VirusTotal report] scanning our infrastructure and searching for particular ports similar to 2013, 2017, 2015 and 2022. Given the truth that we might intercept boundary visitors and examine particular PCAP (packet seize) dumps, we used all these assaults to determine varied C2 servers for which we additionally hunted internally, to make sure that no inside system is compromised.

Network Assurance, by Ryan MacLennan and Adam Kilgore

Black Hat USA 2023 is the primary time we deployed a brand new community efficiency monitoring answer named ThousandEyes. There was a proof of idea of ThousandEyes capabilities at Black Hat Asia 2023, investigating a report of sluggish community entry. The investigation recognized the difficulty was not with the community, however with the latency in connecting to a server in Ireland from Singapore. We have been requested to proactively convey this community visibility and assurance to Las Vegas.

ThousandEyes makes use of each stationary Enterprise Agents and cellular Endpoint Agents to measure community efficiency standards like availability, throughput, and latency. The picture under exhibits among the metrics captured by ThousandEyes, together with common latency info within the high half of the picture, and Layer 3 hops within the backside half of the picture with latency tracked for every community leg between the Layer 3 hops.

The ThousandEyes internet GUI can present information for one or many TE brokers. The screenshot under exhibits a number of brokers and their respective paths from their deployment factors to the Black Hat.com web site.

We additionally created a set of customized ThousandEyes dashboards for the Black Hat conference that tracked combination metrics for the entire deployed brokers.

ThousandEyes Deployment

Ten ThousandEyes Enterprise Agents have been deployed for the convention. These brokers have been moved all through totally different convention areas to watch community efficiency for essential occasions and companies. Endpoint Agents have been additionally deployed on laptops of NOC technical affiliate personnel and used for cellular diagnostic info in several investigations.

Coming into Black Hat with information of how the convention might be arrange was key in figuring out how we’d deploy ThousandEyes. Before we arrived on the convention, we made a preliminary plan on how we’d deploy brokers across the convention. This included what sort of machine would run the agent, the connection sort, and tough places of the place they’d be arrange. In the picture under you possibly can see we deliberate to deploy ThousandEyes brokers on Raspberry Pi’s and a Meraki MX equipment

The plan was to run all of the brokers on the wi-fi community. Once we arrived on the convention, we began prepping the Pi’s for the ThousandEyes picture that was supplied within the UI (User Interface). The under picture exhibits us getting the Pi’s out of their packaging and setting them up for the imaging course of. This included putting in heatsinks and a fan.

After all of the Pi’s have been prepped, we began flashing the ThousandEyes (TE) picture onto every SD-Card. After flashing the SD-Cards, we wanted besides them up, get them related to the dashboard after which work on enabling the wi-fi. While we had a enterprise case that known as for wi-fi TE brokers on Raspberry Pi, we did must clear a hurdle or wi-fi not being formally supported for the Pi TE agent. We needed to undergo a strategy of unlocking (jailbreaking) the brokers, putting in a number of networking libraries to allow the wi-fi interface, after which create boot up scripts to start out the wi-fi interface, get it related, and alter the routing to default to the wi-fi interface. You can discover the code and information at this GitHub repository.

We confirmed that the wi-fi configurations have been working correctly and that they’d persist throughout boots. We began deploying the brokers across the convention as we deliberate and waited for all of them to come back up on our dashboard. Then we have been prepared to start out monitoring the convention and supply Network Assurance to Black Hat. At least that’s what we thought. About half-hour after every Pi got here up in our dashboard, it might mysteriously go offline. Now we had some points we wanted to troubleshoot.

Troubleshooting the ThousandEyes Raspberry Pi Deployment

Now that our Pi’s had gone offline, we wanted to determine what was happening. We took some again with us and allow them to run in a single day with one utilizing a wired connection and one on a wi-fi connection. The wi-fi one didn’t keep up all evening, whereas the wired one did. We observed that the wi-fi machine was considerably hotter than the wired one and this led us to the conclusion that the wi-fi interface was inflicting the Pi’s to overheat.

This conundrum had us confused as a result of we’ve our personal Pi’s, with no heatsinks or followers, utilizing wi-fi at house they usually by no means overheat. One concept we had was that the heatsinks weren’t cooling adequately as a result of the Pi kits we had used a thermal sticker as a substitute of thermal paste and clamp like a typical laptop. The different was that the fan was not pushing sufficient air out of the case to maintain the interior temperature low. We reconfigured the fan to make use of extra voltage and flipped the fan from pulling air out of the case to pushing air in and onto the elements. While a fan positioned instantly on a CPU ought to pull the new air off the CPU, orienting the Raspberry Pi case fan to blow cooler air instantly onto the CPU may end up in decrease temperatures. After re-orienting the fan, to blow onto the CPU, we didn’t have any new heating failures.

Running a few Pi’s with the brand new fan configuration all through the day proved to be the answer we wanted. With our mounted Pi’s now staying cooler, we have been in a position to full a steady deployment of ThousandEyes brokers across the convention.

ThousandEyes Use Case

Connectivity issues with the coaching rooms have been reported through the early days of the convention. We utilized a number of totally different strategies to gather diagnostic information instantly from the reported downside areas. While we had ThousandEyes brokers deployed all through the convention heart, downside reviews from particular person rooms typically required a direct strategy that introduced a TE agent on to the issue space, typically focusing on a particular wi-fi AP (Access Points) to gather diagnostic information from.

One particular use case concerned a report from the Jasmine G coaching room. A TE engineer traveled to Jasmine G and used a TE Endpoint Agent on a laptop computer to connect with the Wi-Fi utilizing the PSK assigned to the coaching room. The TE engineer talked to the coach, who shared a particular internet useful resource that their coaching session trusted. The TE engineer created a particular check for the room utilizing the web useful resource and picked up diagnostic information which confirmed excessive latency.

During the gathering of the info, the TE agent related to 2 totally different wi-fi entry factors close to the coaching room and picked up latency information for each paths. The connection by one of many APs confirmed considerably greater latency than the opposite AP, as indicated by the purple traces within the picture under.

ThousandEyes can generate searchable reviews based mostly on check information, similar to the info proven within the prior two screenshots. After capturing the check information above, a report was generated for the dataset and shared with the wi-fi crew for troubleshooting. 

Mobile Device Mangement, by Paul Fidler and Connor Loughlin

For the seventh consecutive Black Hat convention, we supplied iOS cellular machine administration (MDM) and safety. At Black Hat USA 2023, we have been requested to handle and safe:

• Registration: 32 iPads
• Session Scanning: 51 iPads
• Lead Retrieval: 550 iPhones and 300 iPads

When we arrived for arrange three days earlier than the beginning of the coaching lessons, our mission was to have a community up and working as quickly as is humanly attainable, so begin managing the 900+ units and examine their standing.

Wi-Fi Considerations

We needed to alter our Wi-Fi authentication schema. In the prior 4 Black Hat conferences, the iOS units have been provisioned with a easy PSK based mostly SSID that was obtainable all over the place all through the venue. Then, as they enrolled, they have been additionally pushed a certificates / Wi-Fi coverage (the place the machine then went off and requested a cert from a Meraki Certificate Authority, guaranteeing that the non-public key resided securely on the machine. At the identical time, the certificates title was additionally written into Meraki’s Cloud Radius.

As the machine now had TWO Wi-Fi profiles, it was now free to make use of its inbuilt prioritisation checklist (extra particulars right here) guaranteeing that the machine joined the safer of the networks (802.1x based mostly, quite than WPA2 / PSK based mostly). Once we have been certain that each one units have been on-line and checking in to MDM, we then eliminated the cert profile from the units that have been solely used for Lead Retrieval, because the purposes used for this have been web going through. Registration units hook up with an software that’s really on the Black Hat community, therefore the distinction in community necessities.

For Black Hat USA 2023, we simply didn’t have time to formulate a plan for the units that will enable those who wanted to have elevated community authentication capabilities (EAP-TLS in all probability), because the units weren’t connecting to a Meraki community anymore, which might have enabled them to make use of the Sentry functionality, however as a substitute an Arista community.

For the longer term, we are able to do one in all two issues:

1. Provision ALL units with the identical Wi-Fi creds (both Registration or Attendee) Wi-Fi on the time of enrolment and add the related safer creds (cert, perhaps) as they enroll to the Registration iPads ONLY
2. More laboriously, provision Registration units and Session Scanning / Lead Retrieval units with totally different credentials on the time of enrolment. This is much less optimum as:
   • We’d have to know forward of time which units are which used for Session Scanning, Lead Retrieval or Registration
   • It would introduce the prospect of units being provisioned with the flawed Wi-Fi community creds

When a Wi-Fi profile is launched on the time of Supervision, it stays on the machine always and can’t be eliminated, so possibility 2 actually does have the chance to introduce many extra points.

Automation – Renaming units

Again, we used the Meraki API and a script that goes off, for a given serial quantity, and renames the machine to match the asset variety of the machine. This has been fairly profitable and, when matched with a coverage exhibiting the Asset quantity on the Home Screen, makes discovering units fast. However, the spreadsheets can have information errors in them. In some instances, the anticipated serial quantity is the machine title and even an IMEI. Whilst we are able to specify MAC, Serial and SM machine ID as an identifier, we are able to’t (but) provide IMEI.

So, I’ve needed to amend my script in order that it, when it first runs, will get your entire checklist of enrolled units and a fundamental set of inventories, permitting us to lookup issues like IMEI, machine title, and many others., returning a FALSE if nonetheless not discovered or returning the Serial if discovered. This was then amended additional to go looking the Name key if IMEI didn’t return something. It might, theoretically, be expanded to incorporate any of the machine attributes! However, I feel we’d run rapidly into false positives.

The similar script was then copied and amended so as to add tags to units. Again, every machine has a persona:

• Registration
• Lead Retrieval
• Session Scanning

Each persona has a distinct display screen structure and software required. So, to make this versatile, we use tags in Meraki Systems Manager communicate. This signifies that in the event you tag a tool, and tag a setting or software, that machine will get that software, and so forth. As Systems Manager helps an entire bunch of tag varieties, this makes it VERY versatile with reference to advanced standards for who will get what!

However, manually tagging units within the Meraki Dashboard would take endlessly, so we are able to utilise an API to do that. I simply needed to change the API name being made for the renaming script, add a brand new column into the CSV with the tag title, and a few different sundry issues. However, it didn’t work. The downside was that the renaming API doesn’t care that the ID that’s used: MAC, Serial or SM Device ID. The Tagging API does, and you need to specify which ID that you simply’re utilizing. So, I’d modified the Alternative Device ID search methodology to return serial as a substitute of SM machine ID. Serial doesn’t exist when doing a tool lookup, however SerialNumber does! A fast edit and a number of other hundred units had been retagged.

Of course, subsequent time, all of this might be accomplished forward of time quite than on the convention! Having good information forward of time is priceless, however you possibly can by no means depend on it!

Caching Server

Downloading iOS 16.6 is a hefty 6GB obtain. And while the delta replace is a mere 260MB, that is nonetheless impactful on the community. Whilst the obtain takes a while, this might be massively improved through the use of a caching server. Whilst there’s many various ways in which this might be achieved, we’re going to analysis utilizing the caching functionality constructed into macOS (please see documentation right here). The rational for that is that:

1. It helps auto uncover, thus there’s no have to construct the content material caching on the fringe of the community. It could be constructed wherever, and the units will auto uncover this
2. It’s astoundingly easy to arrange
3. It might be caching each OS (Operating System) updates AND software updates

Whilst there wasn’t time to get this arrange for Black Hat USA 2023, this might be put into manufacturing for future occasions. The one factor we can’t clear up is the humongous period of time the machine must put together a software program replace for set up!

Wireless

Predictably (and I solely say that as a result of we had the identical situation final yr with Meraki as a substitute of Arista doing the Wi-Fi), the Registration iPads suffered from astoundingly poor obtain speeds and latency, which can lead to the Registration app hanging and attendees not having the ability to print their badges.

We have three necessities in Registration:

• General Attendee Wi-Fi
• Lead Retrieval and Session Scanning iOS units
• Registration iOS units

The situation stems from when each Attendee SSID and Registration SSID are being broadcast from the identical AP. It simply will get hammered, ensuing within the aforementioned points.

The takeaway from that is:

1. There must be a devoted SSID for Registration units
2. There must be a devoted SSID all through Black Hat for Sessions Scanning and Lead Retrieval (This could be the identical SSID, simply dynamic or id (naming modifications relying on vendor) PSK)
3. There must be devoted APs for the iOS units in heavy visitors areas and
4. There must be devoted APs for Attendees in heavy visitors areas

Lock Screen Message

Again, one other studying that got here too late. Because of the vulnerability that was mounted in iOS 16.6 (which got here out the very day that the units have been shipped from Choose2Rent to Black Hat, who ready them), a substantial period of time was spent updating the units. We can add a Lock Screen message to the units, which present states: ASSET # – SERIAL # Property of Swapcard

Given {that a} go to to a easy webpage was sufficient to make the machine weak, it was crucial that we up to date as many as we might.

However, while we might see with ease the OS model in Meraki Systems Manager, this wasn’t the case on the machine: You’d must go and open Settings > General > About to get the iOS Version.

So, the ideas occurred to me to make use of the Lock Screen Message to indicate the iOS model as nicely! We’d do that with a easy change to the profile. As the OS Version modifications on the machine, Meraki Systems Manager would see that the profile contents had modified and push the profile once more to the machine! One to implement for the subsequent Black Hat!

The Ugly….

On the night of the day of the Business Hall, there was a brand new model of the Black Hat / Lead Retrieval app printed within the Apple App Store. Unfortunately, in contrast to Android, there’s no profiles for Apple that decide the precedence of App updates from the App Store. There is, nevertheless, a command that may be issued to examine for and set up updates.

In three hours, we managed to get practically 25% of units up to date, however, if the person is utilizing the app on the time of the request, they’ve the ability to say no the replace.

The Frustrating…

For the primary time, we had a number of units go lacking. It’s unsure as as to whether these units are misplaced or stolen, however…

In previous Black Hat occasions, once we’ve had the synergy between System Manager and Meraki Wi-Fi, it’s been trivial, as inbuilding GPS (Global Positioning System) just isn’t existent, to have a single click on between machine and AP and vice versa. We’ve clearly misplaced that with one other vendor doing Wi-Fi, however, on the very least, we’ve been in a position to feed again the MAC of the machine and get an AP location.

However, the opposite irritating factor is that the units are NOT in Apple’s Automated Device Enrollment. This signifies that we lose among the safety performance: Activation Lock, the flexibility to drive enrollment into administration after a tool wipe, and many others.

All just isn’t misplaced although: Because the units are enrolled and supervised, we are able to put them into Lost Mode which locks the machine, permits us to place a persistent message on the display screen (even after reboot) and be sure that the cellphone has an audible warning even when muted.

You can discover the code and information at this GitHub repository and the information in this weblog submit.

SOC Cubelight, by Ian Redden

The Black Hat NOC Cubelight was impressed by a number of initiatives primarily the 25,000 LED Adafruit Matrix Cube (Overview | RGB LED Matrix Cube with 25,000 LEDs | Adafruit Learning System). Other than the mounting and orientation of this 5-sided dice, that’s the place the Cubelight differs from different initiatives.

The Raspberry Zero 2W powered gentle makes use of customized written Python to show alerts and statistics from:

• Cisco Umbrella
• WebWitness
  • Number of clear-text passwords noticed and protocol breakdown
  • TLS encrypted visitors vs non-encrypted visitors
• Cisco ThousandEyes
  • BGP Reachability
  • Total Alerts
  • DNS Resolution in milliseconds
  • HTTP Server Availability (%)
  • Endpoint Average Throughput (Mbps)
  • Endpoint Latency

Automating the Management of Umbrella Internal Networks, by Christian Clausen

The Black Hat community is in actual fact a group of over 100 networks, every devoted to logical segments together with the NOC infrastructure, particular person coaching lessons, and the general public attendee wi-fi. DNS decision for all these networks is supplied by Umbrella Virtual Appliances: native resolvers deployed onsite. These resolvers helpfully present the interior IP tackle (and subsequently community subnet) for DNS queries. This info is beneficial for enrichment within the SOAR and XDR merchandise utilized by NOC workers. But quite than having to manually reference a spreadsheet to map the particular community to a question, we are able to robotically label them within the Umbrella reporting information.

Cisco Umbrella permits for the creation of “Internal Networks” (a listing of subnets that map to a selected web site and label).

With these networks outlined, NOC workers can see the title of the community within the enriched SOAR and XDR information and have extra context when investigating an occasion. But manually creating so many networks can be error susceptible and time-consuming. Luckily, we are able to use the Umbrella API to create them.

The community definitions are maintained by the Black Hat NOC workers in a Google Sheet; and is repeatedly up to date because the community is constructed, and entry factors deployed. To sustain with any modifications, we leveraged the Google Sheets API to always ballot the community info and reconcile it with the Umbrella Internal Networks. By placing this all collectively in a scheduled process, we are able to maintain the community location information correct even because the deployment evolves and networks transfer.

DNS Visibility, Statistics, and Shoes by Alex Calaoagan

Another Black Hat has come and gone, and, if DNS visitors is any indication, this was by far the most important with near 80 million DNS requests made. In comparability, final yr we logged simply over 50 million. There are a number of elements within the soar, the first being that we now, due to Palo Alto Networks, seize customers that hardcode DNS on their machines. We did the identical factor in Singapore.

If you missed it, right here’s the gist: Palo Alto Networks NAT’ed the masked visitors by our Umbrella digital home equipment on web site. Traffic beforehand masked was now seen and trackable by VLAN. This added visibility improved the standard of our statistics, supplying information that was beforehand a black field. Check again in 2024 to see how this new info tracks.

Digging into the numbers, we witnessed simply over 81,000 safety occasions, an enormous drop off from latest years. 1.3 million requests have been logged final yr, nevertheless that quantity was closely pushed by Dynamic DNS and Newly Seen area occasions. Take away these two excessive quantity classes, and the numbers observe significantly better.

As at all times, we proceed to see an increase in app utilization at Black Hat:

• 2019: ~3,600
• 2021: ~2,600
• 2022: ~6,300
• 2023: ~7,500

Two years faraway from the pandemic, plainly Black Hat is again on its pure progress trajectory, which is superior to see.

Looking at Social Media utilization, you can too see that the group at Black Hat remains to be dominated by Gen X-ers and Millennials with Facebook being #1, although the Gen Z crowd is making their presence felt with TikTookay at #2. Or is that this a sign of social media managers being savvier? I’m guessing it’s a little bit of each.

Curious what courting app dominated Black Hat this yr? Tinder outpaced Grindr with over double the requests made.

Among the numerous developments I noticed on the present ground, one actually caught with me, and it’s one all Vendors hopefully paid shut consideration to.

Of all of the shows and demoes I watched or noticed gathered, one single giveaway drew the most important and most constant crowds (and most leads).

It’s an merchandise close to and pricey to my coronary heart, and if it’s not close to and pricey to your coronary heart, I’m certain it’s to somebody in your circle. Whether it’s on your youngsters, spouse, accomplice, or shut good friend, once you’re away out of your family members for an prolonged interval, nothing suits higher as an” I missed you” convention reward, except the attendee goes after it for themselves.

What is it, you ask? Shoes. Nikes to be particular. Jordans, Dunks, and Air Maxes to be much more particular. I counted three cubicles making a gift of customized kicks, and each drawing I witnessed (signed up for 2 myself) had crowds flowing into aisles, standing room solely. And sure, like somebody you possible know, I’m a Sneakerhead.

Black Hat has at all times had a pleasant subculture twang to it, although it has dulled over time. You don’t see many excessive mohawks or Viking hats today. Maybe that enjoyable nonetheless exists at Defcon, however Black Hat is now all Corporate, on a regular basis. Loads has modified since my first Black Hat at Caeser’s Palace in 2011, it truly is a disgrace. That’s why seeing sneaker giveaways makes me smile. They remind me of the subculture that outlined Black Hat again within the day.

The Black Hat present ground itself has change into a Nerd/Sneakerhead showcase. I noticed a pair of Tiffany Dunks and a number of other totally different iterations of Travis Scott’s collabs. I even noticed a pair of De La Soul Dunks (one in all my private favorites, and really uncommon). I feel excessive finish kicks have formally change into socially acceptable as enterprise informal, and it warms my coronary heart.

The ethical of this little commentary? Vendors, in the event you’re studying this and have had hassle within the lead gathering division, the reply is easy. Shoes. We want extra sneakers.

Cheers from Las Vegas ????.

—-

We are pleased with the collaboration of the Cisco crew and the NOC companions. Black Hat Europe might be in December 2023 on the London eXcel Centre. 

Acknowledgments

Thank you to the Cisco NOC crew:

• Cisco Secure: Christian Clasen, Alex Calaoagan, Aditya Sankar, Ben Greenbaum, Ryan Maclennan, Ian Redden, Adam Kilgore; with digital help by Steve Nowell
• Meraki Systems Manager: Paul Fidler and Connor Loughlin
• Talos Incident Response: Jerzy ‘Yuri’ Kramarz

Also, to our NOC companions: WebWitness (particularly David Glover, Iain Davidson and Alessandro Zatti), Palo Alto Networks (particularly Jason Reverri), Corelight (particularly Dustin Lee), Arista (particularly Jonathan Smith), Lumen and your entire Black Hat / Informa Tech workers (particularly Grifter ‘Neil Wyler,’ Bart Stump, Steve Fink, James Pope, Mike Spicer, Sandy Wenzel, Heather Williams, Jess Stafford and Steve Oldenbourg).

About Black Hat

For 26 years, Black Hat has supplied attendees with the very newest in info safety analysis, improvement, and developments. These high-profile world occasions and trainings are pushed by the wants of the safety group, striving to convey collectively the perfect minds within the business. Black Hat evokes professionals in any respect profession ranges, encouraging progress and collaboration amongst academia, world-class researchers, and leaders in the private and non-private sectors. Black Hat Briefings and Trainings are held yearly within the United States, Europe and USA. More info is obtainable at: Black Hat.com. Black Hat is dropped at you by Informa Tech.

---

We’d love to listen to what you assume. Ask a Question, Comment Below, and Stay Connected with Cisco Secure on social!

Cisco Secure Social Channels

Instagram
Facebook
Twitter
LinkedIn

Share: