Enumerating AWS Resources
As an Information Security practitioner, the first thing want to know WHERE and WHAT all of your resources are. In public clouds, you would THINK this would be an easy task. For Microsoft Azure, it is. But for AWS, it surprisingly is not.
Azure provides an “all resources” view which can span multiple subscriptions within the tenant. In AWS, you have to navigate to virtually every resource type (roughly 300+) in every geographic region (roughly 26) and look. Ouch! Even programattically, AWS does not provide “enumerate all resources” API. Surprised? I certainly was. I challenged a number of AWS experts and AWS Re:Invent with this very question, to see how they would approach the problem.
Alas, there are a few tricks that can help soften the impact of this odd
and frustrating limitation. In any of the solutions below, you will need to have full list/read access to all resource types in order to “see” them. Note that the built-in AWS ReadOnlyAccess policy does NOT have sufficient permission to see all resources.
UPDATE: On Nov 8, 2022, AWS announced AWS Resource Explorer, a managed capability that simplifies the search and discovery of your AWS resources across AWS Regions. It’s free, but the catch is you have to enable it by creating an index in each region. Furthermore, the API will only return 1000 resources (even with pagination).
Billing Data / Cost Explorer
The one definitiove source of truth is the billing data. The old saying is “follow the money”. The billing data lists everything that you pay for, and how much each item costs on a daily basis. While the billing data provides a list of all resources, it does NOT include the detailed configuration of each. But armed with the list of ARNs, it is easier to retrieve the details for each resource. The downside to this approach is that any resources created since the last billing day cycle won’t appear (generally 24 hours).
You can view the billing data in the AWS console (portal) using AWS Cost Explorer.
Tag Editor
Another tactical approach is to use the AWS Tag Editor to find some but not all resources. There are several limitations:
- The underlying GetResources API is only capable of enumerating resources which have one or more tags (or have been previously tagged).
SEE: https://docs.aws.amazon.com/resourcegroupstagging/latest/APIReference/API_GetResources.html - Not all AWS resources are capable of being tagged. This will only work with supported resources types. SEE: https://docs.aws.amazon.com/ARG/latest/userguide/supported-resources.html
For more informatioin on the AWS tag editor, see: https://docs.aws.amazon.com/ARG/latest/userguide/tag-editor.html
The AWS CLI can also be used for each region:
$ aws resourcegroupstaggingapi get-resources — region region_nameAWS Config
Yet another approach is to use AWS Config, which provides a detailed view of the configuration of AWS resources in each AWS account. This includes how the resources are related to one another and how they were configured in the
past so that you can see how the configurations and relationships change over time.
However, AWS Config only collects information about EC2/VPC-related resources, not everything in your AWS account. It does not cover services such as lambdas, SNS, SQS, etc.
The following AWS CLI can be done for each region and each resource type:
$ aws configservice list-discovered-resourcesDrawbacks here is that you must enable AWS config and even then it only captures resources as and when they change. AWS config can also be costly.
The AWS Config console page may be accessed at:
https://console.aws.amazon.com/config/home?region=us-east-1#/resource-listingCloudWatch
Another approach might be to use cloudwatch events over time to construct a view of all resources. A lambda function could be tied to the events which then manages the list of resources along with the latest configuration in a Neptune or RDS database. Food for thought…
Multiple AWS Accounts
Most organizations have multiple AWS accounts, which further exacerbates the resource enumeration challenge. Currently, the options for enumerating resources across multiple AWS accounts are limited to 3rd party tools,
some which are free / open source, in additon to paid solutions.
Resources
Open source tools (mostly dated) and conversation threads on the subject include:
- https://github.com/awslabs/aws-multi-account-viewer
- https://github.com/nccgroup/aws-inventory (dated solution)
- https://github.com/devops-israel/aws-inventory
- https://stackoverflow.com/questions/44391817/is-there-a-way-to-list-all-resources-in-aws
- https://forums.aws.amazon.com/thread.jspa?threadID=235322
- https://github.com/lesterw1/AwsServices (generates a list of services)
- BotoCore — Botocore is the open-source low-level interface to a growing number of Amazon Web Services. Botocore serves as the foundation for the AWS-CLI command line utilities.
In comparison, other cloud providers provide more efficient solutions to resource enumeration.
Microsoft Azure
A customer is able to see all resources across all their subscriptions in a single view in the Azure portal. There is an enumerate resources REST API (and PowerShell cmdlet) which will list all resources across all subscriptions in a given subscription. Additionally, Azure Resource Graph provides kusto queries which can be used to rapidly query all resources across all subscriptions within the tenant. Very impressive and very fast.
Google Cloud Platform (GCP)
All resources in Google Cloud Platform (GCP) are organized into a hierarchy under service accounts, with each node (Organizations, Folders, Projects, and so forth) having a reference to its parent. This makes it easier to programmatically iterate trhough and enumerate all resources using a single API. See: https://cloud.google.com/resource-manager/docs/listing-all-resources
Oracle Cloud
The Oracle Tenancy Explorer can show you all of the resources which you have created (and for some time also Terminated) in your Compartments. You can find it under Governance tab in OCI Console.
