Internet-Scale analysis of AWS Cognito Security

This talk will show the results of an internet-scale analysis of the security of AWS Cognito configurations. During this research it was possible to identify 2500 identity pools, which were used to gain access to more than 13000 S3 buckets (which are not publicly exposed), 1200 DynamoDB tables and 1500 Lambda functions.

The talk starts with an introduction to the AWS Cognito service and how it can be configured by the developers to give end-users direct access to AWS resources such as S3 and DynamoDB. Access is restricted by IAM policies which are under the developer's control and, in many cases, do not follow the least privilege principle.

The configuration weakness is first explained step by step for a specific AWS account and Cognito identity pool using a series of demos, the same concepts are then automated to perform an internet-scale analysis of AWS Cognito configurations.

Because Cognito identity pool IDs are UUID4 it was necessary to download thousands of APKs from the Google Play store, decompile them and extract the identifiers. Other sources such as Common Crawl were also used to identify valid identifiers. The tools used to perform these tasks will be made public.

Each Cognito identity pool that was configured with an unauthenticated role was analyzed using an in-depth permission brute-force tool that identifies potential breaches to least privilege principle.

The talk ends with recommendations for developers that want to configure the service in a secure manner, and an analysis of potential reasons for this widespread issue such as poor documentation and examples on AWS site.

Transcript

1. Internet-Scale analysis of AWS Cognito Security Black Hat 2019 @AndresRiancho
2. Patient zero Cloud Security Assessment
3. Stumble upon AWS Cognito During a white-box cloud security assessment I used my ReadOnly permissions and the CloudTrail logs to enumerate all AWS services used in the account Cognito appeared in the list Had no idea what it was AWS Cognito logo
4. Fell in love Identity pools enable you to grant your users access to AWS services Read the documentation and found…
5. Full AWS account compromise Using AWS Cognito misconfigurations I was able to compromise the AWS account in four steps. 1. Read the identity pool ID from the AWS console
6. cognito.get_aws_credentials() 2. Write boto3 code to get AWS credentials from the identity pool def get_pool_credentials(region, identity_pool): client = boto3.client('cognito-identity', region_name=region) _id = client.get_id(IdentityPoolId=identity_pool) _id = _id['IdentityId'] credentials = client.get_credentials_for_identity(IdentityId=_id) access_key = credentials['Credentials']['AccessKeyId'] secret_key = credentials['Credentials']['SecretKey'] session_token = credentials['Credentials']['SessionToken'] identity_id = credentials['IdentityId'] return access_key, secret_key, session_token, identity_id Identity pool ID (UUID4) AWS credentials
7. Privilege escalation 3. Enumerated permissions for the unauthenticated role 4. Escalated privileges to full account compromise using excessive Lambda Function permissions Reported this vulnerability during the assessment as: Least privilege principle not used in unauthenticated Cognito role.
8. Got one. Can I get them all? During the cloud security assessment I identified and exploited one instance of Cognito misconfiguration. A quick online search showed that there was no previous security research on AWS Cognito.
9. How many AWS accounts are at risk? Which are the most common and insecure permissions granted by developers? Is it possible to perform an Internet-Scale analysis of AWS Cognito security?
10. What's next Understand Cognito and it's vulnerabilities Grep the Internet Statistics Root cause analysis
11. Introduction to AWS Cognito Granting end-users access to AWS
12. What Is Amazon Cognito? Amazon Cognito provides authentication, authorization, and user management for your web and mobile apps. The two main components of Amazon Cognito are: • User pools are user directories that provide sign-up and sign-in options for your app users • Identity pools enable developers to grant end users access to AWS services
13. Amazon Cognito use case CoolCatPics mobile application wants to allow users to upload pictures directly to S3 and the associated metadata to be stored in DynamoDB. The application will authenticate users with a Cognito user pool and Facebook authentication. Authenticated users can write to the S3 bucket and DynamoDB, unauthenticated users can only list and view S3 bucket contents.
14. Amazon Cognito from mobile apps AWS Cognito identity pool provides users with AWS credentials to consume S3 and DynamoDB. The mobile application uses the AWS SDK for Android or iOS to interact with Cognito and once the credentials have been obtained consume the S3 and DynamoDB APIs.
15. AWS API from the browser CoolCatPics wants to have a web client for their application. In this scenario the same services are used: Cognito, S3 and DynamoDB, but the mobile application is replaced by the user's browser. AWS API calls are sent directly from the browser using the AWS JavaScript SDK.
16. Create new identity pool
17. Assign IAM roles to identities
18. IAM policy example { "Version": "2012-10-17", "Statement": [ { "Sid": "ListObjectsInBucket", "Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": ["arn:aws:s3:::bucket-name"] }, { "Sid": "AllObjectActions", "Effect": "Allow", "Action": "s3:*Object", "Resource": ["arn:aws:s3:::bucket-name/*"] } ] }
19. Created! AWS Cognito identity pool is ready to use. Notice that the identity pool identifiers are randomly generated UUID4. This was one of the main problems to solve when trying to perform an Internet-Scale analysis of AWS Cognito security because only users that know the UUID can interact with the identity pool.
20. Internet Scale Automation, automation, automation
21. Internet Scale analysis data = [] for identity_pool_id in get_identity_pools(the_internet): credentials = get_pool_credentials(identity_pool_id) permissions = enumerate_permissions(credentials) score = score_permissions(permissions) data_i = (identity_pool, credentials, permissions, score) data.append(data_i) pretty_graphs_and_stats(data)
22. Challenge #1: Identity Pool UUID4 Identity pool IDs are randomly generated and with enough entropy to discourage brute-force attacks. The solution is to extract these IDs from the client applications: • Google Play Store • Web applications ◦ Common Crawl ◦ GitHub ◦ Shodan ◦ Zoomeye ◦ Google ◦ Yandex
23. The initial plan was to download all apps from Google Play, decompile them and extract identity pool IDs. But Google play has ~2.6M apps and multiple protections against crawlers. Mobile apps which use the AWS SDK for Android were the most important ones, found a paid service that allows users to search Google play and filter by the libraries inside the APK.
24. The search results contained 13000 application names (eg. com.whatsapp). Decided to use alternative sites such as apkpure and apkmirror which are less restrictive with crawlers, this made the download process easier (no bypass of Google Play protections).
26. <script> window.AWS.config.region = 'us-east-1'; window.AWS.config.credentials = new window.AWS.CognitoIdentityCredentials({ IdentityPoolId: 'us-east-1:d917fd38-4...' }); The goal was to obtain all sites that use the AWS JavaScript SDK: <script src="/path/to/amazon-cognito-identity.min.js"></script> <script src="/path/to/aws-sdk-2.6.10.js"></script> Google only indexes text
27. Google only indexes text Identity pool IDs, AWS JavaScript SDK methods and classes have very specific patterns if only it would be possible to... grep the web... egrep -r -i -e '(...)' /huge-network-fs/internet/
29. Common crawl is an open repository of web crawl data that can be accessed and analyzed by anyone. Stats from the April crawl: 2.5 billion web pages or 198 TiB of uncompressed content
30. Tried to process the data with AWS Elastic MapReduce and cc-mrjob but found stability issues. Created cc-lambda, a tool that uses AWS Lambda to parallelize the process of searching through common crawl data. • 1000 concurrent AWS Lambda functions • Download warc archive, decompress, search using Python regular expression engine • Store matches in S3 egrep -r -i -e '(...)' /internet
31. $ python cc-lambda.py Overall progress: 1.55% Going to process 250 WARC paths Got futures from map(), waiting for results... crawl-data/CC-MAIN-2019-09/.../CC-MAIN-20190215183319-20190215205319-00000.warc.gz - Time (seconds): 191 - Processed pages: 44969 - Ignored pages: 93005 - Matches: {'aws_re_matcher': 9, 'cognito_matcher': 4}
32. cc-lambda uses the pywren library to abstract function return value storage, error handling, timeouts and and retries. The common crawl results are stored in S3 buckets which are part of the Amazon Public Datasets program, there is no cost for downloading all the data, and data transfer between S3 and Lambda is super fast. AWS Lambda function cost will be 95% of your bill when running this tool, the remaining 5% is the cost associated with storing matches in S3. ~300 USD
33. Other (boring) sources The research also included other sources, which were easier to consume and returned a very small number of matches: • GitHub • Shodan • Zoomeye • Google • Yandex
35. Challenge #2: Enumerate permissions In the AWS cloud there are two ways to enumerate permissions for a given credential set: • Use the IAM service to get the role's permissions. In most cases this will fail because the role itself has no permission for the IAM API. • Call each AWS API and analyze the response. Brute-force
36. Enumerate permissions / Avoiding jail • Enumerate Get* / List* / Describe*. Try anything else and you might change (break) the target AWS account. • Never send API calls that disclose user-data such as S3 bucket contents, DynamoDB table contents, etc.
37. Enumerate Permissions / Performance There are thousands of API calls, so speed quickly became an issue with existing tools. Pacu and several other tools and scripts perform permission enumeration but were missing at least one of the required features. Wrote enumerate-iam and integrated it into the main script • Threads and AWS service connection pool for performance • Dynamic test generation based on documentation found in the aws-js-sdk repository
38. enumerate-permissions.py $ ./enumerate-iam.py --access-key AKIA... --secret-key StF0q... [INFO] Starting permission enumeration for access-key-id "AKIA..." [INFO] -- gamelift.list_builds() worked! [INFO] -- sqs.list_queues() worked! ... [INFO] -- ec2.describe_addresses() worked!
39. Data and statistics
40. Privileges and roles This research focuses only on the unauthenticated roles associated with Cognito identity pools. Did not confirm, but common sense indicates that: The results obtained from this research would have been much worse if Cognito's authenticated role would have been part of the analysis. privileges(unauth_role) <= privileges(auth_role)
41. Identity pool sources Source Count Google Play 2627 GitHub 264 Common Crawl 167 Yandex 62 Zoomeye 35 Google 4 BuiltWith 1 Total 3160
42. Usable identity pools State Count Research 2504 Only authenticated users 308 Does not exist 245 Invalid configuration 103
43. Insecure configurations How many of the 2504 identity pools are poorly configured? But most importantly, how do we define “poorly configured”? For this it is important to remember the example use-case: the mobile application reads-writes to S3 and DynamoDB, and invokes lambda functions. The mobile application never needs to list_* because it knows where to store the data, which lambda functions to call, etc. In +80% cases anything matching list_* is not following the last privilege principle API Call Count % over total s3.list_buckets() 548 21.88% dynamodb.list_backups() 96 3.83% dynamodb.list_tables() 101 4.03% lambda.list_functions() 98 3.91%
44. Sensitive data
45. S3 buckets Data Count Sensitive 906 Other 12590 Total 12590
46. DynamoDB Data Count Sensitive 37 Other 1141 Total 1141
47. Lambda function environment variables Was able to list 1572 lambda functions using the lambda.list_functions() API call. The results show at least 78 environment variables that contain: • API keys for third-party services • AWS access keys • Database credentials • Passwords
48. Root cause analysis Why * 5
49. Insecure by default documentation Uploading Photos to Amazon S3 from a Browser is a tutorial on how to configure Cognito and AWS JS SDK to list, write, read and delete images to S3. • Allow unauthenticated Cognito role to s3:* on a specific bucket • Note added after contacting AWS security: This security posture is useful in this example to keep it focused on the primary goals of the example. In many live situations, however, tighter security, such as using authenticated users and object ownership, is highly advisable.
50. Restrictions on Unauthenticated Cognito roles Cognito allows only 26 services to be associated with the unauthenticated role. For example, it is impossible to use an IAM role with EC2:* for unauthenticated access. But the allowed services include DynamoDB, S3, IoT, Lambda, SimpleDB, SES, SNS and SQS.
51. Restrictions on authenticated Cognito roles Cognito imposes no restrictions on the permissions a developer can set on Cognito authenticated role. And in most Cognito applications users can create their own users by using the registration flow.
52. Developer can shoot himself in the foot The existing restrictions are not enough. The insecure Cognito configuration falls on the client's side of the shared responsibility model. But there is certainly a trend, a big percentage of Cognito identity pools are insecurely configured. AWS needs to review decisions made during the design phase. Changes in the user interface and configuration of the S3 service that prevent public S3 buckets is a good example of AWS revisiting their decisions.
53. Developers Least privilege principle
54. Least privilege principle and more... These are the most important tips for developers using Cognito: • Always follow the least privilege principle when configuring the IAM roles associated with Cognito. In other words, if the IAM policy contains * you are doing it wrong. • Remember object level permissions. This was not even discussed in this talk, but keep this in mind: Not all users should be able to read all objects in the S3 bucket, not all users should be able to read all rows in a DynamoDB table.
55. Bonus! Hard-coded credentials
56. Hard-coded credentials Since I was going to grep the internet… I included regular expressions to identify hard-coded AWS credentials! • Found 280 hard coded credentials ◦ 26 (9%) were root accounts ◦ 38 (13.5%) had high privileges that granted access to RDS, EC2 or IAM • Sources ◦ 25 findings came from common crawl ◦ 1 finding came from GitHub Reported them to AWS' security team. Customers were contacted and leaked credentials disabled.
57. Closing thoughts
58. Key takeaways These are the three most important things to remember: • AWS Cognito is commonly misconfigured and easy to exploit • Use cc-lambda to grep the Internet and identify vulnerabilities • Use enumerate-iam to enumerate IAM permissions in a fast, safe and in-depth manner Follow me on twitter @AndresRiancho for more interesting cloud security research
59. Future work This research could be extended as follows: • Extract identity pools from iOS applications • Similar services, maybe in Azure, GCP, Alibaba. • Authenticated role analysis • Privilege escalation analysis (danger!) I’m most likely not going to follow-up on all this, but feel free to contact me if you want to and need guidance.
60. Thanks!
61. For hire Does your company or startup need these services? • Cloud Security Assessment • Intro to AWS Hacking training • Application Penetration Test • Source Code Review Let me know, I can help you deliver secure web applications.