--- title: "Shorten your detection engineering feedback loops with Grimoire" description: "Announcing the release of Grimoire, a new open source tool from Datadog, designed to help accelerate detection engineering" author: "Christophe Tafani-Dereeper" date: 2024-08-09 category: "open source software" tags: ["open source", "threat detection"] url: "https://securitylabs.datadoghq.com/articles/announcing-grimoire/" --- Today, on the occasion of our DEFCON Cloud Village talk "[Catch Them All! Detection Engineering and Purple Teaming in the Cloud](https://dc32.cloud-village.org/#talks?collapseChristopheTafani-Dereeper)", we're excited to announce the release of [Grimoire](https://github.com/dataDog/grimoire)—an open source project that allows you to build data sets based on audit logs for common AWS attack techniques. ![Grimoire Logo](https://securitylabs.dd-static.net/img/announcing-grimoire/grimoire-logo.png?auto=format) ## Shortening the feedback loops of detection engineering When engineers craft new detection rules, the process usually involves three steps: 1. Reproducing ("detonating") the attack technique in a live environment 2. Analyzing generated logs, metrics and artifacts generated from step 1 3. Developing the detection rule and its underlying logic based on the findings However, this process can be challenging. In AWS environments, CloudTrail logs typically generate high volumes of data, making it difficult to sift through the noise and identify the specific, relevant events triggered by the detonation. Additionally, the workflow often requires context switching between a terminal, the AWS Console, and/or an external SIEM. Grimoire is our solution to streamline this process by shortening feedback loops and reducing context switching when creating detection rules. ## Meet Grimoire Grimoire is an open source project designed to simplify the identification of cloud audit logs generated by common attacks. It operates by first detonating an attack technique, then polling [CloudTrail event history](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html) to pinpoint the exact logs triggered by the attack. The project is written in Go and supports two methods for detonating attack techniques: 1. Transparently through [Stratus Red Team](https://stratus-red-team.cloud/), which leverages Grimoire’s growing catalog of 38 AWS attack techniques (and counting) 2. By spinning up an interactive shell, allowing you to manually use the AWS CLI to execute custom attacks In both scenarios, Grimoire injects a unique HTTP user-agent to accurately identify the specific CloudTrail events generated by the detonation. ![Grimoire and CloudTrail](https://securitylabs.dd-static.net/img/announcing-grimoire/grimoire-cloudtrail.png?auto=format) We are excited to announce that Grimoire is now open sourced under the permissive Apache 2.0 license. The project is available [on GitHub](https://github.com/DataDog/grimoire), and you can easily install it using go install: ```bash go install -v github.com/datadog/grimoire/cmd/grimoire@latest ``` Alternatively, you can install Grimoire using Homebrew: ```bash brew tap datadog/grimoire https://github.com/DataDog/grimoire brew install datadog/grimoire/grimoire ``` ## Getting started with Grimoire To get started with Grimoire, start by making sure you're authenticated against an AWS account. Once authenticated, you can select any [Stratus Red Team AWS attack technique](https://stratus-red-team.cloud/attack-techniques/AWS/) to detonate. For example, if you want to analyze the logs generated by the '[Create an administrative IAM User](https://stratus-red-team.cloud/attack-techniques/AWS/aws.persistence.iam-create-admin-user/)' attack, you can instruct Grimoire to detonate it and retrieve the relevant CloudTrail logs by using: ``` grimoire stratus-red-team --attack-technique aws.persistence.iam-create-admin-user ``` After a few minutes, Grimoire will locate the relevant CloudTrail logs and stream them to your output file. Once the detonation is complete, Stratus Red Team will automatically clean up any cloud resources that were created during the test. ![Grimoire and Stratus Red Team Demonstration](https://securitylabs.dd-static.net/img/announcing-grimoire/grimoire-stratus-demo.png?auto=format) ## Detonating custom attack techniques Grimoire also allows you to detonate custom attacks from an interactive shell. Running `grimoire shell` will launch a new shell where you can manually execute attacks using the AWS CLI. Grimoire injects a unique UUID into the user agent of the AWS CLI, enabling it to accurately identify the logs generated. Once you’ve finished running your commands, Grimoire will poll CloudTrail to retrieve the corresponding logs. ![Demonstration of a custom attack](https://securitylabs.dd-static.net/img/announcing-grimoire/grimoire-custom-attack-demo.png?auto=format) ## New Grimoire-powered "detonation logs" in the Stratus Red Team website Grimoire powers the latest addition to the Stratus Red Team website: detonation logs. For every AWS attack technique, documentation now includes the raw CloudTrail logs that this attack technique generates. ![Grimoire detonation logs](https://securitylabs.dd-static.net/img/announcing-grimoire/grimoire-detonation-logs.png?auto=format) *Grimoire detonation logs (click to enlarge)* ## What's next? We're eager to hear from you! If you have any questions, thoughts or suggestions about Grimoire, shoot us a message at [securitylabs@datadoghq.com](mailto:securitylabs@datadoghq.com) or open an issue. You can also [subscribe to our monthly newsletter](#subscribe) to receive our latest research in your inbox, or use our [RSS feed](https://securitylabs.datadoghq.com/rss/feed.xml).