AWS Logs Insights

The basic parts of a Logs insights query is as follows.

1. Fields: These are keys of the AWS log message format. There are usually two values (when you look at the AWS console)

   1. timestamp: This is pretty straight forward.
   2. message: The entire logged message in JSON or plain text.
   3. logStream: The CloudWatch Logstream
   4. log: The specific log.

2. Parse: Parse the text of the message based on certain conditions. See below for details.

3. Filter: Use this to filter the output.

   For example: filter using timestamp

   filter @timestamp >= timestamp('2024-08-10T00:00:00Z')

   Filter using text in the message.

   filter @message like /ERROR|WARN/

   Filter using Log stream.

   filter @logStream like <log-stream-pattern>

4. Sort: Sort the output: For example; descending using timestamp

   sort @timestamp desc

5. Limit: Limit the output to the first x results.

   limit 30

6. Display: Display only certain fields

   display @timestamp,level

Parsing

Since the entire second field of logs insights is message , we can apply additional queries on the text (or JSON) message.

We’d need to apply a regex to parse the messages out of the logs.

| parse @message '"level":"*"' as level

{ "level": "info", "grpc.method": "GET" }

References

1. General
2. Querying