Ugly Green Logo

Container Registry

GitHub Organization

Slack Workspace

3 October 2022

Catching that VPC Flow Log Wave

by John Lukach

Millions, billions, trillions, or more of VPC Flow Logs got collected into a central S3 bucket. What can we do with the data that adds value to the business to support security and troubleshooting for continued collection justification?


Image Source:

Please ensure to capture Version 5 of VPC Flows to obtain the most valuable artifacts for tracing network connectivity across the environment.

Kinesis Firehose is a new feature released in September 2022 that works excellent for cross-account VPC Flow Log shipping.


Choose Apache Parquet as the Log format, a columnar data that provides a cost-effective option as Athena bills by the volume of data searched.

Enable Hive-compatible S3 prefixes to make adding new partitions more efficient without requiring additional Glue resources.


Paying by the GB requires a way to narrow the search without having to go back to the well. An index with the IP Address as the primary key and a sort key with additional characteristics are available to help pivot during the investigation.

If you need a way to operationalize your VPC Flows Logs, this option might be the way to go!

tags: AWS - CDK - Flows - Index - Lake - VPC