Logo

A powerful, easily deployable network traffic analysis tool suite for network security monitoring

Quick Start

Documentation

Components

Supported Protocols

Configuring

Arkime

Dashboards

Hedgehog Linux

Contribution Guide

Deploying Malcolm on Amazon Web Services (AWS)

Installing prerequisites

The sections below make use of various command line tools. Installation may vary from platform to platform; however, this section gives some basic examples of how to install these tools in Linux environments. Not every guide in this document requires each of the following commands.

$ curl -fsSL \
    -o /tmp/awscli.zip \
    "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip"
$ unzip -d /tmp /tmp/awscli.zip
…
$ sudo /tmp/aws/install
You can now run: /usr/local/bin/aws --version
$ aws --version
aws-cli/2.26.2 Python/3.13.2 Linux/6.1.0-32-amd64 exe/x86_64.ubuntu.24

$ curl -fsSL \
    -o /tmp/eksctl.tar.gz \
    "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_Linux_$(uname -m | sed 's/^x86_64$/amd64/').tar.gz"
$ tar -xzf /tmp/eksctl.tar.gz -C /tmp && rm /tmp/eksctl.tar.gz
$ sudo mv /tmp/eksctl /usr/local/bin/
$ eksctl version
0.207.0

$ curl -fsSL \
    -o /tmp/kubectl \
    "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/$(uname -m | sed 's/^x86_64$/amd64/' | sed 's/^aarch64$/arm64/')/kubectl"
$ chmod 755 /tmp/kubectl
$ sudo mv /tmp/kubectl /usr/local/bin/
$ kubectl version
Client Version: v1.32.3

$ curl -fsSL \
    -o /tmp/get_helm.sh \
    https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
$ chmod 700 /tmp/get_helm.sh
$ /tmp/get_helm.sh
$ helm version
version.BuildInfo{Version:"v3.17.3", GitCommit:"e4da49785aa6e6ee2b86efd5dd9e43400318262b", GitTreeState:"clean", GoVersion:"go1.23.7"}

$ PACKER_VERSION="$(curl -fsSL 'https://releases.hashicorp.com/packer/' | grep -Po 'href="/packer/[^"]+"' | sort --version-sort | cut -d'/' -f3 | tail -n 1)"
$ curl -fsSL \
    -o /tmp/packer.zip \
    "https://releases.hashicorp.com/packer/${PACKER_VERSION}/packer_${PACKER_VERSION}_linux_$(uname -m | sed 's/^x86_64$/amd64/' | sed 's/^aarch64$/arm64/').zip"
$ unzip -d /tmp /tmp/packer.zip
$ chmod 755 /tmp/packer
$ sudo mv /tmp/packer /usr/local/bin/
$ packer --version
Packer v1.12.0

Amazon EC2 Instance Types

Malcolm is a resource-intensive tool: instance types should meet Malcolm’s minimum system requirements. Some AWS EC2 instance types meeting recommended minimum requirements:

Installing Malcolm in an EC2 instance

This section outlines the process of using the AWS Command Line Interface (CLI) to instantiate an EC2 instance running Malcolm. This section assumes good working knowledge of Amazon Web Services (AWS).

Instance creation

These steps are to be run on a Linux, Windows, or macOS system in a command line environment with the AWS Command Line Interface (AWS CLI) installed. Users should adjust these steps to their own use cases in terms of naming resources, setting security policies, etc.

$ aws ec2 create-key-pair \
    --key-name malcolm-key \
    --query "KeyMaterial" \
    --output text > ./malcolm-key.pem

$ chmod 600 ./malcolm-key.pem

$ aws ec2 create-security-group \
    --group-name malcolm-sg \
    --description "Malcolm SG"
…

$ PUBLIC_IP=#.#.#.#

$ for PORT in 22 443; do \
    aws ec2 authorize-security-group-ingress \
        --group-name malcolm-sg \
        --protocol tcp \
        --port $PORT \
        --cidr $PUBLIC_IP/32; \
done
…

$ aws ec2 describe-images \
    --owners 099720109477 \
    --filters "Name=name,Values=ubuntu-minimal/images/*/ubuntu-noble-24.04-ARCH*" \
    --query "Images[*].[Name,ImageId,CreationDate]" \
    --output text | sort
…

$ aws ec2 run-instances \
    --image-id AMI_ID \
    --instance-type INSTANCE_TYPE \
    --key-name malcolm-key \
    --security-group-ids malcolm-sg \
    --block-device-mappings "[{\"DeviceName\":\"/dev/sda1\",\"Ebs\":{\"VolumeSize\":100,\"VolumeType\":\"gp3\"}}]" \
    --count 1 \
    --tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=Malcolm}]"
…

$ aws ec2 describe-instances \
    --filters "Name=tag:Name,Values=Malcolm" \
    --query "Reservations[].Instances[].{ID:InstanceId,IP:PublicIpAddress,State:State.Name}" \
    --output table
…

Malcolm setup

The next steps are to be run as the ubuntu user inside the EC2 instance, either connected via Session Manager or via SSH using the key pair created in the first step.

$ sudo apt-get -y update
…
$ sudo apt-get -y install --no-install-recommends \
    curl \
    unzip \
    python3 \
    python3-dialog \
    python3-dotenv \
    python3-pip \
    python3-ruamel.yaml
…

$ curl -OJsSLf https://github.com/cisagov/Malcolm/releases/latest/download/malcolm-25.04.0-docker_install.zip

$ ls -l malcolm*.zip
-rw-rw-r-- 1 ubuntu ubuntu 191053 Apr 10 14:26 malcolm-25.04.0-docker_install.zip

$ unzip malcolm-25.04.0-docker_install.zip
Archive:  malcolm-25.04.0-docker_install.zip
  inflating: install.py
  inflating: malcolm_20250401_225238_df27028c.README.txt
  inflating: malcolm_20250401_225238_df27028c.tar.gz
  inflating: malcolm_common.py
  inflating: malcolm_kubernetes.py
  inflating: malcolm_utils.py

"docker info" failed, attempt to install Docker? (Y / n): y

Attempt to install Docker using official repositories? (Y / n): y

Apply recommended system tweaks automatically without asking for confirmation? y
…

$ cd ~/malcolm

$ ./scripts/auth_setup

all        Configure all authentication-related settings
…

Running Malcolm

$ cd ~/malcolm

$ ./scripts/start
…
logstash-1 | [2025-04-10T15:03:28,294][INFO ][logstash.agent ] Pipelines running {:count=>6, :running_pipelines=>[:"malcolm-input", :"malcolm-output", :"malcolm-suricata", :"malcolm-enrichment", :"malcolm-beats", :"malcolm-zeek"], :non_running_pipelines=>[]}

Started Malcolm

Malcolm services can be accessed at https://<IP address>/
------------------------------------------------------------------------------

$ cd ~/malcolm

$ ./scripts/status
NAME                          IMAGE                                                      COMMAND                  SERVICE             CREATED         STATUS                   PORTS
malcolm-api-1                 ghcr.io/idaholab/malcolm/api:25.04.0-arm64                 "/usr/bin/tini -- /u…"   api                 7 minutes ago   Up 7 minutes (healthy)   5000/tcp
malcolm-arkime-1              ghcr.io/idaholab/malcolm/arkime:25.04.0-arm64              "/usr/bin/tini -- /u…"   arkime              7 minutes ago   Up 7 minutes (healthy)   8000/tcp, 8005/tcp, 8081/tcp
malcolm-arkime-live-1         ghcr.io/idaholab/malcolm/arkime:25.04.0-arm64              "/usr/bin/tini -- /u…"   arkime-live         7 minutes ago   Up 7 minutes (healthy)
malcolm-dashboards-1          ghcr.io/idaholab/malcolm/dashboards:25.04.0-arm64          "/usr/bin/tini -- /u…"   dashboards          7 minutes ago   Up 7 minutes (healthy)   5601/tcp
malcolm-dashboards-helper-1   ghcr.io/idaholab/malcolm/dashboards-helper:25.04.0-arm64   "/usr/bin/tini -- /u…"   dashboards-helper   7 minutes ago   Up 7 minutes (healthy)   28991/tcp
malcolm-file-monitor-1        ghcr.io/idaholab/malcolm/file-monitor:25.04.0-arm64        "/usr/bin/tini -- /u…"   file-monitor        7 minutes ago   Up 7 minutes (healthy)   3310/tcp, 8440/tcp
malcolm-filebeat-1            ghcr.io/idaholab/malcolm/filebeat-oss:25.04.0-arm64        "/usr/bin/tini -- /u…"   filebeat            7 minutes ago   Up 7 minutes (healthy)
malcolm-freq-1                ghcr.io/idaholab/malcolm/freq:25.04.0-arm64                "/usr/bin/tini -- /u…"   freq                7 minutes ago   Up 7 minutes (healthy)   10004/tcp
malcolm-htadmin-1             ghcr.io/idaholab/malcolm/htadmin:25.04.0-arm64             "/usr/bin/tini -- /u…"   htadmin             7 minutes ago   Up 7 minutes (healthy)   80/tcp
malcolm-keycloak-1            ghcr.io/idaholab/malcolm/keycloak:25.04.0-arm64            "/usr/bin/tini -- /u…"   keycloak            7 minutes ago   Up 7 minutes (healthy)   8080/tcp, 8443/tcp, 9000/tcp
malcolm-logstash-1            ghcr.io/idaholab/malcolm/logstash-oss:25.04.0-arm64        "/usr/bin/tini -- /u…"   logstash            7 minutes ago   Up 7 minutes (healthy)   5044/tcp, 9001/tcp, 9600/tcp
malcolm-netbox-1              ghcr.io/idaholab/malcolm/netbox:25.04.0-arm64              "/usr/bin/tini -- /u…"   netbox              7 minutes ago   Up 7 minutes (healthy)   9001/tcp
malcolm-nginx-proxy-1         ghcr.io/idaholab/malcolm/nginx-proxy:25.04.0-arm64         "/sbin/tini -- /usr/…"   nginx-proxy         7 minutes ago   Up 7 minutes (healthy)   0.0.0.0:443->443/tcp
malcolm-opensearch-1          ghcr.io/idaholab/malcolm/opensearch:25.04.0-arm64          "/usr/bin/tini -- /u…"   opensearch          7 minutes ago   Up 7 minutes (healthy)   9200/tcp, 9300/tcp, 9600/tcp, 9650/tcp
malcolm-pcap-capture-1        ghcr.io/idaholab/malcolm/pcap-capture:25.04.0-arm64        "/usr/bin/tini -- /u…"   pcap-capture        7 minutes ago   Up 7 minutes (healthy)
malcolm-pcap-monitor-1        ghcr.io/idaholab/malcolm/pcap-monitor:25.04.0-arm64        "/usr/bin/tini -- /u…"   pcap-monitor        7 minutes ago   Up 7 minutes (healthy)   30441/tcp
malcolm-postgres-1            ghcr.io/idaholab/malcolm/postgresql:25.04.0-arm64          "/sbin/tini -- /usr/…"   postgres            7 minutes ago   Up 7 minutes (healthy)   5432/tcp
malcolm-redis-1               ghcr.io/idaholab/malcolm/redis:25.04.0-arm64               "/sbin/tini -- /usr/…"   redis               7 minutes ago   Up 7 minutes (healthy)   6379/tcp
malcolm-redis-cache-1         ghcr.io/idaholab/malcolm/redis:25.04.0-arm64               "/sbin/tini -- /usr/…"   redis-cache         7 minutes ago   Up 7 minutes (healthy)   6379/tcp
malcolm-suricata-1            ghcr.io/idaholab/malcolm/suricata:25.04.0-arm64            "/usr/bin/tini -- /u…"   suricata            7 minutes ago   Up 7 minutes (healthy)
malcolm-suricata-live-1       ghcr.io/idaholab/malcolm/suricata:25.04.0-arm64            "/usr/bin/tini -- /u…"   suricata-live       7 minutes ago   Up 7 minutes (healthy)
malcolm-upload-1              ghcr.io/idaholab/malcolm/file-upload:25.04.0-arm64         "/usr/bin/tini -- /u…"   upload              7 minutes ago   Up 7 minutes (healthy)   22/tcp, 80/tcp
malcolm-zeek-1                ghcr.io/idaholab/malcolm/zeek:25.04.0-arm64                "/usr/bin/tini -- /u…"   zeek                7 minutes ago   Up 7 minutes (healthy)
malcolm-zeek-live-1           ghcr.io/idaholab/malcolm/zeek:25.04.0-arm64                "/usr/bin/tini -- /u…"   zeek-live           7 minutes ago   Up 7 minutes (healthy)

Deploying Malcolm on Amazon Elastic Kubernetes Service (EKS)

This section outlines the process of setting up a cluster on Amazon Elastic Kubernetes Service (EKS) using Amazon Web Services (AWS).

These instructions assume good working knowledge of AWS and EKS. Good documentation resources can be found in the AWS documentation, the EKS documentation and the EKS Workshop.

This section covers two deployment options: deploying Malcolm in a standard Kubernetes cluster on Amazon EKS, and deploying Malcolm with EKS on Fargate.

Deploying with EKS

$ aws ec2 create-vpc \
  --cidr-block 10.0.0.0/16 \
  --region us-east-1 \
  --tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=malcolm-vpc}]'
…

$ VPC_ID=$(aws ec2 describe-vpcs \
              --filters "Name=tag:Name,Values=malcolm-vpc" \
              --query "Vpcs[0].VpcId" \
              --output text)

$ echo $VPC_ID

$ aws ec2 create-subnet \
    --vpc-id $VPC_ID \
    --cidr-block 10.0.1.0/24 \
    --availability-zone us-east-1a \
    --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=public-subnet-1},{Key=kubernetes.io/role/elb,Value=1}]'
…

$ aws ec2 create-subnet \
    --vpc-id $VPC_ID \
    --cidr-block 10.0.2.0/24 \
    --availability-zone us-east-1b \
    --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=public-subnet-2},{Key=kubernetes.io/role/elb,Value=1}]'
…

$ aws ec2 create-subnet \
    --vpc-id $VPC_ID \
    --cidr-block 10.0.3.0/24 \
    --availability-zone us-east-1a \
    --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=private-subnet-1},{Key=kubernetes.io/role/internal-elb,Value=1}]'
…

$ aws ec2 create-subnet \
    --vpc-id $VPC_ID \
    --cidr-block 10.0.4.0/24 \
    --availability-zone us-east-1b \
    --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=private-subnet-2},{Key=kubernetes.io/role/internal-elb,Value=1}]'
…

$ PUBLIC_SUBNET_IDS=$(aws ec2 describe-subnets \
                          --filters "Name=vpc-id,Values=$VPC_ID" \
                                   "Name=tag:kubernetes.io/role/elb,Values=1" \
                          --query "Subnets[*].SubnetId" \
                          --output text)

$ PRIVATE_SUBNET_IDS=$(aws ec2 describe-subnets \
                          --filters "Name=vpc-id,Values=$VPC_ID" \
                                   "Name=tag:kubernetes.io/role/internal-elb,Values=1" \
                          --query "Subnets[*].SubnetId" \
                          --output text)

$ echo $PUBLIC_SUBNET_IDS
$ echo $PRIVATE_SUBNET_IDS

$ for SUBNET in $PUBLIC_SUBNET_IDS; do \
    aws ec2 modify-subnet-attribute \
      --subnet-id $SUBNET \
      --map-public-ip-on-launch; \
done

$ aws ec2 create-internet-gateway \
    --tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=malcolm-igw}]'
…

$ IGW_ID=$(aws ec2 describe-internet-gateways \
              --filters "Name=tag:Name,Values=malcolm-igw" \
              --query "InternetGateways[0].InternetGatewayId" \
              --output text)

$ echo $IGW_ID

$ aws ec2 attach-internet-gateway \
      --internet-gateway-id $IGW_ID \
      --vpc-id $VPC_ID

$ aws ec2 create-route-table \
    --vpc-id $VPC_ID \
    --tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=malcolm-public-rt}]' \
    --output text
…

$ PUBLIC_RT_ID=$(aws ec2 describe-route-tables \
    --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=malcolm-public-rt" \
    --query 'RouteTables[0].RouteTableId' \
    --output text)

$ echo $PUBLIC_RT_ID

$ for SUBNET in $PUBLIC_SUBNET_IDS; do \
  aws ec2 associate-route-table \
    --subnet-id $SUBNET \
    --route-table-id $PUBLIC_RT_ID; \
done
…

$ aws ec2 create-route \
  --route-table-id $PUBLIC_RT_ID \
  --destination-cidr-block 0.0.0.0/0 \
  --gateway-id $IGW_ID
…

$ aws ec2 allocate-address \
    --domain vpc \
    --output text
…

$ EIP_ALLOC_ID=$(aws ec2 describe-addresses \
    --filters "Name=domain,Values=vpc" \
    --query 'Addresses[0].AllocationId' \
    --output text)

$ echo $EIP_ALLOC_ID

$ aws ec2 create-nat-gateway \
    --subnet-id $(echo $PUBLIC_SUBNET_IDS | awk '{print $1}') \
    --allocation-id $EIP_ALLOC_ID \
    --output text
…

$ NAT_GW_ID=$(aws ec2 describe-nat-gateways \
    --filter "Name=subnet-id,Values=$(echo $PUBLIC_SUBNET_IDS | awk '{print $1}')" \
    --query 'NatGateways[0].NatGatewayId' \
    --output text)

$ echo $NAT_GW_ID

$ aws ec2 wait nat-gateway-available --nat-gateway-ids $NAT_GW_ID

$ aws ec2 create-route-table \
    --vpc-id $VPC_ID \
    --tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=malcolm-private-rt}]' \
    --output text
…

$ PRIVATE_RT_ID=$(aws ec2 describe-route-tables \
    --filter "Name=vpc-id,Values=$VPC_ID" \
    --query 'RouteTables[?Tags[?Key==`Name` && Value==`malcolm-private-rt`]].RouteTableId' \
    --output text)

$ echo $PRIVATE_RT_ID

$ for SUBNET in $PRIVATE_SUBNET_IDS; do \
    aws ec2 associate-route-table \
    --subnet-id $SUBNET \
    --route-table-id $PRIVATE_RT_ID; \
done
…

$ aws ec2 create-route \
    --route-table-id $PRIVATE_RT_ID \
    --destination-cidr-block 0.0.0.0/0 \
    --nat-gateway-id $NAT_GW_ID
…

# cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: malcolm-cluster
  region: us-east-1

vpc:
  id: $VPC_ID
  subnets:
    public:
      us-east-1a:
        id: $PUBLIC_SUBNET_ID_A
      us-east-1b:
        id: $PUBLIC_SUBNET_ID_B
    private:
      us-east-1a:
        id: $PRIVATE_SUBNET_ID_A
      us-east-1b:
        id: $PRIVATE_SUBNET_ID_B

nodeGroups:
  - name: private-nodes
    instanceType: t2.2xlarge
    desiredCapacity: 2
    minSize: 1
    maxSize: 3
    privateNetworking: true

$ export VPC_ID

$ export PUBLIC_SUBNET_ID_A=$(aws ec2 describe-subnets \
                                --filters "Name=vpc-id,Values=$VPC_ID" \
                                   "Name=tag:kubernetes.io/role/elb,Values=1" \
                                   "Name=availability-zone,Values=us-east-1a" \
                                --query "Subnets[*].SubnetId" \
                                --output text)

$ export PUBLIC_SUBNET_ID_B=$(aws ec2 describe-subnets \
                                --filters "Name=vpc-id,Values=$VPC_ID" \
                                   "Name=tag:kubernetes.io/role/elb,Values=1" \
                                   "Name=availability-zone,Values=us-east-1b" \
                                --query "Subnets[*].SubnetId" \
                                --output text)

$ export PRIVATE_SUBNET_ID_A=$(aws ec2 describe-subnets \
                                --filters "Name=vpc-id,Values=$VPC_ID" \
                                   "Name=tag:kubernetes.io/role/internal-elb,Values=1" \
                                   "Name=availability-zone,Values=us-east-1a" \
                                --query "Subnets[*].SubnetId" \
                                --output text)

$ export PRIVATE_SUBNET_ID_B=$(aws ec2 describe-subnets \
                                --filters "Name=vpc-id,Values=$VPC_ID" \
                                   "Name=tag:kubernetes.io/role/internal-elb,Values=1" \
                                   "Name=availability-zone,Values=us-east-1b" \
                                --query "Subnets[*].SubnetId" \
                                --output text)

$ envsubst < cluster.yaml | eksctl create cluster -f -
…

$ eksctl utils associate-iam-oidc-provider \
    --region=us-east-1 --cluster=malcolm-cluster --approve
…

$ kubectl create namespace malcolm
…

Deploying with EKS on Fargate

$ eksctl create cluster \
    --name malcolm-cluster \
    --region us-east-1 \
    --fargate \
    --vpc-nat-mode HighlyAvailable \
    --with-oidc \
    --vpc-cidr 10.0.0.0/16 \
    --node-private-networking

$ kubectl create namespace malcolm

$ for ROLE in $(grep -h role: ./Malcolm/kubernetes/*.yml | awk '{print $2}' | sort -u); do \
    eksctl create fargateprofile \
        --cluster malcolm-cluster \
        --region us-east-1 \
        --name malcolm-"$ROLE" \
        --namespace malcolm \
        --labels role="$ROLE"; \
done

$ PRIVATE_SUBNET_IDS=$(aws ec2 describe-subnets \
    --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:aws:cloudformation:logical-id,Values=SubnetPrivate*" \
    --query 'Subnets[*].SubnetId' --output text)

$ echo $PRIVATE_SUBNET_IDS

Common Steps for EKS Deployments

$ aws iam create-policy \
  --policy-name AmazonEKS_EFS_CSI_Driver_Policy \
  --policy-document "$(curl -fsSL 'https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/refs/heads/master/docs/iam-policy-example.json')"
…

$ eksctl create iamserviceaccount \
    --cluster malcolm-cluster \
    --namespace kube-system \
    --name efs-csi-controller-sa \
    --attach-policy-arn arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):policy/AmazonEKS_EFS_CSI_Driver_Policy \
    --approve \
    --override-existing-serviceaccounts \
    --region us-east-1
…

$ aws iam create-policy \
  --policy-name AmazonAWS_Load_Balancer_Controller_Policy \
  --policy-document "$(curl -fsSL 'https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/install/iam_policy.json')"
…

$ eksctl create iamserviceaccount \
    --cluster malcolm-cluster \
    --namespace kube-system \
    --name aws-alb-controller-sa \
    --attach-policy-arn arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):policy/AmazonAWS_Load_Balancer_Controller_Policy \
    --approve \
    --override-existing-serviceaccounts \
    --region us-east-1
…

$ helm repo add efs-csi-driver https://kubernetes-sigs.github.io/aws-efs-csi-driver
…
$ helm repo update
…
$ helm install efs-csi-driver efs-csi-driver/aws-efs-csi-driver \
  -n kube-system \
  --set controller.serviceAccount.create=false \
  --set controller.serviceAccount.name=efs-csi-controller-sa
…

$ aws efs create-file-system \
    --creation-token malcolm-efs \
    --encrypted \
    --region us-east-1 \
    --tags "Key=Name,Value=malcolm-efs" \
    --performance-mode generalPurpose \
    --throughput-mode bursting
…

$ EFS_ID=$(aws efs describe-file-systems --creation-token malcolm-efs \
    --query 'FileSystems[0].FileSystemId' --output text)

$ echo $EFS_ID

$ for AP in config opensearch opensearch-backup pcap runtime-logs suricata-logs zeek-logs; do \
    aws efs create-access-point \
            --file-system-id $EFS_ID \
            --client-token $(head -c 1024 /dev/urandom 2>/dev/null | tr -cd 'a-f0-9' | head -c 32) \
            --root-directory "Path=/malcolm/$AP,CreationInfo={OwnerUid=1000,OwnerGid=1000,Permissions=0770}" \
            --tags "Key=Name,Value=$AP"; \
done
…

$ VPC_ID=$(aws eks describe-cluster --name malcolm-cluster \
        --query "cluster.resourcesVpcConfig.vpcId" --output text)

$ echo $VPC_ID

$ aws ec2 create-security-group \
    --group-name malcolm-efs-sg \
    --description "Security group for Malcolm EFS" \
    --vpc-id $VPC_ID
…

$ EFS_SG_ID=$(aws ec2 describe-security-groups \
    --filters "Name=group-name,Values=malcolm-efs-sg" "Name=vpc-id,Values=$VPC_ID" \
    --query 'SecurityGroups[0].GroupId' --output text)

$ echo $EFS_SG_ID

$ for subnet in $PRIVATE_SUBNET_IDS; do \
    aws efs create-mount-target \
        --file-system-id $EFS_ID \
        --subnet-id $subnet \
        --security-groups $EFS_SG_ID; \
done
…

$ NODE_ROLE_NAME=$(aws iam list-roles \
                     --query "Roles[?contains(RoleName, 'eksctl-malcolm-cluster-nodegroup')].RoleName" \
                     --output text)

$ echo $NODE_ROLE_NAME

$ EFS_ARN="arn:aws:elasticfilesystem:us-east-1:$(aws sts get-caller-identity --query Account --output text):file-system/${EFS_ID}"

$ echo $EFS_ARN

$ aws iam put-role-policy \
  --role-name $NODE_ROLE_NAME \
  --policy-name AllowEFSAccess \
  --policy-document file://<(cat <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "elasticfilesystem:DescribeMountTargets",
        "elasticfilesystem:DescribeFileSystems",
        "elasticfilesystem:ClientMount",
        "elasticfilesystem:ClientWrite"
      ],
      "Resource": "$EFS_ARN"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeAvailabilityZones"
      ],
      "Resource": "*"
    }
  ]
}
EOF
)

$ helm repo add eks https://aws.github.io/eks-charts
…
$ helm repo update
…
$ helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=malcolm-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-alb-controller-sa \
  --set region=us-east-1 \
  --set vpcId=$VPC_ID
…

$ aws acm request-certificate \
  --domain-name malcolm.example.org \
  --validation-method DNS \
  --region us-east-1
…

$ CERT_ARN=$(aws acm list-certificates \
    --region us-east-1 \
    --query "CertificateSummaryList[?DomainName=='malcolm.example.org'].CertificateArn" \
    --output text)

$ echo $CERT_ARN

$ VALIDATION_RECORD=$(aws acm describe-certificate \
  --certificate-arn "$CERT_ARN" \
  --region us-east-1 \
  --query "Certificate.DomainValidationOptions[0].ResourceRecord" \
  --output json)

$ echo $VALIDATION_RECORD

$ aws acm describe-certificate \
  --certificate-arn "$CERT_ARN" \
  --region us-east-1 \
  --query "Certificate.Status"
…

$ HTTPS_HOSTNAME=$(kubectl get ingress malcolm-ingress-https -n malcolm -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
$ LOGSTASH_HOSTNAME=$(kubectl get service malcolm-nlb-logstash -n malcolm -o jsonpath='{.status.loadBalancer.ingress[*].hostname}')
$ FILEBEAT_HOSTNAME=$(kubectl get service malcolm-nlb-tcp-json -n malcolm -o jsonpath='{.status.loadBalancer.ingress[*].hostname}')

$ echo $HTTPS_HOSTNAME
$ echo $LOGSTASH_HOSTNAME
$ echo $FILEBEAT_HOSTNAME

$ ./Malcolm/scripts/stop -f "${KUBECONFIG:-$HOME/.kube/config}"
…

Generating a Malcolm Amazon Machine Image (AMI)

This section outlines the process of using packer’s Amazon AMI Builder to create an EBS-backed Malcolm AMI for either the x86-64 or arm64 CPU architecture. This section assumes good working knowledge of Amazon Web Services (AWS).

The files referenced in this section can be found in scripts/third-party-environments/aws/ami.

$ cp ./packer_vars.json.example ./packer_vars.json

$ packer validate packer_build.json
The configuration is valid.

$ AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY \
    AWS_SECRET_ACCESS_KEY=YOUR_AWS_SECRET_KEY \
    packer build -var-file=packer_vars.json packer_build.json

amazon-ebs: output will be in this color.

==> amazon-ebs: Prevalidating any provided VPC information
==> amazon-ebs: Prevalidating AMI Name: malcolm-v25.04.0-x86_64-2024-10-10T15-41-32Z
    amazon-ebs: Found Image ID: ami-xxxxxxxxxxxxxxxxx

...

==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Skipping Enable AMI deprecation...
==> amazon-ebs: Adding tags to AMI (ami-xxxxxxxxxxxxxxxxx)...
==> amazon-ebs: Tagging snapshot: snap-xxxxxxxxxxxxxxxxx
==> amazon-ebs: Creating AMI tags
    amazon-ebs: Adding tag: "Malcolm": "idaholab/Malcolm/v25.04.0"
    amazon-ebs: Adding tag: "source_ami_name": "al2023-ami-ecs-hvm-2023.0.20241003-kernel-6.1-x86_64"
==> amazon-ebs: Creating snapshot tags
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished after 19 minutes 57 seconds.

==> Wait completed after 19 minutes 57 seconds

==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-1: ami-xxxxxxxxxxxxxxxxx

$ aws ec2 describe-images \
    --owners self \
    --filters "Name=root-device-type,Values=ebs" \
    --filters "Name=name,Values=malcolm-*" \
    --query "Images[*].[Name,ImageId,CreationDate]" \
    --output text | sort

malcolm-v25.03.1-arm64-2025-03-31T18-28-00Z     ami-xxxxxxxxxxxxxxxxx   2025-03-31T18:33:12.000Z
malcolm-v25.03.1-x86_64-2025-03-31T18-13-34Z    ami-xxxxxxxxxxxxxxxxx   2025-03-31T18:19:17.000Z

Launching an EC2 instance from the Malcolm AMI

$ aws ec2 create-key-pair \
    --key-name malcolm-key \
    --query "KeyMaterial" \
    --output text > ./malcolm-key.pem

$ chmod 600 ./malcolm-key.pem

$ aws ec2 create-security-group \
    --group-name malcolm-sg \
    --description "Malcolm SG"
…

$ PUBLIC_IP=#.#.#.#

$ for PORT in 22 443; do \
    aws ec2 authorize-security-group-ingress \
        --group-name malcolm-sg \
        --protocol tcp \
        --port $PORT \
        --cidr $PUBLIC_IP/32; \
done
…

$ aws ec2 run-instances \
    --image-id AMI_ID \
    --instance-type INSTANCE_TYPE \
    --key-name malcolm-key \
    --security-group-ids malcolm-sg \
    --block-device-mappings "[{\"DeviceName\":\"/dev/sda1\",\"Ebs\":{\"VolumeSize\":100,\"VolumeType\":\"gp3\"}}]" \
    --count 1 \
    --tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=Malcolm}]"
…

$ aws ec2 describe-instances \
    --filters "Name=tag:Name,Values=Malcolm" \
    --query "Reservations[].Instances[].{ID:InstanceId,IP:PublicIpAddress,State:State.Name}" \
    --output table
…

$ INSTANCE_IP=$(aws ec2 describe-instances \
                  --filters "Name=tag:Name,Values=Malcolm" \
                  --query "Reservations[].Instances[].PublicIpAddress" \
                  --output text)

$ ssh -o IdentitiesOnly=yes -i ./malcolm-key.pem ec2-user@$INSTANCE_IP

Using MFA

Users with AWS MFA requirements may receive an UnauthorizedOperation error when performing the steps outlined above. If this is the case, the following workaround may allow the build to execute (thanks to this GitHub comment):

  1. Remove the access_key and secret_key lines from the builders section of packer_build.json (right below "type": "amazon-ebs")
  2. Run aws ec2 describe-instances --profile=xxxxxxxx (replacing xxxxxxxx with the credential profile name) to cause aws to authenticate (prompting for the MFA code) and cache the credentials
  3. At the bash command line, run: eval "$(aws configure export-credentials --profile xxxxxxxx --format env)" to load the current AWS credentials into environment variables in the current session
  4. Run the packer build command as described above

Attribution

Amazon Web Services, AWS, Fargate, the Powered by AWS logo, Amazon Elastic Kubernetes Service (EKS), and Amazon Machine Image (AMI) are trademarks of Amazon.com, Inc. or its affiliates. The information about providers and services contained in this document is for instructional purposes and does not constitute endorsement or recommendation.