Logo

A powerful, easily deployable network traffic analysis tool suite for network security monitoring

Quick Start

Documentation

Components

Supported Protocols

Configuring

Arkime

Dashboards

Hedgehog Linux

Contribution Guide

Deploying Malcolm on Amazon Elastic Kubernetes Service (EKS)

This document outlines the process of setting up a cluster on Amazon Elastic Kubernetes Service (EKS) using Amazon Web Services in preparation for Deploying Malcolm with Kubernetes.

This is a work-in-progress document that is still a bit rough around the edges. Users will need to replace things such as cluster-name and us-east-1 with the values that are appliable to the cluster. Any feedback is welcome in the relevant issue on GitHub.

This document assumes good working knowledge of Amazon Web Services (AWS) and Amazon Elastic Kubernetes Service (EKS). Good documentation resources can be found in the AWS documentation, the EKS documentation and the EKS Workshop.

Prerequisites

Procedure

  1. Create a Virtual Private Cloud (VPC)
    • subnets in at least 2 availability zones
    • tag private subnets with kubernetes.io/role/internal-elb: 1
    • tag public subnets with kubernetes.io/role/elb: 1
    • enable “auto-assign public IP address” for public subnets
  2. Create a security group for the VPC
  3. Create an Elastic Kubernetes Service (EKS) cluster
  4. Generate a kubeconfig file to use with Malcolm’s control scripts (malcolmeks.yaml is used in this example)
     aws eks update-kubeconfig --region us-east-1 --name cluster-name --kubeconfig malcolmeks.yaml
    
  5. Create a node group
    • For x86-64 instances c4.4xlarge, t2.2xlarge, and t3a.2xlarge seem to be good instance types for Malcolm; or , for arm64 instances, m6gd.2xlarge, m6g.2xlarge, m7g.2xlarge, and t4g.2xlarge; but users’ needs may vary (see recommended system requirements for Malcolm)
    • set the nodes to run on the VPC’s public subnets
  6. Deploy metrics-server
     kubectl --kubeconfig=malcolmeks.yaml apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
    
  7. Associate IAM OIDC provider with cluster
     eksctl utils associate-iam-oidc-provider --region=us-east-1 --cluster=cluster-name --approve
    
  8. Deploy the AWS Load Ballancer Controller add-on
  9. deploy Amazon EFS CSI driver
    • review Prerequisites
    • follow steps for Create an IAM policy and role
    • follow steps for Install the Amazon EFS driver
    • follow steps for Create an Amazon EFS file system
  10. Set up access points, and note the Access point IDs to put in the YAML in the next step

    name mountpoint access point ID
    config /malcolm/config fsap-…
    opensearch /malcolm/opensearch fsap-…
    opensearch-backup /malcolm/opensearch-backup fsap-…
    pcap /malcolm/pcap fsap-…
    runtime-logs /malcolm/runtime-logs fsap-…
    suricata-logs /malcolm/suricata-logs fsap-…
    zeek-logs /malcolm/zeek-logs fsap-…
  11. Create manifest for persistent volumes and volume claims from the EFS file system ID and access point IDs
  12. Finish the configuration then start Malcolm as described in Deploying Malcolm with Kubernetes

Attribution

Amazon Web Services, AWS, the Powered by AWS logo, and Amazon Elastic Kubernetes Service (EKS) are trademarks of Amazon.com, Inc. or its affiliates. The information about providers and services contained in this document is for instructional purposes and does not constitute endorsement or recommendation.