Is there a way to create EMR security config with CloudFormation/Terraform

Question

I want to archive the logic similar to the CLI command: aws emr create-security-configuration --name [name] --security-configuration ... and use it further in the Terraform script.

Answer 1

You can follow the basic examples here https://www.terraform.io/docs/providers/aws/r/security_group.html and here https://www.terraform.io/docs/providers/aws/r/emr_cluster.html.

It would be similar to:

resource "aws_security_group" "sg" {
  name = "allow_all"
  description = "Allow all inbound traffic"

  ingress {
      from_port = 0
      to_port = 0
      protocol = "-1"
      cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
      from_port = 0
      to_port = 0
      protocol = "-1"
      cidr_blocks = ["0.0.0.0/0"]
      prefix_list_ids = ["pl-12c4e678"]
  }
}

resource "aws_emr_cluster" "emr-test-cluster" {
  name          = "emr-test-arn"
  release_label = "emr-4.6.0"
  applications  = ["Spark"]

  termination_protection = false
  keep_job_flow_alive_when_no_steps = true

  ec2_attributes {
    subnet_id                         = "${aws_subnet.main.id}"
    emr_managed_master_security_group = "${aws_security_group.sg.id}"
    emr_managed_slave_security_group  = "${aws_security_group.sg.id}"
    instance_profile                  = "${aws_iam_instance_profile.emr_profile.arn}"
  }
...
}

Answer 2

Update 06/07/2017: As of Jun 6 2017, the AWS::EMR::SecurityConfiguration resource is now available in CloudFormation, and as of May 11 2017 (v0.9.5) the emr_security_configuration resource is available in Terraform.

---

Unfortunately, it doesn't look like it is currently possible to specify a SecurityConfiguration for the RunJobFlow API using either CloudFormation's AWS::EMR::Cluster CloudFormation Resource or Terraform's aws_emr_cluster resource, and there are no resources that correspond to the CreateSecurityConfiguration API.

The EMR Security Configuration feature was added on Sep 21 2016, and there is typically a lag between new feature announcements and their corresponding support in existing CloudFormation resources.

Although Terraform tends to be updated more quickly as it is an open-source project with a larger development community, the aws_emr_cluster resource is still relatively new (released Oct 6 2016). I've opened a GitHub issue tracking a feature request for this implementation.

As a workaround for now, you could create a Custom Resource that calls the CreateSecurityConfiguration and RunJobFlow APIs directly.