Enforce policies on provisioning of cloud resources using OPA and Terraform
Introduction
Previously I have written an article about how to provision CosmosDb in Azure using Terraform, link below ->
Today, we will look into “how to enforce guidelines as policies” on these on demand provisioning of cloud resources. One such tool which can help us in enforcing what project teams can provision is by using Open Policy Agent (OPA). It provides a declarative language (Rego) to input our policy as code, which in turn helps us evaluate our policies automatically.
For example, company policy states → Servers reachable from the Internet must not expose the insecure 'http' protocol.
The rego sample for this can be as follows,
violation[server.id] { # server is in violation if...some server
public_server[server] # if its public server andserver.protocols[_] == "http" # contains "http" protocol.}
Complete example with input and its Rego policy is at → https://play.openpolicyagent.org/p/oG4ujcYPAb Run evaluate to get output.
In lay man terms, basically we are enforcing business policies; by writing them in plain English (almost), automatically with out manual intervention by our applications, which can be CI/CD pipeline tools, IaC/CM tools .,
OPA & Rego
OPA (Open Policy Agent) is an open source policy engine, that lets us set our policy as code, using their declarative language (Rego), and enforces it across the entire stack.
We need to provide a structured data as input, specifically in JSON format and OPA evaluates it against the set policy and returns evaluated result.
Installing OPA (pronounced Oh-PA):
On Mac OS:
Download latest OPA binary by using the below command from terminal.
curl -L -o opa https://openpolicyagent.org/downloads/latest/opa_darwin_amd64then run the following commands to make sure its running under local
chmod +x opa
mv ./opa /usr/local/bin/
opa versionResult should be something similar to below
Version: 0.22.0
Build Commit: 3d5d330
Build Timestamp: 2020–07–16T14:53:51Z
Build Hostname: 0165ad1ca858REGO (pronounced ray-go) :
All policies are written using REGO language. Lets go through some important points in REGO, more information can be found at → https://www.openpolicyagent.org/docs/latest/policy-language/
Scalar values:
These are like constants or variables, and these can be strings, numbers, booleans or null. There are composite values also, which are lists, collections. Use assignment “:=” operator.
For example,
cube := {“width”: 3, “height”: 4, “depth”: 5}You can query as cube.width, which will return ‘3’.
Reference:
The input JSON data can be referenced as a global variable in OPA as “input”, then we can traverse through the JSON using “.” operator, for example, for input JSON as follows, you can reference “https” as
input.servers[0].protocols[0]
[
{
“id”: “app”,
“ports”: [
“p1”,
“p2”,
“p3”
],
“protocols”: [
“https”,
“ssh”
]
}
]If when referencing, the path you have mentioned doesn’t exist, then OPA returns “undefined decision” as its output.
Expressions:
We can write expressions against input data and comparing data, like input.servers[0].id == “app”, which is true. Rego provides some of the inbuilt functions for this, like aggregations (count), arithmetic, regular expressions etc., a complete list of those can be found here → https://www.openpolicyagent.org/docs/latest/policy-reference
We can use ‘;’ operator for “AND” operation and in a single line. Or we can use multiple lines for the same result.
input.servers[0].id == “app” ; input.servers[0].protocols[0] == “https”ORinput.servers[0].id == “app”
input.servers[0].protocols[0] == “https”
Both yields same results, that is true.
Variables:
Variables can be referenced as input JSON.
Variables are immutable, you cannot assign a value to a variable twice.
s := input.servers[0]
s := input.servers[1]Fails with error
Iteration:
Unlike other languages, Rego doesn’t have any specific keyword to indicate loop, we can use something like “some” or “_” for indicating iterations, for example,
input.servers[_].protocols[_] == “http”This means, is there any server with protocol “http” in the input.
some i, j; input.servers[i].protocols[j] == “http”Similar result as above
Rules:
Please go through this link here for understanding rules as they form basic for creating policies in Rego for OPA. → https://www.openpolicyagent.org/docs/latest/#rules
Since, we have gone through some of the basics, please test out some simple polices using the playground below, you can change input and the policy (left hand side).
Playground:
To test your rego policy against any input JSON, please follow the playground link that is available online for practice → https://play.openpolicyagent.org/p/BTncJZFgDw
Implementation
Now you have looked at how to write a basic policy, let’s move a step further from previous provisioning blog, into securely provisioning CosmosDb in Azure using Terraform by enforcing policies. We will implement a basic policy and block any provisioning request that isn’t following the policy.
Let’s first set up policy for our implementation, simples rules are enough. In our CosmosDb case for example, we can restrict our account to be created only in a particular region. If user doesn’t use that region, we can fail evaluation and stop provisioning.
From below, we will continue from where we left off on the first blog (posted top of this blog). There we have provisioned a very basic CosmosDb account. Now, if you have gone through that blog, we have started with “terraform init” command. After that, run the following,
terraform plan --out cosmos.binary
terraform show -json cosmos.binary > cosmos.jsonNow, we have a JSON as input, let’s create a policy document show as below. Save this file as sample.rego. Which basically checks if the resource that was created is in “central US” region or not.
package cosmosdb
default isValid = false
deny[msg] {
where := resource_changes[_]
not startswith(where, "centralus")
msg := sprintf("Location must be `centralus`; found `%v`", [where])
}
resource_changes[c] {
c := input.resource_changes[_].change.after.geo_location[_].location
}isValid {
count(deny) == 0
}
Now, on your Mac or any other OS, please run the following command.
opa eval --format pretty --data sample.rego --input cosmos.json data.cosmosdbYou should see the following output, and you can look for the particular variable that we created like isValid or deny, shown below.
{
"deny": [
"Location must be `centralus`; found `eastus`",
"Location must be `centralus`; found `eastus2`"
],
"resource_changes": [
"eastus",
"eastus2"
],
"isValid": false
}For variables run the following commands,
$ opa eval --format pretty --data sample.rego --input cosmos.json data.cosmosdb.deny
[
"Location must be `eastus1`; found `eastus`",
"Location must be `eastus1`; found `eastus2`"
]$ opa eval --format pretty --data sample.rego --input cosmos.json data.cosmosdb.isValid
false
If you notice that the “isValid” returns the value false and based on that we can stop or proceed with the resource provisioning in the cloud.
Conclusion:
OPA helps us enforce policies, simple or complex, using simple true/false statements written in Rego. From the implementation portion, its evident that we can enforce policies using our apps, by calling “opa” and providing it a JSON input with rego policy and it will evaluate whether, the input JSON is valid or compliant or following-policy or not.
Hope this helps your research or your interest on how to automate policy enforcement with out manual intervention.
OPA works well with Kubernetes also, which makes it a very very powerful tool. I am looking forward to research into that area. K8S already provides admission controls, controllers etc., but this would act as extra layer protection that can be used to enforce our own policies.
