Filling out a Template

In this second part of the Pkl tutorial, you will learn how to write one (part of a) configuration in terms of another. You will also find and fill out an existing template.

Composing configurations

Amending

The central mechanism in Pkl for expressing one (part of a) configuration in terms of another is amending. Consider the following example.

amendingObjects.pkl

bird {
  name = "Pigeon"
  diet = "Seeds"
  taxonomy {
    kingdom = "Animalia"
    clade = "Dinosauria"
    order = "Columbiformes"
  }
}

parrot = (bird) {
  name = "Parrot"
  diet = "Berries"
  taxonomy {
    order = "Psittaciformes"
  }
}

Parrot and Pigeon have nearly identical properties. They only differ in their name and taxonomy, so if you have already written out bird, you can say that parrot is just like bird except name is "Parrot", diet is "Berries" the taxonomy.order is "Psittaciformes". When you run this, Pkl expands everything fully.

bird {
  name = "Pigeon"
  diet = "Seeds"
  taxonomy {
    kingdom = "Animalia"
    clade = "Dinosauria"
    order = "Columbiformes"
  }
}
parrot {
  name = "Parrot"
  diet = "Berries"
  taxonomy {
    kingdom = "Animalia"
    clade = "Dinosauria"
    order = "Psittaciformes"
  }
}

So far, you have only worked with Dynamic objects. Pkl also offers Typed objects. Amending allows you to override, amend, and add new properties to a dynamic object. Typed objects will only let you amend or override existing properties, not add entirely new ones. The next part of the tutorial discusses types in more detail. There, you see that amending never changes the type of the object.

You can also amend nested objects. This allows you to only describe the difference with the outermost object for arbitrarily deeply nested structures. Consider the following example.

nestedAmends.pkl

stockPigeon {
  name = "Stock pigeon"
  diet = "Seeds"
  taxonomy {
    kingdom = "Animalia"
    clade = "Columbimorphae"
    order = "Columbiformes"
    species = "Columba oenas"
  }
}

woodPigeon = (stockPigeon) {
  name = "Common wood pigeon"
  taxonomy { (1)
    species = "Columba palumbus"
  }
}

dodo = (woodPigeon) { (2)
  name = "Dodo"
  extinct = true (3)
  taxonomy {
    species = "Raphus cucullatus"
  }
}

1	This amends `species`, as it occurs in `stockPigeon`.
2	Amended objects can, themselves, be amended.
3	New fields can be added to objects when amending.

Notice how you only have to change taxonomy.species. In this example, stockPigeon.taxonomy has kingdom, clade, order and species. You are amending stockPigeon, to define woodPigeon. They have the same taxonomy, except for species. This notation says that everything in taxonomy should be what it is in the object you are amending (stockPigeon), except for species, which should be "Columba palumbus" .

For the input above, Pkl produces the following output.

stockPigeon {
  name = "Stock pigeon"
  diet = "Seeds"
  taxonomy {
    kingdom = "Animalia"
    clade = "Columbimorphae"
    order = "Columbiformes"
    species = "Columba oenas"
  }
}
woodPigeon {
  name = "Common wood pigeon"
  diet = "Seeds"
  taxonomy {
    kingdom = "Animalia"
    clade = "Columbimorphae"
    order = "Columbiformes"
    species = "Columba palumbus"
  }
}
dodo {
  name = "Dodo"
  diet = "Seeds"
  taxonomy {
    kingdom = "Animalia"
    clade = "Columbimorphae"
    order = "Columbiformes"
    species = "Raphus cucullatus"
  }
  extinct = true
}

So far, you have only amended properties. Since you refer to them by name, it makes sense that you "overwrite" the value from the object you’re amending. What if you include elements or entries in an amends expression?

amendElementsAndEntries.pkl

favoriteFoods {
  "red berries"
  "blue berries"
  ["Barn owl"] {
    "mice"
  }
}

adultBirdFoods = (favoriteFoods) {
  [1] = "pebbles" (1)
  "worms" (2)
  ["Falcon"] { (3)
    "insects"
    "amphibians"
  }
  ["Barn owl"] { (4)
    "fish"
  }
}

1	Explicitly amending by index replaces the element at that index.
2	Without explicit indices, Pkl can’t know which element to overwrite, so, instead, it adds an element to the object you’re amending.
3	When you write "new" entries (using a key that does not occur in the object you’re amending), Pkl also adds them.
4	When you write an entry using a key that exists, this notation amends its value.

Pkl can’t know which of the favoriteFoods to overwrite only by their value. When you want to replace an element, you have to explicitly amend the element at a specific index. This is why a "plain" element in an amends expression is added to the object being amended. Result:

favoriteFoods {
  ["Barn owl"] {
    "mice"
  }
  "red berries"
  "blue berries"
}
adultBirdFoods {
  ["Barn owl"] {
    "mice"
    "fish"
  }
  "red berries"
  "pebbles"
  ["Falcon"] {
    "insects"
    "amphibians"
  }
  "worms"
}

Modules

A .pkl file describes a module. Modules are objects that can be referred to from other modules. Going back to the example above, you can write parrot as a separate module.

pigeon.pkl

name = "Common wood pigeon"
diet = "Seeds"
taxonomy {
  kingdom = "Animalia"
  clade = "Dinosauria"
  species = "Columba palumbus"
}

You can import this module and express parrot like you did before.

parrot.pkl

import "pigeon.pkl" (1)

parrot = (pigeon) {
  name = "Great green macaw"
  diet = "Berries"
  taxonomy {
    species = "Ara ambiguus"
  }
}

1	Importing `pigeon.pkl` creates the object `pigeon`, so you can refer to `pigeon` in this code, like you did before.

If you run Pkl on both, you will see that it works. Looking at the result, however, you see a (possibly) unexpected difference.

$ pkl eval /Users/me/tutorial/pigeon.pkl
name = "Common wood pigeon"
diet = "Seeds"
taxonomy {
  kingdom = "Animalia"
  clade = "Dinosauria"
  species = "Columba palumbus"
}

$ pkl eval /Users/me/tutorial/parrot.pkl
parrot {
  name = "Great green macaw"
  diet = "Berries"
  taxonomy {
    kingdom = "Animalia"
    clade = "Dinosauria"
    species = "Ara ambiguus"
  }
}

The object pigeon is "spread" in the top-level, while parrot is a nested and named object. This is because writing parrot {…} defines an object property in the "current" module.

In order to say that "this module is an object, amended from the pigeon module," you use an amends clause.

parrot.pkl

amends "pigeon.pkl" (1)

name = "Great green macaw"

1	"This" module is the same as `"pigeon.pkl"`, except for what is in the remainder of the file.

As a first intuition, think of "amending a module" as "filling out a form."

Amending templates

A Pkl file can be either a template or a "normal" module. This terminology describes the intended use of the module and doesn’t imply anything about its structure. In other words: just by looking at Pkl code, you can’t tell whether it is a template or a "normal" module.

AcmeCICD.pkl

module AcmeCICD

class Pipeline {
  name: String(nameRequiresBranchName)?

  hidden nameRequiresBranchName = (_) ->
      if (branchName == null)
        throw("Pipelines that set a 'name' must also set a 'branchName'.")
      else true

  branchName: String?
}

timeout: Int(this >= 3)

pipelines: Listing<Pipeline>

output {
  renderer = new YamlRenderer {}
}

Remember that amending is like filling out a form. That’s exactly what you’re doing here; you’re filling out "work order forms".

Next, add a time-out of one minute for your job.

cicd.pkl

amends "AcmeCICD.pkl"

timeout = 1

Unfortunately, Pkl does not accept this configuration and provides a rather elaborate error message:

–– Pkl Error –– (1)
Type constraint `this >= 3` violated. (2)
Value: 1 (3)

225 | timeout: Int(this >= 3)? (4)
                   ^^^^^^^^^
at AcmeCICD#timeout (file:///Users/me/tutorial/AcmeCICD.pkl, line 8)

3 | timeout = 1 (5)
              ^
at cicd#timeout (file:///Users/me/tutorial/cicd.pkl, line 3)

90 | text = renderer.renderDocument(value) (6)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (https://github.com/apple/pkl/blob/e4d8c882d/stdlib/base.pkl#L90)

1	Pkl found an error.
2	Which error Pkl found.
3	What the offending value is.
4	Where Pkl found its expectation (line 8 of the amended module).
5	Where Pkl found the offending value (line 3 of the input module).
6	What Pkl evaluated to discover the error.

When Pkl prints source locations, it also prints clickable links for easy access. For local files, it generates a link for your development environment (configurable in ~/.pkl/settings.pkl). For packages imported from elsewhere, if available, Pkl produces https:// links to their repository.

Pkl complains about a type constraint. Pkl’s type system doesn’t just protect you from providing a String where you expected an Int, it even checks which values are allowed. In this case, the minimum time-out is three minutes. If you change the value to 3, Pkl accepts your configuration.

$ pkl eval cicd.pkl
timeout: 3
pipelines: []

You can now define a pipeline. Start off by specifying the name of the pipeline and nothing else.

cicd.pkl

amends "AcmeCICD.pkl"

timeout = 3
pipelines {
  new { (1)
    name = "prb"
  }
}

1	There is no pipeline object to amend. The `new` keyword gives you an object to amend.

So far, you’ve defined objects the same way you amended them. If the name foo didn’t occur before, foo { … } creates a property called foo and assigns to it the contents in place of the … (given that … is not valid Pkl, but an abbreviation for illustration purposes). If foo is an existing object, this notation is an amend expression; resulting in a new object (value), but not a new (named) property. Since pipelines is a listing, you can add elements by writing expressions in an amend expression.

In this case, though, there is no object to amend. Writing myNewPipeline { … } defines a property, but listings may only include elements. This is where you can use the keyword new.

new gives you an object to amend. Pkl derives from the context in which new is used and what the object to amend should look like. This is called the default value for the context. The next part goes into detail about how Pkl does this.

Running Pkl on your new configuration produces a verbose error.

cicd.pkl

–– Pkl Error ––
Pipelines that set a 'name' must also set a 'branchName'.

8 | throw("Pipelines that set a 'name' must also set a 'branchName'.")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at AcmeCICD#Pipeline.nameRequiresBranchName.<function#1> (file:///Users/me/tutorial/AcmeCICD.pkl, line 8)

6 | name = "prb"
           ^^^^^
at cicd#pipelines[#1].name (file:///Users/me/tutorial/cicd.pkl, line 6)

90 | text = renderer.renderDocument(value)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (https://github.com/apple/pkl/blob/e4d8c882d/stdlib/base.pkl#L90)

You have hit another type constraint, like timeout: Int(this >= 3) before. In this case, the error message consists of an English language sentence, instead of Pkl code. When constraints are complicated or very application specific, template authors can throw a more descriptive error message like this.

The message is quite instructive, so you can fix the error by adding a branchName.

cicd.pkl

amends "AcmeCICD.pkl"

timeout = 3
pipelines {
  new {
    name = "prb"
    branchName = "main"
  }
}

and indeed

$ pkl eval -f yml /Users/me/tutorial/cicd.pkl
timeout: 3
pipelines:
- name: prb
  branchName: main