Pkl 0.27 Release Notes

Pkl 0.27 was released on November 5th, 2024.
The latest bugfix release is 0.27.1. (All Versions)

This release brings improvements in typechecking of Listing and Mapping, the ability to use readers from external processes, as well as a new import graph analyzer API.

The next release (0.28) is scheduled for February 2025.

Please send feedback and questions to GitHub Discussions, or submit an issue on Github.

Pkl is hosted on GitHub. To get started, follow Installation.

Highlights 💖

News you don’t want to miss.

Improved typechecking of Listing and Mapping types

Typechecking of Listing and Mapping types has been improved (#628, #725, #740, #752, #778, #781).

Today, the typecheck listing: Listing<E> immediately evaluates all listing elements to check that they have type E. Likewise, the typecheck mapping: Mapping<K, V> immediately evaluates all mapping values to check that they have type V.

For example, the typecheck listing: Listing<Bird> proceeds as follows:

  1. Check that listing has type Listing

  2. Evaluate each listing element and check that it has type Bird

This behavior is different from how the rest of Pkl works. Generally, Pkl only evaluates code that affects program output.

For example, consider the following program:

class Bird {
  name: String
  canFly: Boolean
}

local bird: Bird = new {
  name = "Pidgy"
  canFly = throw("uh oh")
}

birdName = bird.name

Even though bird.canFly throws an error, the above program succeeds because bird.canFly is not part of the program’s output and hence is never accessed (Note that bird is a local property).

Typechecks of Mapping and Listing types have been changed to match this behavior. Mapping and listing values are now only typechecked if and when they are accessed.

Mapping keys are still eagerly checked.

This change causes some previously failing programs to evaluate successfully:

local myNumbers: Listing<Int> = new { 1; 2; "uh oh" }
result = myNumbers[0]

In Pkl 0.26 and below, the above program fails with a type mismatch error because element "uh oh" is typechecked when myNumbers is accessed. In Pkl 0.27, the same program succeeds, because only element myNumbers[0] is part of the program’s output and its typecheck succeeds.

As another consequence of this change, some Pkl programs now complete more quickly:

local allBirds: Mapping<String, Bird> = import*("**/bird.pkl")

environment: "prod"|"qa"

cluster: String

myBird = allBirds["\(environment)/\(cluster)/bird.pkl"] (1)

In Pkl 0.26 and below, all modules matching **/bird.pkl are imported when allBirds is accessed and typechecked. This can take a long time. In Pkl 0.27, only module "\(environment)/\(cluster)/bird.pkl" is imported because only this module is part of the program’s output.

To learn more about this change, consult SPICE-0010.

External readers

A new feature has been added to allow Pkl to spawn external processes to read resources and modules (#660, #762, #766, #770).

Today, users who use Pkl as a library can define custom module and resource readers. This allows authors to extend how Pkl performs I/O.

For example, users can implement a reader that reads the secret: scheme, where they define exactly how the bytes are fetched in the host runtime.

result = read("secret:mypassword") (1)
1 secret: is a custom scheme defined by the host runtime.

However, CLI users have been left out.

In Pkl 0.27, a new extension point is added to allow users to implement module and resource readers as external processes. When run, Pkl will spawn the external process, and talk to the process via message passing.

To learn more about this feature, consult SPICE-0009.

Thanks to @HT154 for contributing the feature!

Import analysis API

A new API has been added to analyze the import graph of Pkl modules (#695).

This API comes in four forms:

  1. A standard library module: pkl:analyze

  2. A CLI command: pkl anaylze imports

  3. A Java API: org.pkl.core.Analyzer

  4. A Gradle API: org.pkl.gradle.task.AnalyzeImportsTask

Some use-cases for this API are:

  • For build tools to perform out-of-date checks. Build tools can invalidate a cached result if any of the transitive modules have changed.

  • Static code analysis, to determine how Pkl modules depend on each other.

With this API, Gradle Plugin now by default computes the transitive modules for many of its tasks.

Here is an example of the CLI in use:

pkl analyze imports -f json myModule.pkl (1)
1 -f means: produce output in JSON.

Produces:

{
  "imports": {
    "file:///my/proj/myModule.pkl": [
      {
        "uri": "projectpackage://example.com/birds@1.0.0#/Bird.pkl"
      }
    ],
    "projectpackage://example.com/birds@1.0.0#/Bird.pkl": []
  },
  "resolvedImports": {
    "file:///my/proj/myModule.pkl": "file:///my/proj/myModule.pkl",
    "projectpackage://example.com/birds@1.0.0#/Bird.pkl": "file:///my/birds/Bird.pkl"
  }
}

From this output, we can see that myModule.pkl imports Bird.pkl from the birds package. We can also see that the module projectpackage://example.com/birds@1.0.0#/Bird.pkl resolves to disk location file:///my/birds/Bird.pkl (we can deduce that birds is a local dependency).

To learn more about this feature, consult SPICE-0001.

Noteworthy 🎶

Ready when you need them.

Colored output

The Pkl CLI will now emit some messages in color (#552, #746, #771, #779).

Here is a sneak peek of colored error messages in action.

syntax highlighted output

Thanks to @thomaspurchas for contributing to this feature!

const local object members

The const modifier can be applied to object members, provided that they are also local (#678).

Currently, regular object members are not allowed to have the const modifier. This introduces an artificial pain point.

For example, given the following module:

amends "Bird.pkl"

import "Bird.pkl"

local northAmerica = "North America"

local typealias NorthAmericanBird = Bird(this.origin == northAmerica) (1)
1 Error: cannot reference northAmerica from here because it is not const.

This is invalid code, because this typealias is referencing a non-const value on the enclosing module. However, is not possible fix this by adding the const modifier to northAmerica. This is because this module amends "Bird.pkl", which means that this module is considered a regular object and not a class. This means that any members declared here are object members, and not class members.

In Pkl 0.27, a new rule is introduced to allow the const modifier to be applied to object members, provided that they are also local.

This change affects object properties, as well as object methods.

To read more about this design, consult SPICE-0011.

Pkl CLI changes

New CLI Flags

Some new common flags have been added to the CLI (#660, #746).

Flag Description

--color

Format messages with ANSI color codes

--external-module-reader

Shell out to a process to read certain modules.

--external-resource-reader

Shell out to a process to read certain resources.

New command: pkl analyze imports

As part of the set of APIs added for import analysis, a new subcommand has been added called analyze imports (#695).

For details, consult the CLI documentation.

Kotlin/Java code generator improvements

Various improvements have been made to the Kotlin and Java code generators (#705, #710, #714, #721, #729).

Java codegen improvements

  • Only generate hashCode(), equals(), and toString() methods for Java classes that are instantiable.

  • Add support for Spring Boot 3.

  • Make module classes also implement serializable.

  • Make empty Java classes instantiable.

Kotlin codegen improvements

  • Skip generation of copy(), equals(), hashCode(), toString() methods for abstract Kotlin classes.

  • Don’t implement Serializable for abstract classes

  • Add support for Spring Boot 3.

Thanks to @translatenix for contributing these improvements!

Gradle Plugin changes

New AnalyzeImportsTask

A new task called org.pkl.gradle.task.AnalyzeImportsTask is introduced (#695).

This task is the Gradle analogy to pkl analyze imports.

Example:

  • build.gradle

  • build.gradle.kts

pkl {
  analyzers {
    imports {
      appConfig {
        sourceModules.add(file("src/main/resources/appConfig.pkl"))
      }
    }
  }
}
pkl {
  analyzers {
    imports {
      register("appConfig") {
        sourceModules.add(file("src/main/resources/appConfig.pkl"))
      }
    }
  }
}

For more details, consult the documentation.

Tracked file outputs

Tasks created by the Pkl plugin now declare tracked output files (#403).

This means that downstream tasks do not need to declare an explicit dependency on the Pkl task.

For example, assuming that evalPkl is an EvalTask:

build.gradle.kts
 val myGradleTask by tasks.registering {
   inputs.files(evalPkl)
-  dependsOn(evalPkl) (1)
 }
1 No longer necessary to declare this dependency.

transitiveModules computed by default

The transitiveModules property of a task is now computed by building the import graph of the source modules (#695).

This means that Pkl-related tasks no longer need to declare their set of transitive modules, because the pkl-gradle plugin will compute this automatically.

This adds latency to each task. To opt out of this behavior, set transitiveModules explicitly.

Standard library changes

Additions to pkl:base

New properties and methods have been added to the classes of pkl:base (#666, #683).

New properties and methods have been added to Listing and Mapping.

One of the goals of this change is to improve the experience of authoring constraints. This eliminates the need to convert to collection types as often.

The added properties and methods are:

With this, the following change can be made to existing constraints:

-ipAddresses: Listing<String>(toList().contains("127.0.0.1"))
+ipAddresses: Listing<String>(contains("127.0.0.1"))

Additionally, a new method is added to String, called String.splitLimit().

Additions to pkl:EvaluatorSettings

New properties have been added to pkl:EvaluatorSettings (#660, #746).

These are:

String to Number conversion improvements.

The String to Number converter methods, for example, String.toInt(), can now handle underscore separators (#578, #580).

This better aligns with the source code representation of number literals.

myNum = "1_000".toInt() (1)
1 Result: 1000

New module: pkl:analyze

As part of Import analysis API, a new standard library module is added called pkl:analyze (#695).

This module provides an API for computing the total import graph given a set of input modules. This API treats the inputs as entrypoints, and produces a graph representing the entire dependency tree, including transitive imports.

Example:

import "pkl:analyze"

importGraph: analyze.ImportGraph = analyze.importGraph(Set("file:///path/to/my/module.pkl"))

For details, see the documentation.

Annotations on PklProject added to packages

When creating packages, some annotations on a PklProject will affect how pkldoc generates documentation.

  • @Unlisted: Omit the package from publishing documentation.

  • @Deprecated: Add information on the package to show that it is deprecated.

Example:

PklProject
@Deprecated
amends "pkl:Project"

Test report improvements

The report generated from pkl test has been overhauled and improved (#498, #682, #738, #771).

  • Tests are grouped into either the facts or examples section.

  • A summary line is added, describing how many tests have passed and failed, and how many assertions have passed and failed.

  • Test results are colored.

  • The ✅ and ❌ emojis are replaced with ✔ and ✘.

  • Thrown errors are reported within the test.

Here is a sneak peek of the new test result output.

test sample

Thanks to @jjmaestro for contributing to this improvement!

Java API additions

New API: org.pkl.core.Analyzer

As part of Import analysis API, a new Java API called org.pkl.core.Analyzer is introduced (#695).

New methods

The following methods are added:

pkl-executor

  • org.pkl.executor.ExecutorException.getPklVersion

  • org.pkl.executor.ExecutorException.ExecutorException(java.lang.String, java.lang.Throwable, java.lang.String)

pkl-core

  • org.pkl.core.module.ModuleKeyFactories.externalProcess(java.lang.String, org.pkl.core.externalreader.ExternalReaderProcess)

  • org.pkl.core.module.ModuleKeyFactories.externalProcess(java.lang.String, org.pkl.core.externalreader.ExternalReaderProcess, long)

Breaking Changes 💔

Things to watch out for when upgrading.

Java API breaking changes

The following Java APIs have breaking changes:

Class/method Breaking change

org.pkl.core.runtime.TestResults

Moved to org.pkl.core.TestResults, turned into a record, and largely restructured.

org.pkl.core.module.ModuleKeyFactories.closeQuietly

Deprecated for removal in favor of org.pkl.core.Closeables.closeQuietly.

Spring Boot 2 support dropped

The Java code generator no longer supports Spring Boot 2.x (#729).

Spring Boot 2 users can continue to use the Pkl 0.26 code generator.

pkl test considered failing when writing examples

If running pkl test results in pkl-expected.pcf files being written, the test now exits with exit code 10 (#738).

Additionally, the junit reports will consider the test as failing.

Listing and Mapping typecheck changes

Due to changes to Listing and Mapping typechecking, some previously failing programs now succeed. This can happen when erroneous code is never evaluated because it does not affect program output:

local myNumbers: Listing<Int> = new { 1; 2; "uh oh" }
result = myNumbers[0] (1)
1 In Pkl 0.26 and below, throws a type mismatch error.

In Pkl 0.27, yields 1.

Miscellaneous 🐸

  • Make pkl-doc top box include a preview of the rest of module-level documentation (#570).

  • Module pkl-core now has a dependency on org.msgpack:msgpack-core (#660). Users concerned about potential version conflicts can use one of Pkl’s shaded fat JARs (pkl-config-java-all, pkl-tools).

  • Make PklProject.deps.json files end in a newline (#644).

  • Fix invalid syntax of code examples in stdlib doc comments (#703).

  • Update Java dependencies (#689, #767).

  • Add jbang catalog support (#655).

  • Documentation improvements (#623, #680, #682, #685, #687, #703, #704, #715, #730, #753).

Bug Fixes 🐜

The following bugs have been fixed.

  • CLI --property flags containing = in property values are not parsed correctly (#630).

  • Thrown PklBugException when reading assets from a local project dependency (#641).

  • Thrown PklBugException when PklProjects have a cyclical local dependency (#731).

  • Thrown exception when sending ReadResourceResponse or ReadModuleResponse where both contents and error are null (#657).

  • Double unary minus is evaluated as single unary minus (#697).

  • Kotlin compiler fails to compiler generated classes with "copy overrides nothing" for subclasses of abstract classes (#569).

  • "Unexpected error" thrown when Module.output has its type overridden (#709).

  • "Unexpected error" thrown when loading files with non-ASCII characters (#653).

  • Type parameters in new Mapping<TypeA, TypeB>/new Listing <Type> are not checked (#405).

  • Comparison methods of pkl:semver are incorrect (#772).

Contributors 🙏

We would like to thank the contributors to this release (in alphabetical order):