Skip to content

feat: implement export filter functionality for resource exports - #1621

Merged
Steve Lee (SteveL-MSFT) merged 34 commits into
PowerShell:mainfrom
Gijsreyn:gh-1486/main/add-postfilter-export
Aug 7, 2026
Merged

feat: implement export filter functionality for resource exports#1621
Steve Lee (SteveL-MSFT) merged 34 commits into
PowerShell:mainfrom
Gijsreyn:gh-1486/main/add-postfilter-export

Conversation

@Gijsreyn

Copy link
Copy Markdown
Collaborator

PR Summary

Adds engine-side post-filtering for export via a new exportFilter resource directive.

PR Context

Fixes #1486.

Copilot AI review requested due to automatic review settings July 11, 2026 13:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements engine-side post-filtering for dsc config export results via a new exportFilter resource directive, enabling consistent wildcard/equality filtering even for resources that don’t implement native export filtering.

Changes:

  • Adds exportFilter to the configuration resource directives schema and applies it in the export pipeline after resource.export(...).
  • Introduces configure::export_filter matching logic (case-insensitive * wildcard for strings; OR across filter objects, AND within each object) plus unit tests.
  • Adds Pester coverage for exportFilter behavior and updates the dsctest export resource to emit a name property for filtering.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tools/dsctest/src/main.rs Emits per-instance name values to enable filter-driven tests.
tools/dsctest/src/export.rs Extends the test export payload schema with optional name.
lib/dsc-lib/src/dscresources/command_resource.rs Adjusts when “export filtering not supported” is raised (only for non-empty input).
lib/dsc-lib/src/configure/mod.rs Wires exportFilter from directives into the export pipeline.
lib/dsc-lib/src/configure/export_filter.rs Adds the engine-side filtering implementation + Rust unit tests.
lib/dsc-lib/src/configure/config_doc.rs Adds export_filter directive field to the config document model/schema.
lib/dsc-lib/locales/en-us.toml Adds localized message for filter reduction logging.
dsc/tests/dsc_export.tests.ps1 Adds integration tests validating exportFilter behavior across scenarios.
dsc/src/resource_command.rs Updates call site for the changed export helper function signature.

Comment thread lib/dsc-lib/src/configure/mod.rs Outdated
Comment thread lib/dsc-lib/src/configure/export_filter.rs Outdated
Comment thread lib/dsc-lib/src/configure/export_filter.rs Outdated

@SteveL-MSFT Steve Lee (SteveL-MSFT) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look fine, but can you remove the wildcard filtering support in the service, firewall, and dism resources? I would expect all the tests to still pass (assuming they don't call the resource exe directly, if so, they would need to be udpated to go through dsc).

@Gijsreyn

Copy link
Copy Markdown
Collaborator Author

Steve Lee (@SteveL-MSFT) - good catch, but it had some caveats. I've replaced them with tests that append * to a known valid identifier. Exact-match and existing AND/OR filtering coverage remained in place.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 3 comments.

Comment thread resources/windows_service/src/service.rs Outdated
Comment thread lib/dsc-lib/src/configure/mod.rs Outdated
Comment thread resources/dism_dsc/src/feature_on_demand/export.rs
@SteveL-MSFT

Copy link
Copy Markdown
Member

Gijs Reijn (@Gijsreyn) you have a test failure and can you see if you can improve the code coverage?

@Gijsreyn

Copy link
Copy Markdown
Collaborator Author

Steve Lee (@SteveL-MSFT) - I'm a bit surprised by the coverage report. The previous run was 41%, and after adding more tests, it dropped to 21% in the latest one.

@SteveL-MSFT

Copy link
Copy Markdown
Member

Gijs Reijn (@Gijsreyn) let me look into the CC report, I think the problem is that deleted lines is being calculated wrong as not covered

Files without LCOV data (e.g., platform-specific code behind #[cfg(windows)]
when only Linux coverage is collected) were incorrectly counting ALL added
lines as uncovered, including non-executable lines like comments and blanks.
This caused coverage to drop significantly when Windows-only files were
modified in a PR.

Fix: skip files without LCOV data since coverage cannot be determined for
uninstrumented code. Also handle the '\ No newline at end of file' diff
marker which could cause off-by-one line number errors after deleted lines.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@SteveL-MSFT

Copy link
Copy Markdown
Member

WG discussed this:

  • remove the exportFilter directive
  • emit INFO message when engine filter is applied
  • the INFO message should indicate this feature as Experimental

@SteveL-MSFT Steve Lee (SteveL-MSFT) moved this from Todo PRs to Reviewed in DSC Working Group Jul 23, 2026
@Gijsreyn

Copy link
Copy Markdown
Collaborator Author

Steve Lee (@SteveL-MSFT) - made the adjustments. Let me know if this is solid.

Comment thread lib/dsc-lib/src/configure/export_filter.rs
Comment thread lib/dsc-lib/src/configure/export_filter.rs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

resources/dism_dsc/optionalfeature.dsc.resource.json:104

  • The schema text for description says it is "Returned by the get operation", but optional_feature::handle_export() now exports full feature info (via get_feature_info()), which likely includes description. The schema should reflect what export actually returns.
                            "description": {
                                "type": "string",
                                "title": "Description",
                                "description": "The description of the optional feature. Returned by the get operation."
                            },

Comment thread resources/dism_dsc/optionalfeature.dsc.resource.json
Comment thread lib/dsc-lib/src/configure/mod.rs
Comment thread dsc/tests/dsc_export.tests.ps1 Outdated
Comment thread resources/dism_dsc/src/optional_feature/export.rs
Comment thread resources/dism_dsc/src/windows_feature/export.rs Outdated
@SteveL-MSFT Steve Lee (SteveL-MSFT) added this to the 3.3-Approved milestone Aug 5, 2026

@SteveL-MSFT Steve Lee (SteveL-MSFT) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, just occurred to me looking over the latest change that the dism resource manifests need to declare they explicitly support filtering. Although for backwards compat it doesn't need to, we want those resource manifests to represent best practice.

@Gijsreyn

Copy link
Copy Markdown
Collaborator Author

Sorry, just occurred to me looking over the latest change that the dism resource manifests need to declare they explicitly support filtering. Although for backwards compat it doesn't need to, we want those resource manifests to represent best practice.

Done! And also bumped up the version number for good measure.

@SteveL-MSFT
Steve Lee (SteveL-MSFT) added this pull request to the merge queue Aug 7, 2026
Merged via the queue into PowerShell:main with commit 6be9f83 Aug 7, 2026
56 of 62 checks passed
Steve Lee (SteveL-MSFT) added a commit that referenced this pull request Aug 10, 2026
* feat: implement export filter functionality for resource exports (#1621)

* feat: implement export filter functionality for resource exports

* Remove comment

* feat: implement export filter functionality for resource exports

* Remove comment

* Fix Copilot remarks

* Wrong commit

* Remove the wildcard support for resources

* Fix Copilot remarks

* Attempt to increase code coverage and fix test

* Add additional test for coverage

* Add test and fix dism_dsc

* fix: correct code coverage calculation for uninstrumented files

Files without LCOV data (e.g., platform-specific code behind #[cfg(windows)]
when only Linux coverage is collected) were incorrectly counting ALL added
lines as uncovered, including non-executable lines like comments and blanks.
This caused coverage to drop significantly when Windows-only files were
modified in a PR.

Fix: skip files without LCOV data since coverage cannot be determined for
uninstrumented code. Also handle the '\ No newline at end of file' diff
marker which could cause off-by-one line number errors after deleted lines.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Refactor work on services

* Remove unused key

* Revert change

* Restore native resource filtering and add engine filtering fallback

* Update resource definitions

* Fix Copilot remark

* remove directive

* Change comment wording and update tests

* Add crate

* Revert change and fix test

* Update resource manifest

---------

Co-authored-by: Steve Lee <slee@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* (GH-1563) Enhance the PSScript resources (#1657)

* (GH-1563) Canonicalize `_inDesiredState` for PSScript schema

Prior to this change, the PSScript resource manifests embedded a schema
that minimally defined the `_inDesiredState` canonical property instead
of referencing the canonical schema for that property.

This change updates the embedded schemas to reference the canonical
schema and adds the embedded schema by ID to the `$defs` keyword.

* (GH-1563) Canonicalize script properties for PSScript schemas

Prior to this change, the embedded schemas in the manifests for the
transitional PowerShell script resources defined the script properties
with a minimal schema allowing the value to be any string or null. They
didn't include any other keywords.

This change:

1. Defines a shared subschema for the script properties, which they now
   reference instead of redefining the constraints for each property.
1. Removed `null` from the allowed types, since defining a script
    property as `null` isn't valid but _not_ specifying the property at
    all _is_ valid.
1. Added the `writeOnly` keyword to the script properties, since
   they're only used for input and not output.
1. Added the `contentMediaType` keyword to the script properties to
   clearly indicate---but not validate---that the value is expected to
   be a PowerShell script.
1. Added the `minLength` keyword to the script properties to ensure that
   an empty string isn't used as a script value.
1. Adds the `title` and `description` keywords to the script properties
   to provide more context for each property in the schema.

* (GH-1563) Canonicalize `input` for PSScript resource schemas

Prior to this change, the `input` property for the PSScript resource
schemas defined the valid types with two problems:

1. It allowed `null` as a valid type, which caused failures when the
   resource invoked the scripts.
1. It didn't allow non-integer numbers, erroneously preventing users
   from defining input values like `3.14`, even though `[3.14]` was
   accepted.

This change:

1. Removes `null` from the valid types for `input`.
1. Adds `number` to the valid types for `input`, allowing non-integer
   numbers to be used as input values.
1. Adds the `title` and `description` keywords to the `input` property,
   providing better documentation for users.

* (GH-1563) Add constraints to PSScript resource schema

Prior to this change, the embedded schemas for the PSScript resources
didn't require any properties to be defined. For input, a user should
always define at least one script property. For output, the resource
should always return one of three value shapes:

1. For `test` operations, the resource should always define the
  `inDesiredState` property.
1. For implemented `get` and `set` operations, the resource should
   define the `output` property when the script emits any output to the
   success stream.
1. When `get` and `set` operations aren't implemented or don't emit any
   output, the resource should return an empty object.

This change adds constraints to the embedded schemas with the `oneOf`
keyword and nested `oneOf`/`anyOf` keywords to enforce these rules.

* (GH-1563) Add docs keywords for PSScript resources

Prior to this change, the embedded schema for the PSScript resources
didn't define the `title` or `description` keywords.

This change:

1. Adds `title` and `description` keywords to the embedded schema for
   the PSScript resources to provide some documentation for them.
1. Defines the `$schema` keyword explicitly.

* (GH-1563) Canonicalize output for PSScript resources

Prior to this change, the PSScript resources:

1. Always returned an array of output objects, even if there was only
   one object.
1. Didn't ensure that enums in script output were serialized as
   strings, causing enums to emit as integers, which is likely not what
   the user intended and makes review more difficult.

This change:

1. Ensures that if a script returns a single object, it's serialized as
   that object instead of nested in an array.
1. Adds the `-EnumAsString` parameter to the `ConvertTo-Json` call for
   emitting script output when invoked through PowerShell to ensure
   that enums are serialized as strings. This enhancement doesn't
   affect Windows PowerShell, which doesn't support the `-EnumAsString`
   parameter.
1. Updates the tests to reflect the new behavior of returning a single
   object instead of an array.
1. Fixes the tests to use correct casing for the script properties,
   now that the schema validation checks for those properties.

* (GH-1563) Fix tests using invalid casing for PSScript resource

Prior to this change, some tests were using invalid casing for the
PSScript resource type, which didn't previously cause errors during
the JSON Schema validation because the schema didn't forbid additional
properties. The tests passed because PowerShell isn't case-sensitive,
so was able to retrieve the properties even with incorrect casing.

With the changes to the JSON schema _requiring_ one or more script
properties, these tests began to fail because the invalidly-cased
property names didn't satisfy the JSON Schema.

This change corrects the casing in the test definitions to ensure both
that the tests are correct and pass the schema validation.

* Apply suggestions from review

Co-authored-by: Steve Lee <slee@microsoft.com>

---------

Co-authored-by: Steve Lee <slee@microsoft.com>

* Add JSON schema caching (#1637)

* Add JSON schema caching

* address copilot feedback

* fix copilot feedback to not deserializing the json

* rename function to be singular

* make helpers scoped to crate

* make cache crate only

* fix: add support for preserving quoted group names in sshd_config (#1639)

* fix: add support for preserving quoted group names in sshd_config

* Update with remarks Tess

---------

Co-authored-by: Gijs Reijn <26114636+Gijsreyn@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Mikey Lombardi (He/Him) <michael.t.lombardi@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Reviewed

Development

Successfully merging this pull request may close these issues.

Allow post-filtering for export

3 participants