feat: implement export filter functionality for resource exports - #1621
Conversation
ba1baec to
664abad
Compare
There was a problem hiding this comment.
Pull request overview
Implements engine-side post-filtering for dsc config export results via a new exportFilter resource directive, enabling consistent wildcard/equality filtering even for resources that don’t implement native export filtering.
Changes:
- Adds
exportFilterto the configuration resource directives schema and applies it in the export pipeline afterresource.export(...). - Introduces
configure::export_filtermatching logic (case-insensitive*wildcard for strings; OR across filter objects, AND within each object) plus unit tests. - Adds Pester coverage for
exportFilterbehavior and updates thedsctestexport resource to emit anameproperty for filtering.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/dsctest/src/main.rs | Emits per-instance name values to enable filter-driven tests. |
| tools/dsctest/src/export.rs | Extends the test export payload schema with optional name. |
| lib/dsc-lib/src/dscresources/command_resource.rs | Adjusts when “export filtering not supported” is raised (only for non-empty input). |
| lib/dsc-lib/src/configure/mod.rs | Wires exportFilter from directives into the export pipeline. |
| lib/dsc-lib/src/configure/export_filter.rs | Adds the engine-side filtering implementation + Rust unit tests. |
| lib/dsc-lib/src/configure/config_doc.rs | Adds export_filter directive field to the config document model/schema. |
| lib/dsc-lib/locales/en-us.toml | Adds localized message for filter reduction logging. |
| dsc/tests/dsc_export.tests.ps1 | Adds integration tests validating exportFilter behavior across scenarios. |
| dsc/src/resource_command.rs | Updates call site for the changed export helper function signature. |
Steve Lee (SteveL-MSFT)
left a comment
There was a problem hiding this comment.
Changes look fine, but can you remove the wildcard filtering support in the service, firewall, and dism resources? I would expect all the tests to still pass (assuming they don't call the resource exe directly, if so, they would need to be udpated to go through dsc).
|
Steve Lee (@SteveL-MSFT) - good catch, but it had some caveats. I've replaced them with tests that append |
|
Gijs Reijn (@Gijsreyn) you have a test failure and can you see if you can improve the code coverage? |
|
Steve Lee (@SteveL-MSFT) - I'm a bit surprised by the coverage report. The previous run was 41%, and after adding more tests, it dropped to 21% in the latest one. |
|
Gijs Reijn (@Gijsreyn) let me look into the CC report, I think the problem is that deleted lines is being calculated wrong as not covered |
Files without LCOV data (e.g., platform-specific code behind #[cfg(windows)] when only Linux coverage is collected) were incorrectly counting ALL added lines as uncovered, including non-executable lines like comments and blanks. This caused coverage to drop significantly when Windows-only files were modified in a PR. Fix: skip files without LCOV data since coverage cannot be determined for uninstrumented code. Also handle the '\ No newline at end of file' diff marker which could cause off-by-one line number errors after deleted lines. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
WG discussed this:
|
|
Steve Lee (@SteveL-MSFT) - made the adjustments. Let me know if this is solid. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
resources/dism_dsc/optionalfeature.dsc.resource.json:104
- The schema text for
descriptionsays it is "Returned by the get operation", butoptional_feature::handle_export()now exports full feature info (viaget_feature_info()), which likely includesdescription. The schema should reflect what export actually returns.
"description": {
"type": "string",
"title": "Description",
"description": "The description of the optional feature. Returned by the get operation."
},
Steve Lee (SteveL-MSFT)
left a comment
There was a problem hiding this comment.
Sorry, just occurred to me looking over the latest change that the dism resource manifests need to declare they explicitly support filtering. Although for backwards compat it doesn't need to, we want those resource manifests to represent best practice.
Done! And also bumped up the version number for good measure. |
* feat: implement export filter functionality for resource exports (#1621) * feat: implement export filter functionality for resource exports * Remove comment * feat: implement export filter functionality for resource exports * Remove comment * Fix Copilot remarks * Wrong commit * Remove the wildcard support for resources * Fix Copilot remarks * Attempt to increase code coverage and fix test * Add additional test for coverage * Add test and fix dism_dsc * fix: correct code coverage calculation for uninstrumented files Files without LCOV data (e.g., platform-specific code behind #[cfg(windows)] when only Linux coverage is collected) were incorrectly counting ALL added lines as uncovered, including non-executable lines like comments and blanks. This caused coverage to drop significantly when Windows-only files were modified in a PR. Fix: skip files without LCOV data since coverage cannot be determined for uninstrumented code. Also handle the '\ No newline at end of file' diff marker which could cause off-by-one line number errors after deleted lines. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Refactor work on services * Remove unused key * Revert change * Restore native resource filtering and add engine filtering fallback * Update resource definitions * Fix Copilot remark * remove directive * Change comment wording and update tests * Add crate * Revert change and fix test * Update resource manifest --------- Co-authored-by: Steve Lee <slee@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * (GH-1563) Enhance the PSScript resources (#1657) * (GH-1563) Canonicalize `_inDesiredState` for PSScript schema Prior to this change, the PSScript resource manifests embedded a schema that minimally defined the `_inDesiredState` canonical property instead of referencing the canonical schema for that property. This change updates the embedded schemas to reference the canonical schema and adds the embedded schema by ID to the `$defs` keyword. * (GH-1563) Canonicalize script properties for PSScript schemas Prior to this change, the embedded schemas in the manifests for the transitional PowerShell script resources defined the script properties with a minimal schema allowing the value to be any string or null. They didn't include any other keywords. This change: 1. Defines a shared subschema for the script properties, which they now reference instead of redefining the constraints for each property. 1. Removed `null` from the allowed types, since defining a script property as `null` isn't valid but _not_ specifying the property at all _is_ valid. 1. Added the `writeOnly` keyword to the script properties, since they're only used for input and not output. 1. Added the `contentMediaType` keyword to the script properties to clearly indicate---but not validate---that the value is expected to be a PowerShell script. 1. Added the `minLength` keyword to the script properties to ensure that an empty string isn't used as a script value. 1. Adds the `title` and `description` keywords to the script properties to provide more context for each property in the schema. * (GH-1563) Canonicalize `input` for PSScript resource schemas Prior to this change, the `input` property for the PSScript resource schemas defined the valid types with two problems: 1. It allowed `null` as a valid type, which caused failures when the resource invoked the scripts. 1. It didn't allow non-integer numbers, erroneously preventing users from defining input values like `3.14`, even though `[3.14]` was accepted. This change: 1. Removes `null` from the valid types for `input`. 1. Adds `number` to the valid types for `input`, allowing non-integer numbers to be used as input values. 1. Adds the `title` and `description` keywords to the `input` property, providing better documentation for users. * (GH-1563) Add constraints to PSScript resource schema Prior to this change, the embedded schemas for the PSScript resources didn't require any properties to be defined. For input, a user should always define at least one script property. For output, the resource should always return one of three value shapes: 1. For `test` operations, the resource should always define the `inDesiredState` property. 1. For implemented `get` and `set` operations, the resource should define the `output` property when the script emits any output to the success stream. 1. When `get` and `set` operations aren't implemented or don't emit any output, the resource should return an empty object. This change adds constraints to the embedded schemas with the `oneOf` keyword and nested `oneOf`/`anyOf` keywords to enforce these rules. * (GH-1563) Add docs keywords for PSScript resources Prior to this change, the embedded schema for the PSScript resources didn't define the `title` or `description` keywords. This change: 1. Adds `title` and `description` keywords to the embedded schema for the PSScript resources to provide some documentation for them. 1. Defines the `$schema` keyword explicitly. * (GH-1563) Canonicalize output for PSScript resources Prior to this change, the PSScript resources: 1. Always returned an array of output objects, even if there was only one object. 1. Didn't ensure that enums in script output were serialized as strings, causing enums to emit as integers, which is likely not what the user intended and makes review more difficult. This change: 1. Ensures that if a script returns a single object, it's serialized as that object instead of nested in an array. 1. Adds the `-EnumAsString` parameter to the `ConvertTo-Json` call for emitting script output when invoked through PowerShell to ensure that enums are serialized as strings. This enhancement doesn't affect Windows PowerShell, which doesn't support the `-EnumAsString` parameter. 1. Updates the tests to reflect the new behavior of returning a single object instead of an array. 1. Fixes the tests to use correct casing for the script properties, now that the schema validation checks for those properties. * (GH-1563) Fix tests using invalid casing for PSScript resource Prior to this change, some tests were using invalid casing for the PSScript resource type, which didn't previously cause errors during the JSON Schema validation because the schema didn't forbid additional properties. The tests passed because PowerShell isn't case-sensitive, so was able to retrieve the properties even with incorrect casing. With the changes to the JSON schema _requiring_ one or more script properties, these tests began to fail because the invalidly-cased property names didn't satisfy the JSON Schema. This change corrects the casing in the test definitions to ensure both that the tests are correct and pass the schema validation. * Apply suggestions from review Co-authored-by: Steve Lee <slee@microsoft.com> --------- Co-authored-by: Steve Lee <slee@microsoft.com> * Add JSON schema caching (#1637) * Add JSON schema caching * address copilot feedback * fix copilot feedback to not deserializing the json * rename function to be singular * make helpers scoped to crate * make cache crate only * fix: add support for preserving quoted group names in sshd_config (#1639) * fix: add support for preserving quoted group names in sshd_config * Update with remarks Tess --------- Co-authored-by: Gijs Reijn <26114636+Gijsreyn@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Mikey Lombardi (He/Him) <michael.t.lombardi@gmail.com>
PR Summary
Adds engine-side post-filtering for export via a new exportFilter resource directive.
PR Context
Fixes #1486.