feat(sdk): assertion statement value json #2895
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implement pluggable assertion signing and validation providers to enable integration with external signing mechanisms like HSMs, smart cards (CAC/PIV), and cloud KMS services. ## What's New ### Core Interfaces - Add `AssertionSigningProvider` interface for custom signing implementations - Add `AssertionValidationProvider` interface for custom validation logic - Maintain full backward compatibility with existing DEK-based assertions ### Built-in Providers - `DefaultSigningProvider/DefaultValidationProvider`: Existing DEK-based behavior - `X509SigningProvider/X509ValidationProvider`: X.509 certificate support with x5c headers - `PKCS11Provider`: Template for hardware token integration ### SDK Integration - Add `WithAssertionSigningProvider()` option for TDF creation - Add `WithReaderAssertionValidationProvider()` option for TDF reading - Automatically fall back to default providers when none specified ### Examples - Add comprehensive assertion CLI commands (sign, verify, list) - Support reading assertions from TDF files - Add `--x509-verify` flag to decrypt command for X.509 validation ## Technical Details The implementation follows a provider pattern that allows developers to: 1. Supply custom signing logic while maintaining SDK compatibility 2. Integrate with hardware security modules and smart cards 3. Use X.509 certificates for identity-based assertions 4. Maintain complete backward compatibility with existing code All providers use the standard SDK assertion binding (`assertionHash` and `assertionSig` claims) ensuring full interoperability between tools. ## Testing - Added provider interface tests with mock implementations - Added X.509 provider tests with self-signed certificates - Verified interoperability with otdfctl-created TDFs - Tested backward compatibility with existing assertions
Enhance `--magic-word` handling with a more robust assertion provider setup, including default validation using `NoopAssertionValidationProvider` and state-aware `MagicWordAssertionProvider`.
… command Streamline assertion provider integration: - Remove unnecessary comments and redundant field (`AssertionProvider`) in `MagicWordAssertionProvider`. - Adjust `WithAssertionProviderFactory` to use a pointer for consistency. - Update regex in `decrypt.go` for precise assertion matching.
Streamline SDK by removing unused `assertion_binding.go`, including obsolete legacy binding logic and functions for TDF assertion handling.
…c key signing providers Transition to `PublicKeySigningProvider`, replacing `DefaultSigningProvider` for consistent key-based signing. Introduce `SystemMetadataAssertionProvider` for system metadata handling. Refine assertion provider mechanism in SDK and CLI examples.
Introduce `Verify` method in `Assertion` to validate binding signatures. Update `AssertionProvider` and `Reader` interfaces to support enhanced validation mechanisms. Replace obsolete aggregate hash logic with root signature for assertion binding. Simplify and extend implementation for better clarity and modularity.
…y-based assertions Transition `PayloadKeyProvider` to `PublicKeySigningProvider` for key-based assertions. Introduce `KeyAssertionProvider` for improved assertion setup. Update related interfaces and examples to use RSA keys for signing and validation. Streamline CLI commands for flexible assertion provider setup.
…ertionRegistry` Introduce `AssertionRegistry` as a replacement for `AssertionProviderFactory`, streamlining assertion management. Update SDK and examples to use assertion builders and validators. Replace provider terminology with builder-focused nomenclature for consistency. Modify TDF assertion setup and CLI examples for improved clarity and usability.
…igning Transition from `SignWithProvider` to `Sign` for improved assertion signing. Simplify and streamline signature generation using direct key-based mechanisms. Update related assertion handling and verification methods to align with new design.
…lidation Streamline assertion handling by replacing `KeyAssertionProvider` with `KeyAssertionBuilder` and `KeyAssertionValidator`. Remove obsolete signing and validation providers for clarity. Update CLI and examples to support private key-based validation registry setup. Refine TDF assertion validation flow for improved security and modularity.
…thAssertionRegistryReader` Update SDK and examples to align with updated nomenclature. Replace references to `WithAssertionProviderFactory` with `WithAssertionRegistryReader` for consistency. Adjust decrypt command and related SDK logic to reflect this change.
…handling Transition from `AssertionBuilder` to `AssertionBinder` for enhanced clarity and modularity in assertion handling. Update SDK and examples to streamline assertion registration and binding. Adjust encryption/decryption commands, SDK logic, and TDF configuration to reflect binder-based design.
…n handling Replace legacy assertion provider interfaces with the binder/validator pattern for improved clarity, modularity, and flexibility. Enable developers to implement custom signing (binders) and validation (validators). Update architecture to support regex-based dispatcher for mixed assertion types, maintain backward compatibility, and improve efficiency with post-creation binding.
…ooting guides Remove inline key parsing logic from examples, centralizing it into reusable functions for clarity and maintainability. Update CLI commands for assertion signing/validation with improved error handling. Add comprehensive documentation for assertion troubleshooting and format details.
…prove context usage Remove obsolete `AssertionSigningProvider` and `AssertionValidationProvider` interfaces in favor of binder/validator pattern. Replace unused and redundant hashing logic. Standardize function signatures by eliminating unused parameters. Enhance clarity in examples with added `//nolint` annotations and improved error messaging.
…ata assertions Introduce support for backwards-compatible dual-mode validation, handling both v1 (legacy) and v2 (current) schema versions. Implement `verifyLegacyAssertion` for v1 signature validation using concatenated hashes. Add comprehensive test cases to ensure accurate schema detection and validation behavior. Update documentation to detail schema versions, validation logic, and migration strategies.
…vider # Conflicts: # sdk/tdf.go
…ttern and context integration Integrate `configBasedAssertionBinder` and `KeyAssertionValidator` for enhanced modularity in assertion creation and verification. Register binders and validators dynamically based on assertion configs and keys. Improve compatibility by maintaining DEK-based validation for assertions without explicit verifiers. Adjust tests to focus on functional correctness, removing brittle size validations.
…validation Introduce `slog` for improved visibility into assertion processes. Add debug and error logs during signature parsing, schema auto-detection, and verification outcomes. Enhance troubleshooting with context-aware details like assertion ID, schema, signature length, and error messages.
Skip unnecessary setup if assertion verification is disabled. Add verification for binding signatures in `SystemMetadataAssertionProvider` to enforce signature trustworthiness. Refine legacy compatibility handling by validating only assertions with bindings.
… for `Binding` Implement `MarshalJSON()` in `Assertion` to omit empty `binding` fields from serialization, aligning with cleaner JSON output standards. Introduce `IsEmpty()` method in `Binding` to simplify emptiness checks.
Update signing key logic to mandate cryptographic bindings for all assertions. Refine error handling and improve signature compatibility by supporting both v1 (legacy) and v2 (current) formats with structured logging. Adjust tests to include signing keys and verify updated binding requirements.
…icit keys Implement automatic signing of unsigned assertions using the DEK (payload key) to enforce mandatory cryptographic bindings. Enhance backward compatibility with legacy SDK formats and maintain secure assertion handling across TDFs. Update decision records to reflect new auto-signing logic.
…gnature handling Refactor assertion components to incorporate `ShouldUseHexEncoding` for encoding format detection directly from the manifest. Simplify constructors for `KeyAssertionBinder`, `DEKAssertionValidator`, and `SystemMetadataAssertionProvider` by removing redundant `useHex` and `aggregateHash` parameters. Update tests, examples, and validation logic for improved clarity, consistency, and tampering protection.
…nifest processing Unified assertion signature computation by introducing `VerifyAssertionSignatureFormat` helper. Refactored `DEKAssertionValidator`, `SystemMetadataAssertionProvider`, and related test helpers to eliminate redundant logic and use standardized manifest processing. Enhanced tampering detection and cross-SDK compatibility with updated encoding format handling.
Adds diagrams for assertions --------- Signed-off-by: Scott Hamrick <2623452+cshamrick@users.noreply.github.com>
…v2 schema support Add support for the new `SystemMetadataSchemaV2`, allowing JSON values to be marshaled and unmarshaled as raw objects instead of escaped strings. Update `SystemMetadataAssertionProvider`, tests, and related logic to ensure backward compatibility with V1 schema while supporting the new schema format. Enhance validation, marshaling, and unmarshaling logic for cross-SDK interoperability and precise handling of schema-specific behaviors.
github-actions
bot
added
comp:sdk
Contributor
Summary of ChangesHello @pflynn-virtru, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the OpenTDF SDK's assertion capabilities by introducing a flexible "Binder/Validator Pattern" for custom assertion providers. This architectural change allows for seamless integration with various hardware security modules and cloud key management services, greatly improving the SDK's extensibility and developer experience. The update also includes comprehensive documentation, a troubleshooting guide, and a new CLI example for managing assertions in TDF files, alongside core SDK changes to support flexible JSON assertion values and efficient in-place manifest updates. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. A binder, a validator, new paths unfold, For secrets to keep, a story untold. With JSON in hand, and modes set just right, OpenTDF's shield, shines ever so bright. Footnotes
|
…vider-statement-value-json # Conflicts: # sdk/tdf.go
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a robust and extensible framework for custom assertion providers in the OpenTDF SDK, a significant and well-executed enhancement. The changes, including the new AssertionBinder and AssertionValidator interfaces, the schema-based routing, and the different verification modes, are well-designed and align with the detailed ADR. The addition of comprehensive documentation, troubleshooting guides, and CLI examples greatly improves developer experience.
My review focuses on a few key areas: ensuring documentation consistency with the implementation, improving the efficiency of a core hashing function, and clarifying some logic in the example code. Overall, this is an excellent contribution that significantly advances the SDK's capabilities.
Comment on lines
+62
to
+147
| ### Registering Custom Assertion Providers | ||
|
|
||
| The SDK supports custom signing and validation logic for assertions through a flexible provider model. The recommended way to manage multiple, conditional providers is to use the `ProviderFactory`. | ||
|
|
||
| The `ProviderFactory` lets you register different providers for different assertions, matched via regular expressions on the assertion ID. It implements the `AssertionSigningProvider` and `AssertionValidationProvider` interfaces itself, so you can configure it once and pass it directly to the SDK. | ||
|
|
||
| This allows you to, for example, use a hardware-based signing provider for assertions with a `piv-` prefix, and a default provider for all others. | ||
|
|
||
| **Example:** | ||
|
|
||
| First, you need an implementation of the `AssertionSigningProvider` and/or `AssertionValidationProvider` interfaces. For this example, we'll create a simple mock. | ||
|
|
||
| ```go | ||
| // my_custom_provider.go | ||
| package main | ||
|
|
||
| import ( | ||
| "context" | ||
| "github.com/opentdf/platform/sdk" | ||
| ) | ||
|
|
||
| // CustomSigner is a custom implementation of an assertion signing provider. | ||
| type CustomSigner struct { | ||
| SignatureValue string | ||
| } | ||
|
|
||
| func (s *CustomSigner) Sign(ctx context.Context, assertion *sdk.Assertion, hash, sig string) (string, error) { | ||
| // In a real implementation, you would use a hardware token, an external service, etc. | ||
| return s.SignatureValue, nil | ||
| } | ||
|
|
||
| func (s *CustomSigner) GetSigningKeyReference() string { | ||
| return "custom-signer-key-ref" | ||
| } | ||
|
|
||
| func (s *CustomSigner) GetAlgorithm() string { | ||
| return "CUSTOM_SIG" | ||
| } | ||
| ``` | ||
|
|
||
| Next, register this provider with the factory and pass the factory to the SDK during TDF creation. | ||
|
|
||
| ```go | ||
| // main.go | ||
| package main | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "github.com/opentdf/platform/sdk" | ||
| ) | ||
|
|
||
| func main() { | ||
| // 1. Create a new provider factory. | ||
| factory := sdk.NewProviderFactory() | ||
|
|
||
| // 2. Create an instance of your custom provider. | ||
| mySigner := &CustomSigner{SignatureValue: "a-very-special-signature"} | ||
|
|
||
| // 3. Register the provider with a regex pattern. | ||
| // This will match any assertion ID that starts with "custom-". | ||
| err := factory.RegisterSigningProvider(`^custom-`, mySigner) | ||
| if err != nil { | ||
| panic(err) | ||
| } | ||
|
|
||
| // You can also set a default provider for assertions that don't match any pattern. | ||
| // factory.SetDefaultSigningProvider(sdk.NewDefaultSigningProvider(...)) | ||
|
|
||
| // 4. When creating a TDF, pass the factory directly as a provider. | ||
| // The factory will dispatch to the correct underlying provider internally. | ||
| _, err = s.CreateTDF( | ||
| ciphertext, | ||
| plaintext, | ||
| sdk.WithDataAttributes("https://example.com/attr/Classification/value/Open"), | ||
| sdk.WithAssertionSigningProvider(factory), // Pass the factory here | ||
| ) | ||
|
|
||
| // Similarly, for reading TDFs: | ||
| // reader, err := s.LoadTDF(input, sdk.WithAssertionValidationProvider(factory)) | ||
|
|
||
| fmt.Println("SDK configured to use custom provider factory.") | ||
| } | ||
| ``` | ||
|
|
||
| The factory will test assertion IDs against registered patterns in the order they were registered and use the first one that matches. If no patterns match, a default provider will be used if one has been set. | ||
|
|
Contributor
There was a problem hiding this comment.
The documentation for "Registering Custom Assertion Providers" appears to be out of sync with the implementation in this pull request. The example uses AssertionSigningProvider and a ProviderFactory, whereas the new implementation introduces AssertionBinder/AssertionValidator interfaces and registers them via WithAssertionBinder/WithAssertionValidator options.
To avoid confusion for developers, please update this documentation to reflect the new Binder/Validator pattern, interfaces, and registration methods introduced in this PR. The example should demonstrate the use of WithAssertionBinder and WithAssertionValidator.
Comment on lines
+80
to
+87
| // TODO SECURITY: Add schema claim to cryptographically bind schema to assertion | ||
| // This prevents schema substitution attacks where an attacker changes Statement.Schema | ||
| // to route the assertion to a different validator with weaker security checks. | ||
| // The schema is included in both the JWT (signed) and the Statement (hashed), | ||
| // providing defense-in-depth against tampering. | ||
| // if err := tok.Set("assertionSchema", a.Statement.Schema); err != nil { | ||
| // return fmt.Errorf("failed to set assertion schema: %w", err) | ||
| // } |
Contributor
There was a problem hiding this comment.
This is a great security consideration. To ensure it's not lost, I recommend creating a follow-up ticket to track the implementation of adding the assertionSchema claim to the JWT. This will be a valuable defense-in-depth measure against schema substitution attacks.
Comment on lines
+103
to
104
| autoconfigure = false | ||
| if !autoconfigure { |
Contributor
There was a problem hiding this comment.
The logic here is a bit confusing. autoconfigure is unconditionally set to false, which makes the following if !autoconfigure condition always true. If the intention is to always disable autoconfiguration for this example command, it would be clearer to remove the conditional check and add a comment explaining why it's disabled.
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
NANOTDF Benchmark Results:
|
Contributor
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
NANOTDF Benchmark Results:
|
Contributor
Contributor
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
NANOTDF Benchmark Results:
|
Contributor
pflynn-virtru
changed the base branch from
main
to
feature/assertion-provider
pflynn-virtru
changed the base branch from
feature/assertion-provider
to
main
pflynn-virtru
changed the base branch from
main
to
feature/assertion-provider
pflynn-virtru
changed the base branch from
feature/assertion-provider
to
main
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed Changes
This pull request introduces comprehensive support for custom assertion providers in the OpenTDF SDK, with a focus on extensibility, developer experience, and practical examples. The changes include a new architectural decision record (ADR) documenting the custom assertion provider approach, a detailed troubleshooting guide, an improved examples CLI README, and a new example command for adding assertions to TDF files.
Major features and documentation improvements:
Custom Assertion Provider Architecture
adr/decisions/2025-10-16-custom-assertion-providers.md) defining the "Binder/Validator Pattern" for custom assertion providers, enabling integration with hardware security modules, smart cards, and cloud KMS. The ADR details interface design, registration, security considerations, and cross-SDK compatibility.Developer Guidance and Troubleshooting
docs/Assertions-Troubleshooting.mdwith solutions to common issues when working with assertion providers, including key loading errors, validation failures, and configuration mistakes. Provides actionable steps and debug tips.Examples and CLI Enhancements
examples/README.mdwith a quick start, detailed descriptions of available examples, assertion provider usage, configuration options, and security notes. Clarifies how to use and extend the examples CLI for custom assertion workflows.assertioncommand (examples/cmd/assertion.go) to the examples CLI, allowing users to add custom assertions (e.g., using a "magic word" provider) to existing TDF files without re-encrypting payload data. Demonstrates the Binder interface in practice.Checklist
Testing Instructions