-
Notifications
You must be signed in to change notification settings - Fork 164
feats: extend ValuePool by new fields : files, and maxMutations #1033
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
de56692
bbe5f01
3da5408
31c022e
581c515
49403fd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -317,6 +317,169 @@ class SimpleClassFuzzTests { | |||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| ## @ValuePool: Guide fuzzing with custom values | ||||||
|
|
||||||
| The `@ValuePool` annotation lets you provide concrete example values of any [supported type](#supported-types) (except for cache-based mutators) that Jazzer's mutators will use when generating test inputs. | ||||||
| This helps guide fuzzing toward realistic or edge-case values relevant to your application. | ||||||
|
|
||||||
| ### Basic Usage | ||||||
|
|
||||||
| You can apply `@ValuePool` in two places: | ||||||
| - **On method parameter (sub-)types** - values apply only to the annotated types | ||||||
| - **On the test method itself** - values propagate to all matching types across all parameters | ||||||
|
|
||||||
| **Example:** | ||||||
| ```java | ||||||
| @FuzzTest | ||||||
| void fuzzTest(Map<@ValuePool(value = {"mySupplier"}) String, Integer> foo) { | ||||||
| // Strings from mySupplier feed the Map's String mutator | ||||||
| } | ||||||
|
|
||||||
| @FuzzTest | ||||||
| void anotherFuzzTest(@ValuePool(value = {"mySupplier"}) Map<String, Integer> foo) { | ||||||
| // Strings from mySupplier feed the Map's String mutator | ||||||
| // Integers from mySupplier feed the Map's Integer mutator | ||||||
| } | ||||||
|
|
||||||
| @FuzzTest | ||||||
| @ValuePool(value = {"mySupplier"}) | ||||||
| void yetAnotherFuzzTest(Map<String, Integer> foo, String bar) { | ||||||
| // Values propagate to ALL matching types: | ||||||
| // - String mutator for Map keys in 'foo' | ||||||
| // - String mutator for 'bar' | ||||||
| // - Integer mutator for Map values in 'foo' | ||||||
| } | ||||||
|
|
||||||
| static Stream mySupplier() { | ||||||
| return Stream.of("example1", "example2", "example3", 1232187321, -182371); | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| ### How Type Matching Works | ||||||
|
|
||||||
| Jazzer automatically routes values to mutators based on type: | ||||||
| - Strings in your value pool → String mutators | ||||||
| - Integers in your value pool → Integer mutators | ||||||
| - Byte arrays in your value pool → byte[] mutators | ||||||
|
|
||||||
| **Type propagation happens recursively by default**, so a `@ValuePool` on a `Map<String, Integer>` will feed both the String mutator (for keys) and Integer mutator (for values). | ||||||
|
|
||||||
| --- | ||||||
|
|
||||||
| ### Supplying Values: Two Mechanisms | ||||||
|
|
||||||
| #### 1. Supplier Methods (`value` field) | ||||||
|
|
||||||
| Provide the names of static methods that return `Stream<?>`: | ||||||
| ```java | ||||||
| @ValuePool(value = {"mySupplier", "anotherSupplier"}) | ||||||
| ``` | ||||||
|
|
||||||
| **Requirements:** | ||||||
| - Methods must be `static` | ||||||
| - Must return `Stream<?>` | ||||||
| - Can contain mixed types (Jazzer routes by type automatically) | ||||||
|
|
||||||
| #### 2. File Patterns (`files` field) | ||||||
|
|
||||||
| Load files as `byte[]` arrays using glob patterns: | ||||||
| ```java | ||||||
| @ValuePool(files = {"*.jpeg"}) // All JPEGs in working dir | ||||||
| @ValuePool(files = {"**.xml"}) // All XMLs recursively | ||||||
| @ValuePool(files = {"/absolute/path/**"}) // All files from absolute path | ||||||
| @ValuePool(files = {"*.jpg", "**.png"}) // Multiple patterns | ||||||
| ``` | ||||||
|
|
||||||
| **Glob syntax:** Follows `java.nio.file.PathMatcher` with `glob:` pattern rules. | ||||||
|
|
||||||
| **You can combine both mechanisms:** | ||||||
| ```java | ||||||
| @ValuePool(value = {"mySupplier"}, files = {"test-data/*.json"}) | ||||||
| ``` | ||||||
|
|
||||||
| --- | ||||||
|
|
||||||
| ### Configuration Options | ||||||
|
|
||||||
| #### Mutation Probability (`p` field) | ||||||
| Controls how often values from the pool are used versus other mutation strategies. | ||||||
| ```java | ||||||
| @ValuePool(value = {"mySupplier"}, p = 0.3) // Use pool values 30% of the time | ||||||
| ``` | ||||||
|
|
||||||
| **Default:** `p = 0.1` (10% of mutations use pool values) | ||||||
| **Range:** 0.0 to 1.0 | ||||||
|
|
||||||
| #### Type Propagation (`constraint` field) | ||||||
|
|
||||||
| Controls whether the annotation affects nested types: | ||||||
| ```java | ||||||
| // Default: RECURSIVE - applies to all nested types | ||||||
| @ValuePool(value = {"mySupplier"}, constraint = Constraint.RECURSIVE) | ||||||
|
|
||||||
| // DECLARATION - applies only to the annotated type, not subtypes | ||||||
| @ValuePool(value = {"mySupplier"}, constraint = Constraint.DECLARATION) | ||||||
| ``` | ||||||
|
|
||||||
| **Example of the difference:** | ||||||
| ```java | ||||||
| // With RECURSIVE (default): | ||||||
| @ValuePool(value = {"valuesSupplier"}) Map<String, Integer> data | ||||||
| // The supplier feed both Map keys AND values | ||||||
|
|
||||||
| // With DECLARATION: | ||||||
| @ValuePool(value = {"valuesSupplier"}, constraint = DECLARATION) Map<String, Integer> data | ||||||
| // The supplier only feeds the Map, NOT keys or values---it should contain Map instances to have effect | ||||||
| ``` | ||||||
|
|
||||||
| --- | ||||||
|
|
||||||
| ### Complete Example | ||||||
| ```java | ||||||
| class MyFuzzTest { | ||||||
| static Stream edgeCases() { | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To avoid raw types?
Suggested change
|
||||||
| return Stream.of( | ||||||
| "", "null", "alert('xss')", // Strings | ||||||
| 0, -1, Integer.MAX_VALUE, // Integers | ||||||
| new byte[]{0x00, 0xFF}, // A byte array | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Causes a compilation error because
Suggested change
|
||||||
| Map.of("Test", 42) // A Map | ||||||
| ); | ||||||
| } | ||||||
|
|
||||||
| @FuzzTest | ||||||
| @ValuePool( | ||||||
| value = {"edgeCases"}, | ||||||
| files = {"test-inputs/*.bin"}, | ||||||
| p = 0.25 // Use pool values 25% of the time | ||||||
| ) | ||||||
| void testParser(String input, Map<String, Integer> config, byte[] data) { | ||||||
| // All three parameters get values from the pool: | ||||||
| // - 'input' gets Strings | ||||||
| // - 'config' keys get Strings, values get Integers, Map itself gets the Map instance | ||||||
| // - 'data' gets bytes from both edgeCases() and *.bin files | ||||||
| } | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| --- | ||||||
|
|
||||||
| #### Max Mutations (`maxMutations` field) | ||||||
|
|
||||||
| After selecting a value from the pool, the mutator can apply additional random mutations to it. | ||||||
| ```java | ||||||
| @ValuePool(value = {"mySupplier"}, maxMutations = 5) | ||||||
| ``` | ||||||
|
|
||||||
| **Default:** `maxMutations = 1` (at most one additional mutation applied) | ||||||
| **Range:** 0 or higher | ||||||
|
|
||||||
| **How it works:** If `maxMutations = 5`, and Jazzer selects the value pool as mutation strategy, Jazzer will: | ||||||
| 1. Select a random value from your pool (e.g., `"alert('xss')"`) | ||||||
| 2. Apply up to 5 random mutations in a row (e.g., `"alert('xss')"` → `"alert(x"` → `"AAAt(x"` → ...) | ||||||
|
|
||||||
| This helps explore variations of your seed values while staying close to realistic inputs. | ||||||
|
|
||||||
|
|
||||||
| ## FuzzedDataProvider | ||||||
|
|
||||||
| The `FuzzedDataProvider` is an alternative approach commonly used in programming | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -27,7 +27,7 @@ | |||||
| import java.lang.annotation.Target; | ||||||
|
|
||||||
| /** | ||||||
| * Provides values to user-selected mutator types to start fuzzing from. | ||||||
| * Provides values to user-selected types that will be used during mutation. | ||||||
| * | ||||||
| * <p>This annotation can be applied to fuzz test methods and any parameter type or subtype. By | ||||||
| * default, this annotation is propagated to all nested subtypes unless specified otherwise via the | ||||||
|
|
@@ -72,17 +72,44 @@ | |||||
| * don't need to match the type of the annotated method or parameter. The mutation framework will | ||||||
| * extract only the values that are compatible with the target type. | ||||||
| */ | ||||||
| String[] value(); | ||||||
| String[] value() default {}; | ||||||
|
|
||||||
| /** | ||||||
| * Specifies glob patterns matching files that should be provided as {@code byte []} to the | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Redundant space? (or is that intentional?)
Suggested change
|
||||||
| * annotated type. The syntax follows closely to Java's {@link | ||||||
| * java.nio.file.FileSystem#getPathMatcher(String) PathMatcher} "glob:" syntax. | ||||||
| * | ||||||
| * <p>Relative glob patterns are resolved against the working directory. | ||||||
| * | ||||||
| * <p>Examples: | ||||||
| * | ||||||
| * <ul> | ||||||
| * <li>{@code *.jpeg} - matches all jpegs in the working directory | ||||||
| * <li>{@code **.xml} - matches all xml files recursively | ||||||
| * <li>{@code src/test/resources/dict/*.txt} - matches txt files in a specific directory | ||||||
| * <li>{@code /absolute/path/to/some/directory/**} - matches all files in an absolute path | ||||||
| * recursively | ||||||
| * <li>{@code {"*.jpg", "**.png"}} - matches all jpg in the working directory, and png files | ||||||
| * recursively | ||||||
| * </ul> | ||||||
| */ | ||||||
| String[] files() default {}; | ||||||
|
|
||||||
| /** | ||||||
| * This {@code ValuePool} will be used with probability {@code p} by the mutator responsible for | ||||||
| * fitting types. | ||||||
| */ | ||||||
| double p() default 0.1; | ||||||
|
|
||||||
| /** | ||||||
| * If the mutator selects a value from this {@code ValuePool}, it will perform up to {@code | ||||||
| * maxMutations} additional mutations on the selected value. | ||||||
| */ | ||||||
| int maxMutations() default 1; | ||||||
|
|
||||||
| /** | ||||||
| * Defines the scope of the annotation. Possible values are defined in {@link | ||||||
| * com.code_intelligence.jazzer.mutation.utils.PropertyConstraint}. By default it's {@code | ||||||
| * com.code_intelligence.jazzer.mutation.utils.PropertyConstraint}. By default, it's {@code | ||||||
| * RECURSIVE}. | ||||||
| */ | ||||||
| String constraint() default RECURSIVE; | ||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To avoid raw types?