Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions docs/mutation-framework.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,169 @@ class SimpleClassFuzzTests {
}
```

## @ValuePool: Guide fuzzing with custom values

The `@ValuePool` annotation lets you provide concrete example values of any [supported type](#supported-types) (except for cache-based mutators) that Jazzer's mutators will use when generating test inputs.
This helps guide fuzzing toward realistic or edge-case values relevant to your application.

### Basic Usage

You can apply `@ValuePool` in two places:
- **On method parameter (sub-)types** - values apply only to the annotated types
- **On the test method itself** - values propagate to all matching types across all parameters

**Example:**
```java
@FuzzTest
void fuzzTest(Map<@ValuePool(value = {"mySupplier"}) String, Integer> foo) {
// Strings from mySupplier feed the Map's String mutator
}

@FuzzTest
void anotherFuzzTest(@ValuePool(value = {"mySupplier"}) Map<String, Integer> foo) {
// Strings from mySupplier feed the Map's String mutator
// Integers from mySupplier feed the Map's Integer mutator
}

@FuzzTest
@ValuePool(value = {"mySupplier"})
void yetAnotherFuzzTest(Map<String, Integer> foo, String bar) {
// Values propagate to ALL matching types:
// - String mutator for Map keys in 'foo'
// - String mutator for 'bar'
// - Integer mutator for Map values in 'foo'
}

static Stream mySupplier() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid raw types?

Suggested change
static Stream mySupplier() {
static Stream<?> mySupplier() {

return Stream.of("example1", "example2", "example3", 1232187321, -182371);
}
```

### How Type Matching Works

Jazzer automatically routes values to mutators based on type:
- Strings in your value pool → String mutators
- Integers in your value pool → Integer mutators
- Byte arrays in your value pool → byte[] mutators

**Type propagation happens recursively by default**, so a `@ValuePool` on a `Map<String, Integer>` will feed both the String mutator (for keys) and Integer mutator (for values).

---

### Supplying Values: Two Mechanisms

#### 1. Supplier Methods (`value` field)

Provide the names of static methods that return `Stream<?>`:
```java
@ValuePool(value = {"mySupplier", "anotherSupplier"})
```

**Requirements:**
- Methods must be `static`
- Must return `Stream<?>`
- Can contain mixed types (Jazzer routes by type automatically)

#### 2. File Patterns (`files` field)

Load files as `byte[]` arrays using glob patterns:
```java
@ValuePool(files = {"*.jpeg"}) // All JPEGs in working dir
@ValuePool(files = {"**.xml"}) // All XMLs recursively
@ValuePool(files = {"/absolute/path/**"}) // All files from absolute path
@ValuePool(files = {"*.jpg", "**.png"}) // Multiple patterns
```

**Glob syntax:** Follows `java.nio.file.PathMatcher` with `glob:` pattern rules.

**You can combine both mechanisms:**
```java
@ValuePool(value = {"mySupplier"}, files = {"test-data/*.json"})
```

---

### Configuration Options

#### Mutation Probability (`p` field)
Controls how often values from the pool are used versus other mutation strategies.
```java
@ValuePool(value = {"mySupplier"}, p = 0.3) // Use pool values 30% of the time
```

**Default:** `p = 0.1` (10% of mutations use pool values)
**Range:** 0.0 to 1.0

#### Type Propagation (`constraint` field)

Controls whether the annotation affects nested types:
```java
// Default: RECURSIVE - applies to all nested types
@ValuePool(value = {"mySupplier"}, constraint = Constraint.RECURSIVE)

// DECLARATION - applies only to the annotated type, not subtypes
@ValuePool(value = {"mySupplier"}, constraint = Constraint.DECLARATION)
```

**Example of the difference:**
```java
// With RECURSIVE (default):
@ValuePool(value = {"valuesSupplier"}) Map<String, Integer> data
// The supplier feed both Map keys AND values

// With DECLARATION:
@ValuePool(value = {"valuesSupplier"}, constraint = DECLARATION) Map<String, Integer> data
// The supplier only feeds the Map, NOT keys or values---it should contain Map instances to have effect
```

---

### Complete Example
```java
class MyFuzzTest {
static Stream edgeCases() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid raw types?

Suggested change
static Stream edgeCases() {
static Stream<?> edgeCases() {

return Stream.of(
"", "null", "alert('xss')", // Strings
0, -1, Integer.MAX_VALUE, // Integers
new byte[]{0x00, 0xFF}, // A byte array
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Causes a compilation error because 0xFF is an int literal > Byte.MAX_VALUE. Maybe choose something else for simplicity (avoiding a (byte) cast)?

Suggested change
new byte[]{0x00, 0xFF}, // A byte array
new byte[]{0x00, 0x7F}, // A byte array

Map.of("Test", 42) // A Map
);
}

@FuzzTest
@ValuePool(
value = {"edgeCases"},
files = {"test-inputs/*.bin"},
p = 0.25 // Use pool values 25% of the time
)
void testParser(String input, Map<String, Integer> config, byte[] data) {
// All three parameters get values from the pool:
// - 'input' gets Strings
// - 'config' keys get Strings, values get Integers, Map itself gets the Map instance
// - 'data' gets bytes from both edgeCases() and *.bin files
}
}
```

---

#### Max Mutations (`maxMutations` field)

After selecting a value from the pool, the mutator can apply additional random mutations to it.
```java
@ValuePool(value = {"mySupplier"}, maxMutations = 5)
```

**Default:** `maxMutations = 1` (at most one additional mutation applied)
**Range:** 0 or higher

**How it works:** If `maxMutations = 5`, and Jazzer selects the value pool as mutation strategy, Jazzer will:
1. Select a random value from your pool (e.g., `"alert('xss')"`)
2. Apply up to 5 random mutations in a row (e.g., `"alert('xss')"` → `"alert(x"` → `"AAAt(x"` → ...)

This helps explore variations of your seed values while staying close to realistic inputs.


## FuzzedDataProvider

The `FuzzedDataProvider` is an alternative approach commonly used in programming
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ load("//bazel:fuzz_target.bzl", "java_fuzz_target_test")

java_fuzz_target_test(
name = "ArgumentsMutatorFuzzTest",
timeout = "long",
srcs = [
"ArgumentsMutatorFuzzTest.java",
"BeanWithParent.java",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,8 @@ private static Path addInputAndSeedDirs(
Paths.get(context.getConfigurationParameter("jazzer.internal.basedir").orElse(""))
.toAbsolutePath();

System.setProperty("jazzer.internal.basedir", baseDir.toString());

// Use the specified corpus dir, if given, otherwise store the generated corpus in a per-class
// directory under the project root.
// The path is specified relative to the current working directory, which with JUnit is the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,17 @@
import java.lang.reflect.AnnotatedType;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;

public final class ArgumentsMutator {
private final ExtendedMutatorFactory mutatorFactory;
private final Method method;
private final InPlaceProductMutator productMutator;

private static final Map<Method, ArgumentsMutator> mutatorsCache = new ConcurrentHashMap<>();

private Object[] arguments;

/**
Expand Down Expand Up @@ -78,15 +82,14 @@ private static String prettyPrintMethod(Method method) {
}

public static ArgumentsMutator forMethodOrThrow(Method method) {
return forMethod(Mutators.newFactory(new ValuePoolRegistry(method)), method)
.orElseThrow(
() ->
new IllegalArgumentException(
"Failed to construct mutator for " + prettyPrintMethod(method)));
}

public static Optional<ArgumentsMutator> forMethod(Method method) {
return forMethod(Mutators.newFactory(new ValuePoolRegistry(method)), method);
return mutatorsCache.computeIfAbsent(
method,
m ->
forMethod(Mutators.newFactory(new ValuePoolRegistry(m)), m)
.orElseThrow(
() ->
new IllegalArgumentException(
"Failed to construct mutator for " + prettyPrintMethod(m))));
}

public static Optional<ArgumentsMutator> forMethod(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
import java.lang.annotation.Target;

/**
* Provides values to user-selected mutator types to start fuzzing from.
* Provides values to user-selected types that will be used during mutation.
*
* <p>This annotation can be applied to fuzz test methods and any parameter type or subtype. By
* default, this annotation is propagated to all nested subtypes unless specified otherwise via the
Expand Down Expand Up @@ -72,17 +72,44 @@
* don't need to match the type of the annotated method or parameter. The mutation framework will
* extract only the values that are compatible with the target type.
*/
String[] value();
String[] value() default {};

/**
* Specifies glob patterns matching files that should be provided as {@code byte []} to the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant space? (or is that intentional?)

Suggested change
* Specifies glob patterns matching files that should be provided as {@code byte []} to the
* Specifies glob patterns matching files that should be provided as {@code byte[]} to the

* annotated type. The syntax follows closely to Java's {@link
* java.nio.file.FileSystem#getPathMatcher(String) PathMatcher} "glob:" syntax.
*
* <p>Relative glob patterns are resolved against the working directory.
*
* <p>Examples:
*
* <ul>
* <li>{@code *.jpeg} - matches all jpegs in the working directory
* <li>{@code **.xml} - matches all xml files recursively
* <li>{@code src/test/resources/dict/*.txt} - matches txt files in a specific directory
* <li>{@code /absolute/path/to/some/directory/**} - matches all files in an absolute path
* recursively
* <li>{@code {"*.jpg", "**.png"}} - matches all jpg in the working directory, and png files
* recursively
* </ul>
*/
String[] files() default {};

/**
* This {@code ValuePool} will be used with probability {@code p} by the mutator responsible for
* fitting types.
*/
double p() default 0.1;

/**
* If the mutator selects a value from this {@code ValuePool}, it will perform up to {@code
* maxMutations} additional mutations on the selected value.
*/
int maxMutations() default 1;

/**
* Defines the scope of the annotation. Possible values are defined in {@link
* com.code_intelligence.jazzer.mutation.utils.PropertyConstraint}. By default it's {@code
* com.code_intelligence.jazzer.mutation.utils.PropertyConstraint}. By default, it's {@code
* RECURSIVE}.
*/
String constraint() default RECURSIVE;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@
import java.util.Optional;
import java.util.function.Predicate;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class ValuePoolMutatorFactory implements MutatorFactory {
/** Types annotated with this marker wil not be re-wrapped by this factory. */
Expand Down Expand Up @@ -75,12 +74,17 @@ private static final class ValuePoolMutator<T> extends SerializingMutator<T> {
private final SerializingMutator<T> mutator;
private final List<T> userValues;
private final double poolUsageProbability;
private final int maxMutations;

ValuePoolMutator(
SerializingMutator<T> mutator, List<T> userValues, double poolUsageProbability) {
SerializingMutator<T> mutator,
List<T> userValues,
double poolUsageProbability,
int maxMutations) {
this.mutator = mutator;
this.userValues = userValues;
this.poolUsageProbability = poolUsageProbability;
this.maxMutations = maxMutations;
}

@SuppressWarnings("unchecked")
Expand All @@ -91,14 +95,9 @@ static <T> SerializingMutator<T> wrapIfValuesExist(
return mutator;
}

Optional<Stream<?>> rawUserValues = valuePoolRegistry.extractRawValues(type);
if (!rawUserValues.isPresent()) {
return mutator;
}

List<T> userValues =
rawUserValues
.get()
valuePoolRegistry
.extractUserValues(type)
// Values whose round trip serialization is not stable violate either some user
// annotations on the type (e.g. @InRange), or the default mutator limits (e.g.
// default List size limits) and are therefore not suitable for inclusion in the value
Expand All @@ -112,7 +111,8 @@ static <T> SerializingMutator<T> wrapIfValuesExist(
}

double p = valuePoolRegistry.extractFirstProbability(type);
return new ValuePoolMutator<>(mutator, userValues, p);
int maxMutations = valuePoolRegistry.extractFirstMaxMutations(type);
return new ValuePoolMutator<>(mutator, userValues, p, maxMutations);
}

/**
Expand Down Expand Up @@ -144,8 +144,8 @@ private static <T> boolean isSerializationStable(SerializingMutator<T> mutator,
@Override
public String toDebugString(Predicate<Debuggable> isInCycle) {
return String.format(
"%s (values: %d p: %.2f)",
mutator.toDebugString(isInCycle), userValues.size(), poolUsageProbability);
"%s (values: %d p: %.2f, maxMutations: %d)",
mutator.toDebugString(isInCycle), userValues.size(), poolUsageProbability, maxMutations);
}

@Override
Expand Down Expand Up @@ -180,19 +180,30 @@ public T init(PseudoRandom prng) {
@Override
public T mutate(T value, PseudoRandom prng) {
if (prng.closedRange(0.0, 1.0) < poolUsageProbability) {
if (prng.choice()) {
return prng.pickIn(userValues);
} else {
// treat the value from valuePool as a starting point for mutation
return mutator.mutate(prng.pickIn(userValues), prng);
value = prng.pickIn(userValues);
// Treat the user value as a starting point for mutation
int mutations = prng.closedRange(0, maxMutations);
for (int i = 0; i < mutations; i++) {
value = mutator.mutate(value, prng);
}
return value;
}
return mutator.mutate(value, prng);
}

@Override
public T crossOver(T value, T otherValue, PseudoRandom prng) {
return mutator.crossOver(value, otherValue, prng);
if (prng.closedRange(0.0, 1.0) < poolUsageProbability) {
value = prng.pickIn(userValues);
// Treat the user value as a starting point for crossOver
int mutations = prng.closedRange(0, maxMutations);
for (int i = 0; i < mutations; i++) {
value = mutator.crossOver(value, prng.pickIn(userValues), prng);
}
return value;
} else {
return mutator.crossOver(value, otherValue, prng);
}
}
}
}
Loading
Loading