diff options
| -rw-r--r-- | CONTRIBUTING.md | 9 | ||||
| -rw-r--r-- | csmith-fuzzing/README | 7 | ||||
| -rw-r--r-- | csmith-fuzzing/README.md | 65 | ||||
| -rwxr-xr-x[-rw-r--r--] | csmith-fuzzing/driver.py | 2 |
4 files changed, 76 insertions, 7 deletions
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 49eacc1b..a3562c8b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,6 +19,7 @@ out to us in a GitHub issue, or stop by - [Testing a Single Header's Bindings Generation and Compiling its Bindings](#testing-a-single-headers-bindings-generation-and-compiling-its-bindings) - [Authoring New Tests](#authoring-new-tests) - [Test Expectations and `libclang` Versions](#test-expectations-and-libclang-versions) + - [Fuzzing `bindgen` with `csmith`](#fuzzing-bindgen-with-csmith) - [Code Overview](#code-overview) - [Pull Requests and Code Reviews](#pull-requests-and-code-reviews) - [Generating Graphviz Dot Files](#generating-graphviz-dot-files) @@ -193,6 +194,14 @@ Where `$VERSION` is one of: depending on which version of `libclang` you have installed. +### Fuzzing `bindgen` with `csmith` + +We <3 finding hidden bugs and the people who help us find them! One way to help +uncover hidden bugs is by running `csmith` to generate random headers to test +`bindgen` against. + +See [./csmith-fuzzing/README.md](./csmith-fuzzing/README.md) for details. + ## Code Overview `bindgen` takes C and C++ header files as input and generates corresponding Rust diff --git a/csmith-fuzzing/README b/csmith-fuzzing/README deleted file mode 100644 index e4c74243..00000000 --- a/csmith-fuzzing/README +++ /dev/null @@ -1,7 +0,0 @@ -Fuzz bindgen with `csmith` https://github.com/csmith-project/csmith . - -Run with `python3 driver.py`. It will run until until it encounters an error in `bindgen`. - -Requires `python3`, `csmith` and `bindgen` to be in `$PATH`. - -csmith is run with `--no-checksum --nomain --max-block-size 1 --max-block-depth 1` which disables the `main` function and makes function bodies as simple as possible as bindgen does not care about them but they cannot be completely disabled in csmith. Run `csmith --help` to see what exactly those options do. diff --git a/csmith-fuzzing/README.md b/csmith-fuzzing/README.md new file mode 100644 index 00000000..258ab5e7 --- /dev/null +++ b/csmith-fuzzing/README.md @@ -0,0 +1,65 @@ +# Fuzzing `bindgen` with `csmith` + +[`csmith`][csmith] generates random C and C++ programs that can be used as test +cases for compilers. When testing `bindgen` with `csmith`, we interpret the +generated programs as header files, and emit Rust bindings to them. If `bindgen` +panics, the emitted bindings won't compile with `rustc`, or the generated layout +tests in the bindings fail, then we report an issue containing the test case! + +<!-- START doctoc generated TOC please keep comment here to allow auto update --> +<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> + + +- [Prerequisites](#prerequisites) +- [Running the Fuzzer](#running-the-fuzzer) +- [Reporting Issues](#reporting-issues) + +<!-- END doctoc generated TOC please keep comment here to allow auto update --> + +## Prerequisites + +Requires `python3`, `csmith` and `bindgen` to be in `$PATH`. + +Many OS package managers have a `csmith` package: + +``` +$ sudo apt install csmith +$ brew install csmith +$ # Etc... +``` + +## Running the Fuzzer + +Run `csmith` and test `bindgen` on the generated test cases with this command: + +``` +$ ./driver.py +``` + +The driver will keep running until it encounters an error in `bindgen`. + +Each invocation of `./driver.py` will use its own temporary directories, so +running it in multiple terminals in parallel is supported. + +`csmith` is run with `--no-checksum --nomain --max-block-size 1 +--max-block-depth 1` which disables the `main` function, and makes function +bodies as simple as possible as `bindgen` does not care about them, but they +cannot be completely disabled in `csmith`. Run `csmith --help` to see what +exactly those options do. + +## Reporting Issues + +Once the fuzz driver finds a test case that causes some kind of error in +`bindgen` or its emitted bindings, it is helpful to +[run C-Reduce on the test case][creducing] to remove the parts that are +irrelevant to reproducing the error. This is ***very*** helpful for the folks +who further investigate the issue and come up with a fix! + +Additionally, mention that you discovered the issue via `csmith` and we will add +the `A-csmith` label. You can find all the issues discovered with `csmith`, and +related to fuzzing with `csmith`, by looking up +[all issues tagged with the `A-csmith` label][csmith-issues]. + +[csmith]: https://github.com/csmith-project/csmith +[creducing]: ../CONTRIBUTING.md#using-creduce-to-minimize-test-cases +[csmith-issues]: https://github.com/rust-lang-nursery/rust-bindgen/issues?q=label%3AA-csmith diff --git a/csmith-fuzzing/driver.py b/csmith-fuzzing/driver.py index f4f00f87..38ae9ff3 100644..100755 --- a/csmith-fuzzing/driver.py +++ b/csmith-fuzzing/driver.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 + import os, sys from subprocess import run, SubprocessError, DEVNULL, PIPE from tempfile import NamedTemporaryFile |