From e03d6fa3a9e59b66b85c547daf84f969dca384df Mon Sep 17 00:00:00 2001 From: Nick Fitzgerald Date: Thu, 5 Oct 2017 10:22:03 -0700 Subject: Make the C-Smith fuzzing README a .md file --- csmith-fuzzing/README | 7 ------- csmith-fuzzing/README.md | 7 +++++++ 2 files changed, 7 insertions(+), 7 deletions(-) delete mode 100644 csmith-fuzzing/README create mode 100644 csmith-fuzzing/README.md diff --git a/csmith-fuzzing/README b/csmith-fuzzing/README deleted file mode 100644 index e4c74243..00000000 --- a/csmith-fuzzing/README +++ /dev/null @@ -1,7 +0,0 @@ -Fuzz bindgen with `csmith` https://github.com/csmith-project/csmith . - -Run with `python3 driver.py`. It will run until until it encounters an error in `bindgen`. - -Requires `python3`, `csmith` and `bindgen` to be in `$PATH`. - -csmith is run with `--no-checksum --nomain --max-block-size 1 --max-block-depth 1` which disables the `main` function and makes function bodies as simple as possible as bindgen does not care about them but they cannot be completely disabled in csmith. Run `csmith --help` to see what exactly those options do. diff --git a/csmith-fuzzing/README.md b/csmith-fuzzing/README.md new file mode 100644 index 00000000..e4c74243 --- /dev/null +++ b/csmith-fuzzing/README.md @@ -0,0 +1,7 @@ +Fuzz bindgen with `csmith` https://github.com/csmith-project/csmith . + +Run with `python3 driver.py`. It will run until until it encounters an error in `bindgen`. + +Requires `python3`, `csmith` and `bindgen` to be in `$PATH`. + +csmith is run with `--no-checksum --nomain --max-block-size 1 --max-block-depth 1` which disables the `main` function and makes function bodies as simple as possible as bindgen does not care about them but they cannot be completely disabled in csmith. Run `csmith --help` to see what exactly those options do. -- cgit v1.2.3 From a25a2f66ba160862f33c76bdb292c7fd42669eaf Mon Sep 17 00:00:00 2001 From: Nick Fitzgerald Date: Thu, 5 Oct 2017 10:26:00 -0700 Subject: Add a blurb about C-Smith fuzzing to CONTRIBUTING.md And link to the csmith-fuzzing/README.md file for more details. Fixes #1038 --- CONTRIBUTING.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 49eacc1b..a3562c8b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,6 +19,7 @@ out to us in a GitHub issue, or stop by - [Testing a Single Header's Bindings Generation and Compiling its Bindings](#testing-a-single-headers-bindings-generation-and-compiling-its-bindings) - [Authoring New Tests](#authoring-new-tests) - [Test Expectations and `libclang` Versions](#test-expectations-and-libclang-versions) + - [Fuzzing `bindgen` with `csmith`](#fuzzing-bindgen-with-csmith) - [Code Overview](#code-overview) - [Pull Requests and Code Reviews](#pull-requests-and-code-reviews) - [Generating Graphviz Dot Files](#generating-graphviz-dot-files) @@ -193,6 +194,14 @@ Where `$VERSION` is one of: depending on which version of `libclang` you have installed. +### Fuzzing `bindgen` with `csmith` + +We <3 finding hidden bugs and the people who help us find them! One way to help +uncover hidden bugs is by running `csmith` to generate random headers to test +`bindgen` against. + +See [./csmith-fuzzing/README.md](./csmith-fuzzing/README.md) for details. + ## Code Overview `bindgen` takes C and C++ header files as input and generates corresponding Rust -- cgit v1.2.3 From d97cba5caec6fa2d784f03ab3eae8ea97c1abeb2 Mon Sep 17 00:00:00 2001 From: Nick Fitzgerald Date: Thu, 5 Oct 2017 10:48:47 -0700 Subject: Make the `csmith` driver script executable --- csmith-fuzzing/driver.py | 2 ++ 1 file changed, 2 insertions(+) mode change 100644 => 100755 csmith-fuzzing/driver.py diff --git a/csmith-fuzzing/driver.py b/csmith-fuzzing/driver.py old mode 100644 new mode 100755 index f4f00f87..38ae9ff3 --- a/csmith-fuzzing/driver.py +++ b/csmith-fuzzing/driver.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 + import os, sys from subprocess import run, SubprocessError, DEVNULL, PIPE from tempfile import NamedTemporaryFile -- cgit v1.2.3 From 29fff6a5a23c7b8e483df00b83352ac8849bf212 Mon Sep 17 00:00:00 2001 From: Nick Fitzgerald Date: Thu, 5 Oct 2017 10:49:11 -0700 Subject: Extend the `csmith` README --- csmith-fuzzing/README.md | 64 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 61 insertions(+), 3 deletions(-) diff --git a/csmith-fuzzing/README.md b/csmith-fuzzing/README.md index e4c74243..258ab5e7 100644 --- a/csmith-fuzzing/README.md +++ b/csmith-fuzzing/README.md @@ -1,7 +1,65 @@ -Fuzz bindgen with `csmith` https://github.com/csmith-project/csmith . +# Fuzzing `bindgen` with `csmith` -Run with `python3 driver.py`. It will run until until it encounters an error in `bindgen`. +[`csmith`][csmith] generates random C and C++ programs that can be used as test +cases for compilers. When testing `bindgen` with `csmith`, we interpret the +generated programs as header files, and emit Rust bindings to them. If `bindgen` +panics, the emitted bindings won't compile with `rustc`, or the generated layout +tests in the bindings fail, then we report an issue containing the test case! + + + + + +- [Prerequisites](#prerequisites) +- [Running the Fuzzer](#running-the-fuzzer) +- [Reporting Issues](#reporting-issues) + + + +## Prerequisites Requires `python3`, `csmith` and `bindgen` to be in `$PATH`. -csmith is run with `--no-checksum --nomain --max-block-size 1 --max-block-depth 1` which disables the `main` function and makes function bodies as simple as possible as bindgen does not care about them but they cannot be completely disabled in csmith. Run `csmith --help` to see what exactly those options do. +Many OS package managers have a `csmith` package: + +``` +$ sudo apt install csmith +$ brew install csmith +$ # Etc... +``` + +## Running the Fuzzer + +Run `csmith` and test `bindgen` on the generated test cases with this command: + +``` +$ ./driver.py +``` + +The driver will keep running until it encounters an error in `bindgen`. + +Each invocation of `./driver.py` will use its own temporary directories, so +running it in multiple terminals in parallel is supported. + +`csmith` is run with `--no-checksum --nomain --max-block-size 1 +--max-block-depth 1` which disables the `main` function, and makes function +bodies as simple as possible as `bindgen` does not care about them, but they +cannot be completely disabled in `csmith`. Run `csmith --help` to see what +exactly those options do. + +## Reporting Issues + +Once the fuzz driver finds a test case that causes some kind of error in +`bindgen` or its emitted bindings, it is helpful to +[run C-Reduce on the test case][creducing] to remove the parts that are +irrelevant to reproducing the error. This is ***very*** helpful for the folks +who further investigate the issue and come up with a fix! + +Additionally, mention that you discovered the issue via `csmith` and we will add +the `A-csmith` label. You can find all the issues discovered with `csmith`, and +related to fuzzing with `csmith`, by looking up +[all issues tagged with the `A-csmith` label][csmith-issues]. + +[csmith]: https://github.com/csmith-project/csmith +[creducing]: ../CONTRIBUTING.md#using-creduce-to-minimize-test-cases +[csmith-issues]: https://github.com/rust-lang-nursery/rust-bindgen/issues?q=label%3AA-csmith -- cgit v1.2.3