Quickchecking crate CLI

Prior to this commit the quickchecking crate used for generating proprty
tests
for bindgen was a [lib] target and had configurations that required
commenting/uncommenting code to enable/disable. This meant it was
inconvienent/prohibitive to configure the property tests on a per-run
basis.
This commit reorganizes the `quickchecking` crate to provide both [lib]
and
[[bin]] targets in order to expose those configurations through a CLI.

The configurations that are exposed through the [[bin]] target's CLI
are:
* Count/number of tests to run.
* Directory to provide fuzzed headers
* Generation range corresponding to the range quickcheck uses to
* generate arbitrary.

__Usage from the__ `tests/quickchecking` __directory__
```bash
quickchecking 0.2.0
Bindgen property tests with quickcheck. Generate random valid C code and
pass it to the csmith/predicate.py script

USAGE:
    quickchecking [OPTIONS]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -c, --count <COUNT>    Count / number of tests to run. Running a
fuzzed header through the predicate.py script can
                           take a long time, especially if the
generation range is large. Increase this number if you're
                           willing to wait a while. [default: 2]
    -p, --path <PATH>      Optional. Preserve generated headers for
inspection, provide directory path for header
                           output. [default: None]
    -r, --range <RANGE>    Sets the range quickcheck uses during
generation. Corresponds to things like arbitrary usize
                           and arbitrary vector length. This number
doesn't have to grow much for that execution time to
                           increase significantly. [default: 32]
```

Because the actual work of running the property tests moved to the
[[bin]]
target, rather than duplicate that code in the `quickchecking` crate's
tests
directory, some actual (very basic) tests for the `quickchecking` crate
were
added.

*Note: I'm not attached to any of the option flags, if there are better
characters/words for any of the options I've exposed I'll be happy to
revise!

Thanks for taking a look, looking forward to feedback!

Closes #1168

r? @fitzgen

1 files changed, 98 insertions, 0 deletions

diff --git a/tests/quickchecking/src/lib.rs b/tests/quickchecking/src/lib.rs
index 3bea8a8e..d8633dfb 100644
--- a/tests/quickchecking/src/lib.rs
+++ b/tests/quickchecking/src/lib.rs
@@ -20,9 +20,107 @@
 //! ```
 //!
 #![deny(missing_docs)]
+#[macro_use]
+extern crate lazy_static;
 extern crate quickcheck;
 extern crate rand;
 extern crate tempdir;
 
+use std::sync::Mutex;
+use quickcheck::{QuickCheck, StdGen, TestResult};
+use std::fs::File;
+use std::io::Write;
+use tempdir::TempDir;
+use std::process::{Command, Output};
+use std::path::PathBuf;
+use std::error::Error;
+use rand::thread_rng;
+
 /// Contains definitions of and impls for types used to fuzz C declarations.
 pub mod fuzzers;
+
+// Global singleton, manages context across tests. For now that context is
+// only the output_path for inspecting fuzzed headers (if specified).
+struct Context {
+    output_path: Option<String>,
+}
+
+// Initialize global context.
+lazy_static! {
+    static ref CONTEXT: Mutex<Context> = Mutex::new(Context { output_path: None });
+}
+
+// Passes fuzzed header to the `csmith-fuzzing/predicate.py` script, returns
+// output of the associated command.
+fn run_predicate_script(header: fuzzers::HeaderC) -> Result<Output, Box<Error>> {
+    let dir = TempDir::new("bindgen_prop")?;
+    let header_path = dir.path().join("prop_test.h");
+
+    let mut header_file = File::create(&header_path)?;
+    header_file.write_all(header.to_string().as_bytes())?;
+    header_file.sync_all()?;
+
+    let header_path_string;
+    match header_path.into_os_string().into_string() {
+        Ok(s) => header_path_string = s,
+        Err(_) => return Err(From::from("error converting path into String")),
+    }
+
+    let mut predicate_script_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    predicate_script_path.push("../../csmith-fuzzing/predicate.py");
+
+    let predicate_script_path_string;
+    match predicate_script_path.into_os_string().into_string() {
+        Ok(s) => predicate_script_path_string = s,
+        Err(_) => return Err(From::from("error converting path into String")),
+    }
+
+    // Copy generated temp files to output_path directory for inspection.
+    // If `None`, output path not specified, don't copy.
+    match CONTEXT.lock().unwrap().output_path {
+        Some(ref path) => {
+            Command::new("cp")
+                .arg("-a")
+                .arg(&dir.path().to_str().unwrap())
+                .arg(&path)
+                .output()?;
+        }
+        None => {}
+    }
+
+    Ok(Command::new(&predicate_script_path_string)
+        .arg(&header_path_string)
+        .output()?)
+}
+
+// Generatable property. Pass generated headers off to run through the
+// `csmith-fuzzing/predicate.py` script. Success is measured by the success
+// status of that command.
+fn bindgen_prop(header: fuzzers::HeaderC) -> TestResult {
+    match run_predicate_script(header) {
+        Ok(o) => return TestResult::from_bool(o.status.success()),
+        Err(e) => {
+            println!("{:?}", e);
+            return TestResult::from_bool(false);
+        }
+    }
+}
+
+/// Instantiate a Quickcheck object and use it to run property tests using
+/// fuzzed C headers generated with types defined in the `fuzzers` module.
+/// Success/Failure is dictated by the result of passing the fuzzed headers
+/// to the `csmith-fuzzing/predicate.py` script.
+pub fn test_bindgen(generate_range: usize, tests: usize, output_path: Option<&str>) {
+    match output_path {
+        Some(path) => {
+            CONTEXT.lock().unwrap().output_path =
+                Some(String::from(PathBuf::from(path).to_str().unwrap()));
+        }
+        None => {} // Path not specified, don't provide output.
+    }
+
+    QuickCheck::new()
+        .tests(tests)
+        .gen(StdGen::new(thread_rng(), generate_range))
+        .quickcheck(bindgen_prop as fn(fuzzers::HeaderC) -> TestResult)
+}