Rewrite the core of the binding generator.

TL;DR: The binding generator is a mess as of right now. At first it was funny
(in a "this is challenging" sense) to improve on it, but this is not
sustainable.

The truth is that the current architecture of the binding generator is a huge
pile of hacks, so these few days I've been working on rewriting it with a few
goals.

 1) Have the hacks as contained and identified as possible. They're sometimes
    needed because how clang exposes the AST, but ideally those hacks are well
    identified and don't interact randomly with each others.

    As an example, in the current bindgen when scanning the parameters of a
    function that references a struct clones all the struct information, then if
    the struct name changes (because we mangle it), everything breaks.

 2) Support extending the bindgen output without having to deal with clang. The
    way I'm aiming to do this is separating completely the parsing stage from
    the code generation one, and providing a single id for each item the binding
    generator provides.

 3) No more random mutation of the internal representation from anywhere. That
    means no more Rc<RefCell<T>>, no more random circular references, no more
    borrow_state... nothing.

 4) No more deduplication of declarations before code generation.

    Current bindgen has a stage, called `tag_dup_decl`[1], that takes care of
    deduplicating declarations. That's completely buggy, and for C++ it's a
    complete mess, since we YOLO modify the world.

    I've managed to take rid of this using the clang canonical declaration, and
    the definition, to avoid scanning any type/item twice.

 5) Code generation should not modify any internal data structure. It can lookup
    things, traverse whatever it needs, but not modifying randomly.

 6) Each item should have a canonical name, and a single source of mangling
    logic, and that should be computed from the inmutable state, at code
    generation.

    I've put a few canonical_name stuff in the code generation phase, but it's
    still not complete, and should change if I implement namespaces.

Improvements pending until this can land:

 1) Add support for missing core stuff, mainly generating functions (note that
    we parse the signatures for types correctly though), bitfields, generating
    C++ methods.

 2) Add support for the necessary features that were added to work around some
    C++ pitfalls, like opaque types, etc...

 3) Add support for the sugar that Manish added recently.

 4) Optionally (and I guess this can land without it, because basically nobody
    uses it since it's so buggy), bring back namespace support.

These are not completely trivial, but I think I can do them quite easily with
the current architecture.

I'm putting the current state of affairs here as a request for comments... Any
thoughts? Note that there are still a few smells I want to eventually
re-redesign, like the ParseError::Recurse thing, but until that happens I'm
way happier with this kind of architecture.

I'm keeping the old `parser.rs` and `gen.rs` in tree just for reference while I
code, but they will go away.

[1]: https://github.com/Yamakaky/rust-bindgen/blob/master/src/gen.rs#L448

24 files changed, 180 insertions, 6 deletions

diff --git a/tests/headers/anon_enum.hpp b/tests/headers/anon_enum.hpp
new file mode 100644
index 00000000..c7405202
--- /dev/null
+++ b/tests/headers/anon_enum.hpp
@@ -0,0 +1,5 @@
+struct Test {
+  int foo;
+  float bar;
+  enum { T_NONE };
+};
diff --git a/tests/headers/anon_enum_whitelist.h b/tests/headers/anon_enum_whitelist.h
new file mode 100644
index 00000000..15cda6b1
--- /dev/null
+++ b/tests/headers/anon_enum_whitelist.h
@@ -0,0 +1,6 @@
+// bindgen-flags: --whitelist-var NODE_.*
+
+enum {
+  NODE_FLAG_FOO,
+  NODE_FLAG_BAR,
+};
diff --git a/tests/headers/anon_union.hpp b/tests/headers/anon_union.hpp
new file mode 100644
index 00000000..126f6a6e
--- /dev/null
+++ b/tests/headers/anon_union.hpp
@@ -0,0 +1,20 @@
+template<typename T>
+struct TErrorResult {
+  enum UnionState {
+    HasMessage,
+    HasException,
+  };
+  int mResult;
+  struct Message;
+  struct DOMExceptionInfo;
+  union {
+    Message* mMessage;
+    DOMExceptionInfo* mDOMExceptionInfo;
+  };
+
+  bool mMightHaveUnreported;
+  UnionState mUnionState;
+};
+
+struct ErrorResult : public TErrorResult<int> {
+};
diff --git a/tests/headers/arg_keyword.hpp b/tests/headers/arg_keyword.hpp
new file mode 100644
index 00000000..9f0af850
--- /dev/null
+++ b/tests/headers/arg_keyword.hpp
@@ -0,0 +1 @@
+void foo(const char* type);
diff --git a/tests/headers/const_ptr.hpp b/tests/headers/const_ptr.hpp
new file mode 100644
index 00000000..66744f8b
--- /dev/null
+++ b/tests/headers/const_ptr.hpp
@@ -0,0 +1,3 @@
+extern "C" {
+  void foo(const void* bar);
+}
diff --git a/tests/headers/const_resolved_ty.h b/tests/headers/const_resolved_ty.h
new file mode 100644
index 00000000..2521e61c
--- /dev/null
+++ b/tests/headers/const_resolved_ty.h
@@ -0,0 +1,3 @@
+typedef unsigned char uint8_t;
+
+void foo(const uint8_t* foo);
diff --git a/tests/headers/const_tparam.hpp b/tests/headers/const_tparam.hpp
new file mode 100644
index 00000000..a2db574c
--- /dev/null
+++ b/tests/headers/const_tparam.hpp
@@ -0,0 +1,4 @@
+template<typename T>
+class C {
+  const T* const foo;
+};
diff --git a/tests/headers/duplicated_constants_in_ns.hpp b/tests/headers/duplicated_constants_in_ns.hpp
index 42197a16..bb343641 100644
--- a/tests/headers/duplicated_constants_in_ns.hpp
+++ b/tests/headers/duplicated_constants_in_ns.hpp
@@ -1,4 +1,4 @@
-// bindgen-flags: --no-namespaced-constants
+// bindgen-flags: --enable-cxx-namespaces
 namespace foo {
     const int FOO = 4;
 }
diff --git a/tests/headers/empty_template_param_name.hpp b/tests/headers/empty_template_param_name.hpp
new file mode 100644
index 00000000..b3360bc9
--- /dev/null
+++ b/tests/headers/empty_template_param_name.hpp
@@ -0,0 +1,4 @@
+template<typename...> using __void_t = void;
+
+template<typename _Iterator, typename = __void_t<>>
+  struct __iterator_traits { };
diff --git a/tests/headers/enum_alias.hpp b/tests/headers/enum_alias.hpp
new file mode 100644
index 00000000..658f8fde
--- /dev/null
+++ b/tests/headers/enum_alias.hpp
@@ -0,0 +1,7 @@
+// bindgen-flags: -- -std=c++11
+
+typedef unsigned char uint8_t;
+
+enum Bar : uint8_t {
+  VAL
+};
diff --git a/tests/headers/in_class_typedef.hpp b/tests/headers/in_class_typedef.hpp
new file mode 100644
index 00000000..dda7472d
--- /dev/null
+++ b/tests/headers/in_class_typedef.hpp
@@ -0,0 +1,10 @@
+
+template<typename T>
+class Foo {
+  typedef T elem_type;
+  typedef T* ptr_type;
+
+  typedef struct Bar {
+    int x, y;
+  } Bar;
+};
diff --git a/tests/headers/inherit_named.hpp b/tests/headers/inherit_named.hpp
new file mode 100644
index 00000000..9881d1b6
--- /dev/null
+++ b/tests/headers/inherit_named.hpp
@@ -0,0 +1,5 @@
+template<typename T>
+class Wohoo {};
+
+template<typename T>
+class Weeee : public T {};
diff --git a/tests/headers/inherit_typedef.hpp b/tests/headers/inherit_typedef.hpp
new file mode 100644
index 00000000..8d699e82
--- /dev/null
+++ b/tests/headers/inherit_typedef.hpp
@@ -0,0 +1,5 @@
+struct Foo {};
+
+typedef Foo TypedefedFoo;
+
+struct Bar: public TypedefedFoo {};
diff --git a/tests/headers/inner_const.hpp b/tests/headers/inner_const.hpp
new file mode 100644
index 00000000..25c2e603
--- /dev/null
+++ b/tests/headers/inner_const.hpp
@@ -0,0 +1,6 @@
+
+class Foo {
+  static int BOO;
+  static Foo whatever;
+  int bar;
+};
diff --git a/tests/headers/jsval_layout_opaque.hpp b/tests/headers/jsval_layout_opaque.hpp
index d432d8d3..85c5be63 100644
--- a/tests/headers/jsval_layout_opaque.hpp
+++ b/tests/headers/jsval_layout_opaque.hpp
@@ -1,7 +1,16 @@
-// bindgen-flags: --match jsval_layout_opaque.hpp --no-type-renaming --no-unstable-rust -- -std=c++11
+// bindgen-flags: --no-unstable-rust -- -std=c++11
+
+/**
+ * These typedefs are hacky, but keep our tests consistent across 64-bit
+ * platforms, otherwise the id's change and our CI is unhappy.
+ */
+typedef unsigned char uint8_t;
+typedef int int32_t;
+typedef unsigned int uint32_t;
+typedef unsigned long long uint64_t;
+typedef unsigned long long size_t;
+typedef unsigned long long uintptr_t;
 
-#include <stdint.h>
-#include <stddef.h>
 
 #define JS_PUNBOX64
 #define IS_LITTLE_ENDIAN
diff --git a/tests/headers/nested_vtable.hpp b/tests/headers/nested_vtable.hpp
new file mode 100644
index 00000000..87d6ce1f
--- /dev/null
+++ b/tests/headers/nested_vtable.hpp
@@ -0,0 +1,8 @@
+class nsISupports {
+public:
+  virtual nsISupports* QueryInterface();
+};
+
+class nsIRunnable : public nsISupports {};
+
+class Runnable : public nsIRunnable {};
diff --git a/tests/headers/opaque_typedef.hpp b/tests/headers/opaque_typedef.hpp
index 2d9a5781..25640738 100644
--- a/tests/headers/opaque_typedef.hpp
+++ b/tests/headers/opaque_typedef.hpp
@@ -14,4 +14,4 @@ class Wat3<3>;
 /** <div rustbindgen opaque></div> */
 typedef RandomTemplate<int> ShouldBeOpaque;
 
-typedef RandomTemplate<int> ShouldNotBeOpaque;
+typedef RandomTemplate<float> ShouldNotBeOpaque;
diff --git a/tests/headers/private.hpp b/tests/headers/private.hpp
index 070bdddc..c0f3ce7f 100644
--- a/tests/headers/private.hpp
+++ b/tests/headers/private.hpp
@@ -15,7 +15,7 @@ struct VeryPrivate {
 
 /** <div rustbindgen private></div> */
 struct ContradictPrivate {
-    /** <div rustbindgen private=false></div> */
+    /** <div rustbindgen private="false"></div> */
     int mNotPrivate;
     int mIsPrivate;
 };
diff --git a/tests/headers/redeclaration.hpp b/tests/headers/redeclaration.hpp
new file mode 100644
index 00000000..d536b158
--- /dev/null
+++ b/tests/headers/redeclaration.hpp
@@ -0,0 +1,7 @@
+extern "C" {
+  void foo();
+}
+
+extern "C" {
+  void foo();
+}
diff --git a/tests/headers/typeref.hpp b/tests/headers/typeref.hpp
new file mode 100644
index 00000000..b94c98ef
--- /dev/null
+++ b/tests/headers/typeref.hpp
@@ -0,0 +1,28 @@
+struct nsFoo;
+
+namespace mozilla {
+
+struct FragmentOrURL { bool mIsLocalRef; };
+struct Position { };
+
+} // namespace mozilla
+
+class Bar {
+  nsFoo* mFoo;
+};
+
+namespace mozilla {
+
+template<typename ReferenceBox>
+struct StyleShapeSource {
+  union {
+    Position* mPosition;
+    FragmentOrURL* mFragmentOrURL;
+  };
+};
+
+} // namespace mozilla
+
+struct nsFoo {
+  mozilla::StyleShapeSource<int> mBar;
+};
diff --git a/tests/headers/unknown_attr.h b/tests/headers/unknown_attr.h
new file mode 100644
index 00000000..f87e9f0b
--- /dev/null
+++ b/tests/headers/unknown_attr.h
@@ -0,0 +1,6 @@
+typedef struct {
+  long long __clang_max_align_nonce1
+      __attribute__((__aligned__(__alignof__(long long))));
+  long double __clang_max_align_nonce2
+      __attribute__((__aligned__(__alignof__(long double))));
+} max_align_t;
diff --git a/tests/headers/virtual_dtor.hpp b/tests/headers/virtual_dtor.hpp
new file mode 100644
index 00000000..c35dcab1
--- /dev/null
+++ b/tests/headers/virtual_dtor.hpp
@@ -0,0 +1,3 @@
+struct nsSlots {
+    virtual ~nsSlots();
+};
diff --git a/tests/headers/what_is_going_on.hpp b/tests/headers/what_is_going_on.hpp
new file mode 100644
index 00000000..078c1ad5
--- /dev/null
+++ b/tests/headers/what_is_going_on.hpp
@@ -0,0 +1,19 @@
+
+struct UnknownUnits {};
+typedef float Float;
+
+template<class units, class F = Float>
+struct PointTyped {
+  F x;
+  F y;
+
+  static PointTyped<units, F> FromUnknownPoint(const PointTyped<UnknownUnits, F>& aPoint) {
+    return PointTyped<units, F>(aPoint.x, aPoint.y);
+  }
+
+  PointTyped<UnknownUnits, F> ToUnknownPoint() const {
+    return PointTyped<UnknownUnits, F>(this->x, this->y);
+  }
+};
+
+typedef PointTyped<UnknownUnits> IntPoint;
diff --git a/tests/headers/whitelist_basic.hpp b/tests/headers/whitelist_basic.hpp
new file mode 100644
index 00000000..661528ba
--- /dev/null
+++ b/tests/headers/whitelist_basic.hpp
@@ -0,0 +1,15 @@
+// bindgen-flags: --whitelist-type WhitelistMe
+
+template<typename T>
+class WhitelistMe {
+  class Inner {
+    T bar;
+  };
+
+  int foo;
+  Inner bar;
+};
+
+struct DontWhitelistMe {
+  void* foo;
+};