Use `.hie` files for the Hyperlinker backend (#977)

# Summary

This is a large architectural change to the Hyperlinker.

  * extract link (and now also type) information from `.hie` instead
    of doing ad-hoc SYB traversals of the `RenamedSource`. Also
    adds a superb type-on-hover feature (#715).

 * re-engineer the lexer to avoid needless string conversions. By going
    directly through GHC's `P` monad and taking bytestring slices, we
    avoid a ton of allocation and have better handling of position
    pragmas and CPP.

In terms of performance, the Haddock side of things has gotten _much_
more efficient. Unfortunately, much of this is cancelled out by the
increased GHC workload for generating `.hie` files. For the full set of
boot libs (including `ghc`-the-library)

  * the sum of total time went down by 9-10% overall
  * the sum of total allocations went down by 6-7%

# Motivation

Haddock is moving towards working entirely over `.hi` and `.hie` files.
This change means we no longer need the `RenamedSource` from
`TypecheckedModule` (something which is _not_ in `.hi` files).

# Details

Along the way a bunch of things were fixed:

 * Cross package (and other) links are now more reliable (#496)
 * The lexer tries to recover from errors on every line (instead of at CPP
    boundaries)
 * `LINE`/`COLUMN` pragmas are taken into account
 * filter out zero length tokens before rendering
 * avoid recomputing the `ModuleName`-based `SrcMap`
 * remove the last use of `Documentation.Haddock.Utf8` (see  #998)
 * restructure temporary folder logic for `.hi`/`.hie` model

3 files changed, 47 insertions, 0 deletions

diff --git a/hypsrc-test/src/ClangCppBug.hs b/hypsrc-test/src/ClangCppBug.hs
new file mode 100644
index 00000000..4b0bc35f
--- /dev/null
+++ b/hypsrc-test/src/ClangCppBug.hs
@@ -0,0 +1,21 @@
+{-# LANGUAGE CPP #-}
+module ClangCppBug where
+
+foo :: Int
+foo = 1
+
+-- Clang doesn't mind these:
+#define BAX 2
+{-# INLINE bar #-}
+
+bar :: Int
+bar = 3
+
+-- But it doesn't like this:
+{-# RULES
+"bar/qux" bar = qux
+"qux/foo" qux = foo
+  #-}
+
+qux :: Int
+qux = 88
diff --git a/hypsrc-test/src/LinkingIdentifiers.hs b/hypsrc-test/src/LinkingIdentifiers.hs
new file mode 100644
index 00000000..4fff9776
--- /dev/null
+++ b/hypsrc-test/src/LinkingIdentifiers.hs
@@ -0,0 +1,14 @@
+-- Tests that the identifers/operators are properly linked even when:
+--
+--   * backquoted, parenthesized, vanilla
+--   * qualified, not-qualified
+--
+module LinkingIdentifiers where
+
+ident :: Int -> Int -> Int
+x `ident` 2 = (x `ident` 2) + (x `LinkingIdentifiers.ident` 2)
+ident x 2 = ident x 2 + LinkingIdentifiers.ident x 2
+
+(++:++) :: Int -> Int -> Int
+x ++:++ 2 = (x ++:++ 2) + (x LinkingIdentifiers.++:++ 2)
+(++:++) x 2 = (++:++) x 2 + (LinkingIdentifiers.++:++) x 2
diff --git a/hypsrc-test/src/PositionPragmas.hs b/hypsrc-test/src/PositionPragmas.hs
new file mode 100644
index 00000000..907316fd
--- /dev/null
+++ b/hypsrc-test/src/PositionPragmas.hs
@@ -0,0 +1,12 @@
+module PositionPragmas where
+
+{-# LINE 8 "hypsrc-test/src/PositionPragmas.hs" #-}
+
+foo :: String
+foo = bar
+
+{-# LINE 23 "hypsrc-test/src/PositionPragmas.hs" #-}
+
+bar :: String
+bar = foo 
+