Improve hyperlinker's 'spanToNewline' (#846)

'spanToNewline' is used to help break apart the source into lines which
can then be partioned into CPP and non-CPP chunks. It is important that
'spanToNewline' not break apart tokens, so it needs to properly handle
things like

  * block comments, possibly nested
  * string literals, possibly multi-line
  * CPP macros, possibly multi-line

String literals in particular were not being properly handled. The fix
is to to fall back in 'Text.Read.lex' to help lex things that are not
comments.

Fixes #837.

1 files changed, 26 insertions, 0 deletions

diff --git a/hypsrc-test/src/CPP.hs b/hypsrc-test/src/CPP.hs
new file mode 100644
index 00000000..f00ce031
--- /dev/null
+++ b/hypsrc-test/src/CPP.hs
@@ -0,0 +1,26 @@
+{-# LANGUAGE CPP #-}
+module CPP where
+
+#define SOMETHING1
+
+foo :: String
+foo = {-  " single quotes are fine in block comments
+          {- nested block comments are fine -}
+       -} "foo"
+
+#define SOMETHING2
+
+bar :: String
+bar = "block comment in a string is not a comment {- "
+
+#define SOMETHING3
+
+-- " single quotes are fine in line comments
+-- {- unclosed block comments are fine in line comments
+
+-- Multiline CPP is also fine
+#define FOO\
+  1
+
+baz :: String
+baz = "line comment in a string is not a comment --"