* haddorg-api

*NOTE*. This is the README file to ~haddorg-api~.  For the README file
to haddock, see README.md.

** Overview

Haddock is a documentation generator for Haskell libraries and it
supports several backends including HTML, latex and hoogle.  The
program logic of the ~haddock~ tool is the ~haddock-api~ package.

~haddorg-api~ adds an org backend of ~haddock-api~, so that one can
invoke haddock to generate org files, of the Org Mode.

** Examples

- [[https://ypei.org/assets/haddorg-output/hierarchy-e2f0094c.org.gz][Haskell Hierarchical Libraries]], built at [[https://gitlab.haskell.org/ghc/ghc/-/commit/e2f0094c315746ff15b8d9650cf318f81d8416d7][ghc commit e2f0094c]].
- [[https://ypei.org/assets/haddorg-output/base-4.16.1.0.org.gz][base-4.16.1.0.org]]
- [[https://ypei.org/assets/haddorg-output/ghc-9.5-e2f0094c.org.gz][ghc-9.5-e2f0094c.org]], GHC API docs built at [[https://gitlab.haskell.org/ghc/ghc/-/commit/e2f0094c315746ff15b8d9650cf318f81d8416d7][ghc commit e2f0094c]].
- [[https://ypei.org/assets/haddorg-output/ghc-lib-parser-9.2.2.20220307.org.gz][ghc-lib-parser-9.2.2.20220307.org]], which can be used as a fake ghc
  for cross-package linking.
- [[https://ypei.org/assets/haddorg-output/haddorg-api-2.26.1.org.gz][haddorg-api-2.26.1.org]]
- [[https://ypei.org/assets/haddorg-output/lens-5.1.org.gz][lens-5.1.org]]
- [[https://ypei.org/assets/haddorg-output/fsd-sqlite-simple-debian.org.gz][fsd + sqlite-simple + debian]], demonstrating cross-package linking
- and [[https://ypei.org/assets/haddorg-output/][more...]], including [[https://ypei.org/assets/haddorg-output/hierarchy-e2f0094c/][all ghc libraries]] built at [[https://gitlab.haskell.org/ghc/ghc/-/commit/e2f0094c315746ff15b8d9650cf318f81d8416d7][ghc commit e2f0094c]].

** Install

#+begin_src sh
git clone https://g.ypei.me/haddock.git
cd haddock
cabal install
#+end_src

This will create a haddock binary under ~$HOME/.cabal/bin~ - make sure
it is in your PATH.

To make cabal use the haddock built with ~haddorg-api~, modify the
~haddock-ghc-x.y.z~ shell script, where ~x.y.z~ is the GHC version.

The file ~haddock-ghc-x.y.z~ is located in the same directory as your
~ghc-x.y.z~ binary.  The following command should print its path

#+begin_src sh
readlink -f `which ghc` | sed 's/\(.*\)ghc\(.*\)/\1haddock-ghc\2/'
#+end_src

For example, if you use ghcup and ghc-9.2.2, then the path should be
~$HOME/.ghcup/ghc/9.2.2/bin/haddock-ghc-9.2.2~.

Once you have located the correct ~haddock-ghc~ script, modify it by
updating the ~exedir~ to ~"$HOME/.cabal/bin"~:

#+begin_src sh
#!/bin/sh
exedir="$HOME/.cabal/bin" # <- update this line
exeprog="haddock"
# ...
#+end_src

** Usage

~haddorg-api~ adds an ~--org~ flag to the ~haddock~ command.

*** Direct invocation

#+begin_src sh
haddock Hello.hs --org
ls *.org # If success, the org file should be placed here
#+end_src

*** With cabal

Follow instructions in [[Install]] to tell cabal to use haddock built with
haddorg-api.  Using lens-5.1 as an example package, to fetch a package
from Hackage, one may use ~cabal unpack~:

#+begin_src sh
cabal unpack lens-5.1
#+end_src

Now cd into the package you want to build the org documentation for,
and run the commands:

#+begin_src sh
cd lens-5.1
cabal haddock --haddock-option=--org
# If success, the org file should be placed under dist-newstyle
ls ./dist-newstyle/build/*/*/lens-5.1/doc/html/lens/lens-5.1.org 
#+end_src

*** With Hadrian

In order to build documentation for GHC API or Haskell Hierarchical
Libraries, one may need to use the GHC build tool hadrian to invoke
Haddock.

It is tricky as the ghc repo tracks [[https://gitlab.haskell.org/ghc/haddock/][its own haddock]] as a submodule,
which is built by hadrian and invoked for the documentation.  Compared
to [[https://github.com/haskell/haddock][the official haddock repo]] this haddock can be ahead in some commits
as it has to build against the bleeding-edge GHC libraries (ghc api,
base, ghc-prim, ...) in ghc repo, and behind in some commits as the
official repo make changes to improve haddock itself.

The haddorg-api repo at <https://g.ypei.me/haddock.git> does have a
[[https://g.ypei.me/haddock.git/?h=ghc-gitlab-ghc-head][ghc-gitlab-ghc-head branch]] to track the ghc-head branch of ghc haddock
repo.  In fact, it was used to build ghc and haskell hierarchical
libraries org documentation so it is not impossible. But this branch
can go out of sync with the official ghc repo, and even if it stays in
sync all the time, one still needs to manually find out which commit
on the ghc haddock repo ghc-head branch corresponds to which commit on
the ghc-gitlab-ghc-head branch.

The following is instructions on how to run haddock with hadrian,
where the reader may need to figure out the merge part, if a suitable
commit on the [[https://g.ypei.me/haddock.git/?h=ghc-gitlab-ghc-head][ghc-gitlab-ghc-head branch]] cannot be found.

First, clone ghc with all its submodules.  For example, to perform a
shallow clone, do:

#+begin_src sh
git clone --depth 1 --recurse-submodules https://gitlab.haskell.org/ghc/ghc.git
#+end_src

Then, update the flags in the hadrian code, merge the org backend
related commits into the haddock submodule located at ~utils/haddock~,
and build the documentation as usual:

#+begin_src sh
cd ghc
# Edit ./hadrian/src/Settings/Builders/Haddock.hs by adding an --org flag.
# Merge org backend commits into ./utils/haddock, resolve any conflicts that
# may arise
./boot && ./configure
hadrian/build docs -j --flavour=Quick
# If success, the org files should be placed under ./_build/doc/html/libraries/
ls ./_build/doc/html/libraries/*/*.org
#+end_src

** Tips

- If you would like cross-package links to work, simply concatenate
  files.  For example, concat the org file of base with that of any
  library that say uses ~String~ in ~Prelude~ will cause ~String~
  links to navigate to the definition of ~String~.

** Coverage and areas for further work

Most features and most packages should work.  However, there are some
features to be completed.  Below is an incomplete list of missing
features, with rare occurrences in real world haskell documentations
marked by (RARE)

- Infix declarations (currently all infix decls are shown as prefix)
- Operator precendences
- Minimal Signatures for classes operations
- Correct linking due to distinction between top level identifiers and
  constructors
- (RARE) Data instance constructors
- (RARE) Linear types
- (RARE) Bundled patterns

Some (rare) problems due to mismatch of haddock markup and org mode:

- (RARE) Lack of distinction between inline and block elements in
  haddock markup
- (RARE) Table column and row spans

Some cosmetic issues:

- Relative heading depth for DocHeaders

*** Linking issue

One classical question for Org Mode users is: One org file per package
or one big org file containing all packages?  Currently haddorg-api
takes the latter approach.  It produces one org file per package, but
cross-package links are generated in a way so that they only work in
one big org file from concatenation of separate package org
files. This may become unwieldy, especially when the big org file
contains large libraries like base and ghc.  For links to work in the
alternative approach, the links have to be aware of org files of other
packages.

** Acknowledgement

Part of the code is adapted from [[https://github.com/lucasvreis/org-parser][org-parser]].

** License, copyright, contributing

Haddock is licensed under modified BSD (aka BSD-3-Clause), with
haddock-api and haddock-library licensed under the FreeBSD license
(aka BSD 2-Clause).  To contribute, see CONTRIBUTING.md.

The Org backend written by Yuchen Pei is under the GNU Affero General
Public License, version 3 or later.  See COPYING.agpl3 for the license
text.  To report issues or send patches regarding the org backend,
email Yuchen at <id@ypei.org>.