Stacktraces are a hot topic in the Clojure community. As a Clojurist you deal with them in different situations. Sometimes you catch them “live”, like an exception just thrown in a REPL. Other times you find them as text, printed in a REPL, or in a log file. Or worst, a printed exception buried inside another string, almost impossible to read. And of course, there are different kinds of formats.
Haystack is a library that can parse and analyze Clojure stacktraces. The parser transforms printed stacktraces back into data and the analyzer enriches stacktrace data with run-time information from the class path.
Haystack was previously used in CIDER for stacktrace analysis. It is not included in CIDER anymore but can still be used as an individual library.
The Haystack stacktrace parser transforms a string that contains a
stacktrace printed in one of the supported formats back into a Clojure
data structure. Given an input, the parser applies some
transformations to it (unwrapping an EDN string for example) and
passes the result to the parser functions registered in the
haystack.parser/default-parsers var. Each of the registered parsers
is tried in order and the first parser that succeeds wins.
On success the parser returns a Clojure map with a similar structure
as Clojure’s Throwable->map function.
On failure the parser returns a map with an :error key, and possibly
other keys describing the error.
A successful parse result can be given to the Haystack analyzer to enrich it with more information.
An Haystack stacktrace parser transforms input into a parse result. On success, the parse result is a enhanced version of the Clojure data representation of a Throwable, a map with the following keys:
:causeThe root cause message as a string.:phaseThe error phase (optional).:viaThe cause chain, with each cause having the keys::atThe top stack element of the cause as a vector (optional).:dataTheex-dataof the cause as a map (optional).:messageThe exception message of the cause as a string.:typeThe exception of the cause as a symbol.:traceThe stack elements (optional, extended by Haystack).
:traceThe root cause stack elements
This is mostly the same format as used by Throwable->map in newer
Clojure versions, except for the additional :trace key in the cause
maps of :via. We added this additional key to keep the trace of the
causes.
Stacktraces are printed in different formats by tools and libraries. Haystack supports the following formats:
:avisoStacktraces printed with the write-exception function of the Aviso library.:clojure.tagged-literalStacktraces printed as a tagged literal, like a java.lang.Throwable printed with the pr function.:clojure.stacktraceStacktraces printed with the print-cause-trace function of the clojure.stacktrace namespace.:clojure.replStacktraces printed with the pst function of the clojure.repl namespace.:javaStacktraces printed with the printStackTrace method of the java.lang.Throwable class.
Let’s say you want to parse the following stacktrace string and turn it back into a data structure for further processing.
(def my-stacktrace-str
(str "clojure.lang.ExceptionInfo: BOOM-1 {:boom \"1\"}\n"
" at java.base/java.lang.Thread.run(Thread.java:829)"))The easiest way to do this is to pass the string to the
haystack.parser/parse function. It will try all registered
parsers and returns the first successful parse result.
(require '[haystack.parser :as stacktrace.parser])
(def my-stacktrace-data
(stacktrace.parser/parse my-stacktrace-str))On success the parser will return a Clojure map in the
Throwable->map format. For the input used above, this data structure
looks like this:
(clojure.pprint/pprint my-stacktrace-data){:cause "BOOM-1",
:data {:boom "1"},
:trace [[java.base/java.lang.Thread run "Thread.java" 829]],
:via
[{:at [java.base/java.lang.Thread run "Thread.java" 829],
:message "BOOM-1",
:type clojure.lang.ExceptionInfo,
:trace [[java.base/java.lang.Thread run "Thread.java" 829]],
:data {:boom "1"}}],
:stacktrace-type :java}Tip: If you know in advance with what kind of stacktrace you are dealing with, pass it directly to the parser for the given format.
The Haystack stacktrace analyzer transforms a stacktrace into an analysis. An analysis is a sequence of Clojure maps, one for each of the causes of the stacktrace, with the following keys:
:classThe exception class as a string.:messageThe exception message as a string.:stacktraceThe stacktrace frames, a list of maps.:dataThe exception data.:locationThe location formation of the exception.
A frame in the :stacktrace is a map with the following keys:
:classThe class name of the frame invocation.:file-urlThe URL of the frame source file.:fileThe file name of the frame source.:flagsThe flags of the frame.:lineThe line number of the frame source.:methodThe method or function name of the frame invocation.:nameThe name of the frame, typically the class and method of the invocation.:typeThe type of invocation (:java,:tooling, etc).
The analyzer accepts either an instance of java.lang.Throwable or a
Clojure map in the Throwable->map format as input.
We can analyze our previously parsed stacktrace by calling the
haystack.analyzer/analyze function on it.
(require '[haystack.analyzer :as stacktrace.analyzer])
(stacktrace.analyzer/analyze my-stacktrace-data)[{:class "clojure.lang.ExceptionInfo",
:message "BOOM-1",
:stacktrace
({:name "java.lang.Thread/run",
:file "Thread.java",
:line 829,
:class "java.lang.Thread",
:method "run",
:type :java,
:flags #{:java},
:file-url
"jar:file:/usr/lib/jvm/openjdk-11/lib/src.zip!/java.base/java/lang/Thread.java"}),
:data "{:boom \"1\"}",
:location {}}]We get back a sequence of maps, one for each cause, which contain additional information about each frame discovered from the class path.
To add support for another stacktrace format, please create a new
parser under the haystack.parser.<NEW-FORMAT> namespace and add it
to the haystack.parser/default-parsers var. The parser should be a
function that accepts a single argument, the input (typically a
string), and returns a map. The parser function should follow the
following rules:
- On success, the parser should return the stacktrace as a map. The
map should be in the
Throwable->mapformat described above with a:stacktrace-typekey that contains the type of stacktrace as a keyword. - On error, the parser should return a map with an
:errorkey and possibly others describing why the input could not be parsed. We use:incorrectif the input does not match the grammar, and:unsupportedif the input type is not supported by the parser. - Ideally, the parser should be tolerant to any garbage before and after the stacktrace to be parsed. This is to not put the burden of exactly figuring out where a stacktrace starts and ends onto clients.
- When skipping garbage at the beginning of a stacktrace do it
efficiently. For example, instead of skipping garbage character by
character and trying your parser with the rest of the string, use
the
haystack.parser.util/seek-to-regexfunction to directly skip to the beginning of the stacktrace, if possible. - Most of the parsers in Haystack are implemented with Instaparse and have a BNF grammar describing the format of the stacktrace. Try to come up with an Instagram grammar for the new stacktrace format as well, unless you have a better, simpler or more efficient way of parsing it (like the Clojure tagged literal parser for example).
Writing a grammar for a stacktrace format might be challenging at times, especially when garbage in the input is involved, which might introduce ambiguities in your grammar. Here are some tips and trick for writing Instaparse grammars:
- Read the documentation, it is good and has many examples.
- Start with the most simple parser, try to pass the exception class or name before building up.
- Use the
:startparameter of the Instaparse parser, to parse input from another start rule. This is useful if your grammar got complex, but you want to try parsing of an individual rule. - Be aware of greedy regex behavior.
- When testing input try it against the raw Instaparse parser first, and only apply the Instaparse transformations when the parser works.
- If your parser fails on an input, reveal hidden information to get a better understanding of what happened.
Here’s how to deploy to Clojars:
git tag -a v0.3.3 -m "0.3.3"
git push --tagsThe Haystack stacktrace analyzer was written by Jeff Valk (@jeffvalk) and was originally part of the cider-nrepl project.
Copyright © 2022-23 Cider Contributors
Distributed under the Eclipse Public License, the same as Clojure.