Can I get the boundaries of a macro in their expansion site?

Question

I want to have the file locations and character positions of where a macroexpansion takes place in order to highlight the macro expansion in a GUI.

For this, I want to be able to refer to the current position of a macro where an expansion takes place from the macro itself.

For example, if I have the following code:

(defun mve ()
  (magic-macro :maybe :args))

I want to be able to expand it to something like

(defun mve ()
  (progn
    (macro-body-stuff)
    "This expansion took place at #P(myfile.lisp) between chars 16 and 40"))

If such functions existed, a minimal example macro could be something along the lines of

(defmacro maybe-macro (&rest r)
  `(progn
     (macro-body-stuff)
     ,(format nil "This expansion took place at ~S between chars ~D and ~D"
             (??:get-macroexpansion-pathname)
             (??:get-macroexpansion-char-start)
             (??:get-macroexpansion-char-end))))

I am also tagging it as reader-macro because I do no know really where should this take place.

Answer 1

You can't do this portably with a normal macro

"This expansion took place at #P(myfile.lisp) between chars 16 and 40"

In general, you won't be able to get that kind of stuff, because it's not available once the form has been read. E.g,. if you have a file with this content:

;; line 0
(some-form arg1)

and a file with this content:

;; line 0
;; line 1
;; line 2
(


some-form



arg1
                )

Conceptually, the compiler is going to be getting the same input. In principle, the reader first reads a form from the file, and then passes it to the compiler. In both cases, the compiler gets the form (some-form arg1). That's what a macro that you write is guaranteed to have access too. An individual implementation might actually make more available to the compiler, but it will be in an implementation-dependent way, and won't necessarily be exposed to you in a portable way.

There are some standard things that the file loader binds when loading a file that can help in providing some of this information, though. For instance, the load function binds special variables with the pathname and truename of the file:

*load-truename* is bound by load to hold the truename of the pathname of the file being loaded.

*load-pathname* is bound by load to hold a pathname that represents filespec merged against the defaults. That is, (pathname (merge-pathnames filespec)).

The implementation dependent extensions that would provide things like line and column numbers, if any, might be accessible in the same way.

But you can sometimes do this with a reader macro

You can't do this with a normal macro portably, since you don't portably have the mechanism to determine where in the file a form was read from. However, a reader macro invokes a function that gets called with the stream that forms are read from, and there are functions for investigating the position within a stream. For instance, here's a file:

(defparameter *begin* nil
  "File position before reading a form prefixed with #@.")

(defparameter *end* nil
  "File position after reading a form prefixed with #@.")

(eval-when (:compile-toplevel :load-toplevel :execute)
  (set-dispatch-macro-character
   #\# #\@
   (lambda (stream char infix-parameter)
     (declare (ignore char infix-parameter))
     (let ((begin (file-position stream))
           (form (read stream t nil t))
           (end (file-position stream)))
       `(let ((*begin* ,begin)
              (*end* ,end))
          ,form)))))

(defun foo ()
  #@(format nil "form began at ~a and ended at ~a."
            *begin* *end*))

Now, after we load it, we can call foo:

CL-USER> (load ".../reader-macro-for-position.lisp")
T
CL-USER> (foo)
"form began at 576 and ended at 650."

That's a bit brittle of course, because the reader macro could be invoked in a way where the the stream isn't one for which file-position makes a whole lot of sense, so you'd want to do some checking on that, and you still need a way to interpret those file positions in terms of line numbers and columns, but this is a good first shot, I think.