Why does "setf" not work when using "let"?

Question

The behavior of setf when combined with let is confusing to me. Here, setf isn't changing the second value of my list.

(defun to-temp-field (lst)
  (let ((old-value (nth 2 lst))
        (new-value 'hello))
    (progn
      (setf old-value new-value)
      lst)))

But if I do it without let, it changes ok:

(defun to-temp-field (lst)
  (let ((new-value 'hello))
    (progn
      (setf (nth 2 lst) new-value)
      lst)))

What's causing this behavior?

Answer 1

setf takes a place and a value, or a number of place/value pairs, and mutates the value of the place to the new value.

In the first code example old-value is a lexical variable bound within the let form. Lexical variables are places, so calling (setf old-value new-value) sets the value of old-value to new-value.

Function forms can also be places, and nth is one function that can specify a place. In the second code example calling (setf (nth 2 lst) new-value) sets the value of the place (nth 2 lst) to the value new-value.

So in the first example the place old-value, a lexical variable, is updated. But in the second example the form (nth 2 lst) is a place, and the object which this form evaluates to (an element of the list lst) is updated.

To be clear: a place is not a value, it is a form.

With (let ((old-value (nth 2 lst)) ;;...)) old-value is bound to the value to which the form (nth 2 lst) evaluates. But with (setf (nth 2 lst) new-value) the form (nth 2 lst) is interpreted as a place. (setf old-value new-value) works the same way: old-value is a form, and a place. Here the variable old-value has been let-bound to the value of the form (nth 2 lst), and it is this value which is replaced via assignment in the call to setf by mutating the binding of old-value. As a result the value of the place old-value has been changed.

A few things to note:

• The second example updates the third element of lst since lists are zero-indexed in Common Lisp.

• There is no need for the progn form inside of let since the body of a let form is an implicit progn.

• Attempting to modify a literal causes undefined behavior in Common Lisp, so you shouldn't call the second function with a quoted list like this: (to-temp-field '(1 2 3)) or with a variable bound to a list literal, but rather like this: (to-temp-field (list 1 2 3)) or with a variable bound to a list that has been otherwise constructed.

Answer 2

setf is a macro that, simply put, understands what is being set and calls the corresponding set based on that, so (setf old-value new value) simply sets the value, whereas in the second case it behaves as expected.

Answer 3

Common Lisp is one of the earliest languages to exhibit call-by-sharing, a paradigm that's now extremely common in dynamically-typed and higher-level programming languages. In a call-by-sharing language, values bound to a variable implicitly undergo a sort of shallow copy, taking a new version of the uppermost pointer or data but leaving any nested data identical.

In a Lisp, this distinction is actually easier to see. The top-level cons cell is copied, but any nested pointers are left alone.

(defun to-temp-field (lst)
  (let ((old-value (nth 2 lst))
        (new-value 'hello))
    (progn
      (setf old-value new-value)
      lst)))

Here, old-value is not a reference to a position in the list. It's a shallow copy of the value at position 2 of the list. As indicated in khachik's answer, setf is actually just a really complicated macro, and macros dispatch their behavior of the syntax they see. So what setf sees as its first argument is old-value. This is happening at compile time, so setf has no way to know that old-value came from a list, or from an nth call, so it just sets the local variable (using basically setq).

Now, consider your second example,

(defun to-temp-field (lst)
  (let ((new-value 'hello))
    (progn
      (setf (nth 2 lst) new-value)
      lst)))

Here, setf sees (nth 2 lst) as its first argument. This is not a local variable. This is actually a function-call-like expression. And as we can see in the hyperspec, nth actually has two entries: one for a normal function call and one for when it's used in setf. That's because, just like we can define normal functions as

(defun nth (n lst)
  ...)

We can separately define a setf accessor as

(defun (setf nth) (n lst)
  ...)

That's not pseudocode. defun actually accepts (setf whatever) as a valid declared function name, and when you use setf on a function call, the macro looks for a function with this name.

Answer 4

setf is a macro and does completely different things when the first argument is a symbol than some accessor for a list structure.

It's very easy to see that when you are supplying a symbol it does something completely different than if you supply the nth form:

> (get-setf-expansion '(setf (nth 2 lst) new-value))
(#:TEMP-2859 #:TEMP-2858 #:TEMP-2857) ;
(2 LST NEW-VALUE) ;
(#:NEW-2856) ;
(FUNCALL #'(SETF SYSTEM::%SETNTH) #:NEW-2856 #:TEMP-2859 #:TEMP-2858 #:TEMP-2857) ;
(SYSTEM::%SETNTH #:TEMP-2859 #:TEMP-2858 #:TEMP-2857)

>  (macroexpand-1 '(setf (nth 2 lst) new-value))
(SYSTEM::%SETNTH 2 LST NEW-VALUE) ;
T

Last line you see it calls internal function %setnth. This is implementation specific so SBCL does does something else. However; This is clearly fetching the place and doing the same as (rplaca (cddr lst) new-value) while when you cached the value from (nth 2 lst) to old-value:

$(get-setf-expansion '(setf old-value 99))
(#:TEMP-3005 #:TEMP-3004) ;
(OLD-VALUE 99) ;
(#:NEW-3003) ;
(FUNCALL #'(SETF SETQ) #:NEW-3003 #:TEMP-3005 #:TEMP-3004) ;
(SETQ #:TEMP-3005 #:TEMP-3004)

> (macroexpand-1 '(setf old-value 99))
(SETQ OLD-VALUE 99) ;
T

This does a setq so it will just rebind the local variable old-value to point to 99. You cannot cache old-place since the macro won't understand it anymore.

Answer 5

Your first snippet works like this:

(defun to-temp-field (l)         

L is a variable, supposed to have a list as its value

    (let ((old-value (nth 2 l))

OLD-VALUE is a variable and is initialized to the third element of the list L

          (new-value 'hello))

NEW-VALUE is a variable and is initialized to the symbol HELLO

        (setf old-value new-value)

the variable OLD-VALUE is set to the value of the variable NEW-VALUE -> HELLO

        l))

L is returned, unchanged.

The second snippet works like this:

(defun to-temp-field (l)

L is a variable, supposed to have a list as its value

    (let ((new-value 'hello))

NEW-VALUE is a variable and is initialized to the symbol HELLO

        (setf (nth 2 l) new-value)

SETF changes the place (NTH 2 L) to the value of the variable NEW-VALUE. This place denotes the third element of the list L. The list is changed, it now has the symbol HELLO as its third element.

        l))

L is returned, changed.