Lambda field capturing local variable .isSynthetic() returns false

Question

In answering this question about lambdas which capture local variables, I defined a simple lambda which captures a local variable, and showed that the lambda has a field with that variable's value. According to various sources (e.g. here, here), when a lambda captures a local variable, its value is stored in a "synthetic" field. This seems to be implied by the Java Virtual Machine Specification (§4.7.8), which says:

A class member that does not appear in the source code must be marked using a Synthetic attribute, or else it must have its ACC_SYNTHETIC flag set. The only exceptions to this requirement are compiler-generated methods which are not considered implementation artifacts, namely the instance initialization method representing a default constructor of the Java programming language (§2.9.1), the class or interface initialization method (§2.9.2), and the Enum.values() and Enum.valueOf() methods.

The lambda's field is not one of the defined exceptions, and the lambda's field is not declared in the source code, so by my understanding the field should be synthetic according to this rule.

The existence of the field can be easily demonstrated via reflection. However, when I check using the Field.isSynthetic method, it actually returns false. The documentation for this method says it:

Returns true if this field is a synthetic field; returns false otherwise.

I'm testing using JShell in Java 10.0.1:

> class A { static Runnable a(int x) { return () -> System.out.println(x); } }
|  created class A

> Runnable r = A.a(5);
r ==> A$$Lambda$15/1413653265@548e7350

> import java.lang.reflect.Field;

> Field[] fields = r.getClass().getDeclaredFields();
fields ==> Field[1] { private final int A$$Lambda$15/1413653265.arg$1 }

> fields[0].isSynthetic()
$5 ==> false

The same behaviour occurs outside of JShell:

import java.lang.reflect.Field;

public class LambdaTest {
    static Runnable a(int x) {
        return () -> System.out.println(x);
    }

    public static void main(String[] args) {
        Runnable r = a(5);
        Field[] fields = r.getClass().getDeclaredFields();
        boolean isSynthetic = fields[0].isSynthetic();
        System.out.println("isSynthetic == " + isSynthetic); // false
    }
}

What's the explanation for this discrepancy? Am I misinterpreting the JVMS, am I misinterpreting the Field.isSynthetic method documentation, are the spec and the docs using the word "synthetic" to mean different things, or is this a bug?

Answer 1

In general, your understanding about the synthetic nature of fields generated for captured variables is right.

When we use the following program

public class CheckSynthetic {
    public static void main(String[] args) {
        new CheckSynthetic().check(true);
    }
    private void check(boolean b) {
        print(getClass());
        print(new Runnable() { public void run() { check(!b); } }.getClass());
        print(((Runnable)() -> check(!b)).getClass());
    }
    private void print(Class<?> c) {
        System.out.println(c.getName()+", synthetic: "+c.isSynthetic());
        Stream.of(c.getDeclaredFields(),c.getDeclaredConstructors(),c.getDeclaredMethods())
            .flatMap(Arrays::stream)
            .forEach(m->System.out.println("\t"+m.getClass().getSimpleName()+' '+m.getName()
                                           +", synthetic: "+m.isSynthetic()));
    }
}

we get something like

CheckSynthetic, synthetic: false
    Constructor CheckSynthetic, synthetic: false
    Method main, synthetic: false
    Method check, synthetic: false
    Method print, synthetic: false
    Method lambda$print$1, synthetic: true
    Method lambda$check$0, synthetic: true
CheckSynthetic$1, synthetic: false
    Field val$b, synthetic: true
    Field this$0, synthetic: true
    Constructor CheckSynthetic$1, synthetic: false
    Method run, synthetic: false
CheckSynthetic$$Lambda$21/0x0000000840074440, synthetic: true
    Field arg$1, synthetic: false
    Field arg$2, synthetic: false
    Constructor CheckSynthetic$$Lambda$21/0x0000000840074440, synthetic: false
    Method run, synthetic: false
    Method get$Lambda, synthetic: false

Prior to JDK-11, you'll also find an entry like

    Method access$000, synthetic: true

in the outer class CheckSynthetic.

So for the anonymous inner class, the fields this$0 and val$b are marked as synthetic, as expected.

For the lambda expression, the entire class has been marked as synthetic, but none of its members.

One interpretation could be that marking a class as synthetic is already sufficient here. Considering JVMS §4.7.8:

A class member that does not appear in the source code must be marked using a Synthetic attribute, or else it must have its ACC_SYNTHETIC flag set.

we could say that when the class does not appear in source code, there is no source code that could be checked for the presence of member declarations.

But more important is that this specification applies to class files and while those of us interested in more details know that under the hood, the reference implementation of LambdaMetafactory will generate byte code in the class file format to create an anonymous class, this is an unspecified implementation detail.

As John Rose puts it:

VM anonymous classes are an implementation detail that is opaque to system components except for the lowest layers of the JDK runtime and the JVM itself. […] Ideally we should not make them visible at all, but sometimes it helps (e.g., with single stepping through BCs).

…

You can't rely on any of this meaning what you think it means, even if it appears to have a classfile structure.

So we shouldn’t reason about this class file structure and only focus on the visible behavior, which is the return value of Field.isSynthetic(). While it’s reasonable to assume that under the hood, this implementation will just report whether the bytecode had the flag or attribute, we have to focus on the bytecode independent contract of isSynthetic:

Returns:

true if and only if this field is a synthetic field as defined by the Java Language Specification.

Which brings us to JLS §13.1:

11. A construct emitted by a Java compiler must be marked as synthetic if it does not correspond to a construct declared explicitly or implicitly in source code, unless the emitted construct is a class initialization method (JVMS §2.9).

Not only is the possibility of a construct to be “declared … implicitly in source code” quiet fuzzy, the requirement to be marked as synthetic is limited to “a construct emitted by a Java compiler”. But the classes generated at runtime for lambda expressions are not emitted by a Java compiler, they are generated automatically by a bytecode factory. This is more than just quibbling, as the entire §13 is about Binary Compatibility, but the ephemeral classes generated within a single runtime are not subject to Binary Compatibility at all, as the current runtime is the only software which has to deal with them.

The requirements on the runtime class are specified in JLS §15.27.4:

The value of a lambda expression is a reference to an instance of a class with the following properties:

• The class implements the targeted functional interface type and, if the target type is an intersection type, every other interface type mentioned in the intersection.

• Where the lambda expression has type U, for each non-static member method m of U:

  If the function type of U has a subsignature of the signature of m, then the class declares a method that overrides m. The method's body has the effect of evaluating the lambda body, if it is an expression, or of executing the lambda body, if it is a block; if a result is expected, it is returned from the method.

  If the erasure of the type of a method being overridden differs in its signature from the erasure of the function type of U, then before evaluating or executing the lambda body, the method's body checks that each argument value is an instance of a subclass or subinterface of the erasure of the corresponding parameter type in the function type of U; if not, a ClassCastException is thrown.

• The class overrides no other methods of the targeted functional interface type or other interface types mentioned above, although it may override methods of the Object class.

So the specification does not cover many properties of the actual class and that’s intentional.

So when the result of Field.isSynthetic() is only determined by the Java Language Specification, but the class of the inspected field is off specification, the result is unspecified.

There’s room for interpretation whether, now that we can observe certain artifacts of a generated class, those artifacts should follow certain expectations regarding a similarity to ordinary classes, but there’s not enough information to discuss that. Most notably, there is not a single word in any of the cited specifications about why we have to mark constructs as synthetic and which consequences the presence or absence of the marker has.

Practical tests revealed that Java compilers, i.e. javac, treat synthetic members as nonexistent when trying to access them on source level, but that has not been specified anywhere. Further, this behavior is not relevant for a runtime generated class which is never seen by a Java compiler. In contrast, for an access via Reflection, the synthetic flag seems to have no effect at all.