Getting 404 when accessing classes annotated with @Path in {war}/WEB-INF/classes in Jetty 9

Question

This is my set-up in Eclipse:

There are 2 projects....clinixwip11Appl and clinixwip11Jtty

The purpose of clinixwip11Jtty is to start an instance of Jetty...as follows....

public class JttyMain{

    public static void main(String[] args) throws Exception {
        
        int             iPort       =   0;
        ClassList       clssList    =   null;
        Server          jttySrvr    =   null;
        ServletHolder   srvtHldr    =   null;
        WebAppContext   wappCtxt    =   null;
        
        if (args.length != 1) {
            System.err.println("Usage: need a relative path to the war file to execute");
            System.exit(1);
        }
        System.setProperty("org.eclipse.jetty.util.log.class", "org.eclipse.jetty.util.log.StrErrLog");
        System.setProperty("org.eclipse.jetty.LEVEL", "INFO");
        
        iPort           =   Integer.parseInt(System.getenv().getOrDefault("PORT", "8080"));
        jttySrvr        =   new Server(iPort);
        System.out.println("jttySrvr created and listening at 8080");
        jttySrvr.addLifeCycleListener(new CustomLifeCycleListener());
        wappCtxt        =   new WebAppContext();
        wappCtxt.setContextPath("/");
        wappCtxt.setWar(args[0]);
        wappCtxt.setConfigurations(new Configuration[]  {
            new AnnotationConfiguration(),
            new WebInfConfiguration(),
         });
        wappCtxt.setAttribute   ("org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern", ".*/target/classes/|.*/classes/.");
        System.out.println("wappCtxt created");
        srvtHldr        =   wappCtxt.addServlet(DefaultServlet.class, "/");
        srvtHldr.setInitParameter("jersey.config.server.provider.packages", "com.applix.cliniX.serverApp");
        srvtHldr.setInitParameter("com.sun.jersey.spi.container.ContainerResponseFilters", "com.applix.cliniX.serverApp.filter.ResponseFilter");
        srvtHldr.setInitParameter("com.sun.jersey.spi.container.ContainerRequestFilters", "com.applix.cliniX.serverApp.filter.RequestFilter");
        System.out.println("srvtHldr created and initialized");
        clssList        =   ClassList.setServerDefault(jttySrvr);
        clssList.addBefore  (
            "org.eclipse.jetty.webapp.WebXmlConfiguration",
            "org.eclipse.jetty.annotations.AnnotationConfiguration"
        );
        jttySrvr.setHandler(wappCtxt);
        System.out.println("before starting jttySrvr");
        jttySrvr.start();
        System.out.println("after starting jttySrvr");
        jttySrvr.join();
        System.out.println("after joining with main thread");
    }
    
    public static class CustomLifeCycleListener implements LifeCycle.Listener   {
        @Override
        public void lifeCycleStarting(LifeCycle event)  {
            System.out.println("JTTYMAIN Starting at " + new Date(System.currentTimeMillis()) + " : " + event);
        }

        @Override
        public void lifeCycleStarted(LifeCycle event)   {
            System.out.println("JTTYMAIN Started: at " + new Date(System.currentTimeMillis()) + " : " + event);
        }

        @Override
        public void lifeCycleFailure(LifeCycle event, Throwable cause)  {
            System.out.println("JTTYMAIN Failure: at " + new Date(System.currentTimeMillis()) + " : " + event);
            cause.printStackTrace(System.out);
        }

        @Override
        public void lifeCycleStopping(LifeCycle event)  {
            System.out.println("JTTYMAIN Stopping: at " + new Date(System.currentTimeMillis()) + " : " + event);
        }

        @Override
        public void lifeCycleStopped(LifeCycle event)   {
            System.out.println("JTTYMAIN Stopped: at " + new Date(System.currentTimeMillis()) + " : " + event);
        }
    }
}

Because this is supposed to run on Google App Engine, there is app.yaml file in clinixwip11Appl/src/main/appengine:

runtime: java11
instance_class: F2
entrypoint: 'java -cp "*" com.applix.clinix.clinixwip11Jtty.JttyMain clinixwip11Appl.war'

When maven builds the 2 projects, there are the two outputs that are deployed to Google App Engine:

(1) clinixwip11Jtty.jar (contains JttyMain.class) (2) clinixwip11Appl.war (contains my appl code)....as follows:

After deploying the code, I can see that the Jetty instances starts and I am successfully able to access /index.html and /main.html which are located at the root of the clinixwip11Appl.war.

However, when any of the classes inside the classes folder of the clinixwip11Appl.war are accessed, I get a 404.

The classes folder contains many compiled classes.....I have reproduced the relevant code from one of the classes below....

@Path("/estbStaffCard")
public  class   EstbStaffCardController {
    
@Context 
    private         HttpServletRequest  request             =   null;
    private static  Logger              logger              =   Logger.getLogger("EstbStaffCardController");
        
    @GET
    @Path("/getEstbForEstbStaffIden")
    @Consumes(MediaType.TEXT_PLAIN)
    @Produces(MediaType.TEXT_PLAIN)
    public  synchronized    Response    getEstbForEstbStaffIden (@QueryParam("queryString") String paramQueryString)    {

        logger.info("//////////////////////////////////////////////////////////////////////////////////////");
        logger.info("begin new method: " + new Object(){}.getClass().getEnclosingMethod().getName());
    .......
        }
    }

So when {URL to app engine}/estbStaffCard/getEstbForEstbStaffIden... is accessed, I get a 404.

I have tried various options of specifying the configuration options in the JttyMain class as specified in numerous threads across the web, especially on stackoverflow, but to no avail.

Any idea exactly what configuration needs to be specified in the JttyMain class above so that the classes inside the classes folder of the clinixwip11Appl.war are honored? I have been at this for the last 3 days. Please help.

At this location inside the war...

the web.xml is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="4.0" xmlns="http://xmlns.jcp.org/xml/ns/javaee"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee 
   http://xmlns.jcp.org/xml/ns/javaee/web-app_4_0.xsd">
<!--  

    <init-param>
        <param-name>org.glassfish.jersey.server.ServerProperties.DisableWADL</param-name>
        <param-value>true</param-value>
    </init-param>
    
    <httpProtocol>
        <customHeaders>
            <add name="X-Frame-Options" value="*" />
        </customHeaders>
    </httpProtocol>
    
--> 
    <servlet>
        <servlet-name>Jersey Web Application</servlet-name>
        <servlet-class>
            org.glassfish.jersey.servlet.ServletContainer
        </servlet-class>
        <init-param>
            <param-name>jersey.config.server.provider.packages</param-name>
            <param-value>com.applix.cliniX.serverApp</param-value>
        </init-param>
        <init-param>
            <param-name>com.sun.jersey.spi.container.ContainerRequestFilters</param-name>
            <param-value>com.applix.cliniX.serverApp.filter.RequestFilter</param-value>
        </init-param>
        <init-param>
            <param-name>com.sun.jersey.spi.container.ContainerResponseFilters</param-name>
            <param-value>com.applix.cliniX.serverApp.filter.ResponseFilter</param-value>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet>
        <servlet-name>_ah_sessioncleanup</servlet-name>
        <servlet-class>com.google.apphosting.utils.servlet.SessionCleanupServlet</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>Jersey Web Application</servlet-name>
        <url-pattern>/*</url-pattern>
    </servlet-mapping>
    <servlet-mapping>
        <servlet-name>_ah_sessioncleanup</servlet-name>
        <url-pattern>/_ah/sessioncleanup</url-pattern>
    </servlet-mapping>
  
    <security-constraint>
        <web-resource-collection>
            <web-resource-name>*</web-resource-name>
            <url-pattern>/*</url-pattern>
        </web-resource-collection>
        <user-data-constraint>
            <transport-guarantee>CONFIDENTIAL</transport-guarantee>
        </user-data-constraint>
    </security-constraint>
    <listener>
        <listener-class>com.applix.cliniX.serverApp.listener.SessionListener</listener-class>
    </listener>
</web-app>

I know that I am not doing something critical that I need to do, or, overdoing something in this section of the JttyMain code:

wappCtxt.setConfigurations(new Configuration[]  {
            new AnnotationConfiguration(),
            new WebInfConfiguration(),
         });
        wappCtxt.setAttribute   ("org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern", ".*/target/classes/|.*/classes/.");
        System.out.println("wappCtxt created");
        srvtHldr        =   wappCtxt.addServlet(DefaultServlet.class, "/");
        srvtHldr.setInitParameter("jersey.config.server.provider.packages", "com.applix.cliniX.serverApp");
        srvtHldr.setInitParameter("com.sun.jersey.spi.container.ContainerResponseFilters", "com.applix.cliniX.serverApp.filter.ResponseFilter");
        srvtHldr.setInitParameter("com.sun.jersey.spi.container.ContainerRequestFilters", "com.applix.cliniX.serverApp.filter.RequestFilter");
        System.out.println("srvtHldr created and initialized");
        clssList        =   ClassList.setServerDefault(jttySrvr);
        clssList.addBefore  (
            "org.eclipse.jetty.webapp.WebXmlConfiguration",
            "org.eclipse.jetty.annotations.AnnotationConfiguration"
        );

Any help will be highly appreciated.

Found Which Jar Files Are Scanned For Discovered Annotations here. So now I am wondering where it is that I am going wrong? Any hint, clue, help?

Update: It appears that Jetty is working off of only one classpath, name the root of the war file because the index.html and main.html and all other resources located therein....css/, images/, html/ etc are being loaded. Only WEB-INF/classes are not being honored. So the problem boils down to letting Jetty know that WEB-INF/classes and WEB-INF/lib also need to considered as part of classpath.

The question is.....how to let let Jetty know that??

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ADDED THE FOLLOWING ON TUE30JAN24 AFTER EXPERIMENTATION +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

jerseyVersion	jettyVersion	jettyServlet	jettyWebApp	java	Results
3.1.5	12.0.5	11.0.19	11.0.19	11	The type jakarta.servlet.Servlet cannot be resolved. It is indirectly referenced from required .class files
3.1.5	10.0.19	10.0.19	10.0.19	11	The type jakarta.servlet.Servlet cannot be resolved. It is indirectly referenced from required .class files
2.41	10.0.19	10.0.19	10.0.19	11	Exception in thread "main" javax.servlet.ServletException: at com.rest.test.App.main ...where the code is jettyServer.start();
2.41	10.0.19	10.0.19	10.0.19	8	Exception in thread "main" javax.servlet.ServletException Caused by: java.lang.IllegalStateException: InjectionManagerFactory not found.
2.25	10.0.19	10.0.19	10.0.19	8	All good, except.......see below....

My goal: Upgrade from Java8 to Java11 as mandated by Google App Engine

The issue: With Java8, Google App Engine used to provide a pre-built implementation of a RESTful server

With Java11, Google App Engine will NOT provide such a server and it will the developer's responsibility to build provide such a server

My conclusion (regarding maven and version stuff) after the above experimentations.... (1) For the aforementioned goal, I will be able to use a Java11 compiler but I will have to set the target to Java8 (2) I will have to use the 2.25 version of the jersey server and 10.0.19 versions of the other artefacts

But there is more.... My app has many services that are accessed by the front-end using REST. My app also has many static resources. So, in order to serve both these resources, if I use the following set-up,

HandlerCollection handlerCollection = new HandlerCollection();
ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
context.setContextPath("/");
handlerCollection.addHandler(context);
        
jettyServer.setHandler(handlerCollection);
        
ServletHolder jerseyServlet = context.addServlet(org.glassfish.jersey.servlet.ServletContainer.class, "/*");
jerseyServlet.setInitOrder(0);
        jerseyServlet.setInitParameter("jersey.config.server.provider.packages", "com.rest.test");
ServletHolder staticHolder = new ServletHolder(new DefaultServlet());
staticHolder.setInitParameter("pathInfoOnly", "true");
URL webAppDir = App.class.getClassLoader().getResource("META-INF/resources");
staticHolder.setInitParameter("resourceBase", webAppDir.toURI().toString());
context.addServlet(staticHolder, "/*");        

jettyServer.start();

I get this error upon server start-up....

Exception in thread "main" java.lang.IllegalStateException: Multiple servlets map to path /*

    

But if I use the following set-up.....

HandlerCollection handlerCollection = new HandlerCollection();
ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
context.setContextPath("/");
handlerCollection.addHandler(context);
jettyServer.setHandler(handlerCollection);
ServletHolder jerseyServlet = context.addServlet(org.glassfish.jersey.servlet.ServletContainer.class, "/");
jerseyServlet.setInitOrder(0);
jerseyServlet.setInitParameter("jersey.config.server.provider.packages", "com.rest.test");
ServletHolder staticHolder = new ServletHolder(new DefaultServlet());
staticHolder.setInitParameter("pathInfoOnly", "true");
URL webAppDir = App.class.getClassLoader().getResource("META-INF/resources");
staticHolder.setInitParameter("resourceBase", webAppDir.toURI().toString());
context.addServlet(staticHolder, "/*");       

I can only access static content but not the REST content

And if I use the following set-up.....

HandlerCollection handlerCollection = new HandlerCollection();
ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
context.setContextPath("/");
handlerCollection.addHandler(context);
jettyServer.setHandler(handlerCollection);
ServletHolder jerseyServlet = context.addServlet(org.glassfish.jersey.servlet.ServletContainer.class, "/rest/*");
jerseyServlet.setInitOrder(0);
jerseyServlet.setInitParameter("jersey.config.server.provider.packages", "com.rest.test");
ServletHolder staticHolder = new ServletHolder(new DefaultServlet());
staticHolder.setInitParameter("pathInfoOnly", "true");
URL webAppDir = App.class.getClassLoader().getResource("META-INF/resources");
staticHolder.setInitParameter("resourceBase", webAppDir.toURI().toString());
context.addServlet(staticHolder, "/*");


**I can access static content and also the REST content**

My question (regarding serving static and REST content) after the above experimentations.... will I now have to go and change my source code (basically prefix every REST resource with /rest) for the above to work? I know for a fact that with Java8 and the default REST implementation provided by GAE, I was able to serve static and REST resources at /

Answer 1

Your org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern setting is only valid for development / testing / IDE time. Get rid of it for production.

Your Main class isn't doing anything special. Why have it?

The WAR file alone has everything it needs in it's WEB-INF/web.xml.

I would just skip the custom entry point, skip the main class, skip the entrypoint jar, and just deploy that WAR file to GAE.

Now, about DefaultServlet and resource bases.

Note: Jersey can serve static resources itself, so take care and understand what is serving your static resources (Jetty or Jersey). If it is Jersey, then you need to consult their documentation on how their lookups work. I can only comment about Jetty.

The ServletContextHandler (and the WebAppContext) has a Resource Base as well, use that, it is what the default servlet uses.

Your extra DefaultServlet is not replacing the standard DefaultServlet, which is still there on the name "default" and the mapping of "/".

The idea and concept of META-INF/resources is already present in the startup of a WebAppContext via the MetaInfConfiguration behaviors.

If you use a ServletContextHandler then you'll need to do all of the extra steps necessary to support your META-INF/resources.

Such as ...

1. Iterate through every JAR you can see from your ServletContextHandler. (this means JAR files on filesystem, and nested JAR files within other archives, like WAR files and their WEB-INF/lib tree)
2. Track every JAR that has a META-INF/resources (including those that use MultiRelease JAR files that have them only in their META-INF/versions/#/META-INF/resources locations).
3. Iterate through the list of JARs and ensure that they only exist as a File system JAR (meaning you have to copy nested JARs from places like file:///path/to/myapp.war!/WEB-INF/lib/foo.jar!/META-INF/resources to a temporary or work directory)
4. Create a Composite / Combined / Collection Resource referencing the URL locations of every relevant META-INF/resources from JAR files that exist on a file system path (lets call this "MetaInfResourcesResource")
5. Merge this new MetaInfResourcesResource with any existing Base Resource(s) from your context.
6. Set this new merged resource as your ServletContextHandler.setBaseResource(Resource)
7. Add the normal and to-spec DefaultServlet (as the behavior for ServletContextHandler is that if you don't add one and the context is started, then the a Default404Servlet is added to satisfy the spec resulting in all static resources being a 404)

The to-spec DefaultServlet setup ...

ServletContextHandler context = ...

ServletHolder defaultHolder = new ServletHolder("default", DefaultServlet.class);
context.addServlet(defaultHolder, "/"); // this MUST be "/", not "/*"

Or...
save yourself some time, and hassles now, and the inevitable bugs in your future.

Setup your build so that it flattens ALL of your META-INF/resources into a single place, and don't use them (meaning eliminate them from your JARs and your dependencies JARs).

The number of people that have weird bugs around META-INF/resources is growing every year, and it usually comes down to conflicts in naming across the META-INF/resources and the fact that the classloader does not have a reliable "load order" during resolution of conflicting resources.