I have used ZAP Desktop using form based authentication, zap runs perfectly fine on Desktop app. However as the web application i am using also has _csrf_token is passed along with username and Password I chose to automate it with manual authentication using selenium.
Below is the error that i am getting -
1112496 [ZAP-ProxyThread-473] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1112601 [ZAP-ProxyThread-481] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1112602 [ZAP-ProxyThread-481] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1112624 [ZAP-ProxyThread-470] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1112624 [ZAP-ProxyThread-470] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1112648 [ZAP-ProxyThread-482] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1112648 [ZAP-ProxyThread-482] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1117079 [ZAP-ProxyThread-488] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1117080 [ZAP-ProxyThread-488] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1117082 [ZAP-ProxyThread-485] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1117088 [ZAP-ProxyThread-485] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1119534 [ZAP-ProxyThread-489] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1119535 [ZAP-ProxyThread-489] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1120768 [ZAP-ProxyThread-490] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1120768 [ZAP-ProxyThread-490] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1120770 [ZAP-ProxyThread-491] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1120770 [ZAP-ProxyThread-491] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1124677 [ZAP-ProxyThread-500] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1124682 [ZAP-SpiderInitThread-4] INFO org.zaproxy.zap.extension.spider.SpiderThread - Starting spidering scan on https://****/web at 2021-10-21T19:12:37.019+0530
1124682 [ZAP-SpiderInitThread-4] INFO org.zaproxy.zap.spider.Spider - Spider initializing...
1124707 [ZAP-SpiderInitThread-4] INFO org.zaproxy.zap.spider.Spider - Starting spider...
1124709 [ZAP-SpiderInitThread-4] INFO org.zaproxy.zap.spider.Spider - Scan will be performed from the point of view of User: abc
1124714 [ZAP-SpiderThreadPool-4-thread-1] INFO org.zaproxy.zap.users.User - Authenticating user: abc
1124715 [ZAP-SpiderThreadPool-4-thread-1] INFO org.zaproxy.zap.users.User - Authentication failed for user: abc
1125460 [ZAP-SpiderThreadPool-4-thread-1] INFO org.zaproxy.zap.spider.Spider - Spidering process is complete. Shutting down...
1125462 [ZAP-SpiderShutdownThread-4] INFO org.zaproxy.zap.extension.spider.SpiderThread - Spider scanning complete: true on https://****/web at 2021-10-21T19:12:37.799+0530
My code looks like below -
@Test
public void zapScanOnTest(String jsonContextsFileName, String testCaseFile, String scanType, String scanPolicyName) throws InterruptedException {
SoftAssert softassert = new SoftAssert();
boolean login = loginPage.login(UsrName, Pwd);
softassert.assertTrue(login, "zapScanTest : Logged in");
Thread.sleep(10000);
runScanAsUserOnURLs(jsonContextsFileName,"abc_"+HostIP, "after_login_url", UsrName,scanType,scanPolicyName);
}
public void runScanAsUserOnURLs(String jsonContextsFileName, String zapContextName, String nodeName,
String UserName, String scanType, String scanPolicyName){
List<ApiResponse> listOfContext;
try {
listOfContext = ((ApiResponseList) clientApi.context.contextList()).getItems();
String contextID = setAndGetContextID(listOfContext, zapContextName);
log.info("Checking if the User already exists in Context if not Add the user to the context");
String userID = setAndGetUserID(UserName, contextID);
log.info("Fetching Json file path and reading all the URL's mentioned in JSON File");
List<String> urlLists = readJsonFileConvertUrlsToList(jsonContextsFileName, nodeName);
includeAllURLSToContext(urlLists, contextID, zapContextName);
spiderCrawlScanAsUser(contextID,userID,urlLists);
}
catch (InterruptedException e) {
e.printStackTrace();
}
catch (ClientApiException e) {
e.printStackTrace();
}
}
public String setAndGetContextID(List<ApiResponse> listOfContext, String contextName) throws ClientApiException {
String contextID = null;
if (listOfContext.isEmpty() || isContextPresent(listOfContext, contextName) == false) {
ApiResponse newContext = clientApi.context.newContext(contextName);
contextID = newContext.toString();
log.info("Context is Created and the ID is : " + contextID);
} else {
Context context = new Context((ApiResponseSet) clientApi.context.context(contextName));
contextID = context.getId();
log.info("ID of existing Context is : " + contextID);
}
listOfContext = ((ApiResponseList) clientApi.context.contextList()).getItems();
return contextID;
}
public boolean isContextPresent(List<ApiResponse> listOfContext, String contextName) {
boolean isPresent = false;
String str = "Context is not available in list : " + listOfContext + " let's create a new context";
log.info("Checking if provided context name " + contextName + " is already present in list of context");
for (int i = 0; i < listOfContext.size(); i++) {
String zapContext = listOfContext.get(i).toString();
if (zapContext.equals(contextName)) {
isPresent = true;
str = "Context Name Already Exists : No need to create a Context again : " + listOfContext;
}
}
log.info(str);
return isPresent;
}
public String setAndGetUserID(String mcUser, String contextID) throws ClientApiException {
String userID = null;
List<ApiResponse> usersListInContext = ((ApiResponseList) clientApi.users.usersList(contextID)).getItems();
if (usersListInContext.isEmpty() || isUserPresentInContext(usersListInContext, mcUser, contextID) == false) {
userID = clientApi.users.newUser(contextID, mcUser).toString();
log.info("User is added to the Context and the user ID is : " + userID);
log.info("Enabling the User");
ApiResponse setUserEnabled = clientApi.users.setUserEnabled(contextID, userID, "true");
log.info("User is Enabled and the status is : " + setUserEnabled);
log.info("Setting Forced User");
ApiResponse setForcedUser = clientApi.forcedUser.setForcedUser(contextID, userID);
log.info("User is set as Forced User and the status is : " + setForcedUser);
log.info("Enabling Forced User Mode");
ApiResponse setForcedUserModeEnabled = clientApi.forcedUser.setForcedUserModeEnabled(true);
log.info("Enabled Forced User Mode and the status is : " + setForcedUserModeEnabled);
} else {
for (ApiResponse userListResponse : usersListInContext) {
String userList = userListResponse.toString(0);
boolean userPresentInContextList = userList.contains("name = " + mcUser);
boolean contextIDPresent = userList.contains("contextId = " + contextID);
if (userPresentInContextList == true && contextIDPresent == true) {
userID = userList.substring(userList.indexOf("id = ") + 5, userList.indexOf("enabled"));
log.info("User ID is : " + userID);
log.info("Enabling the User");
ApiResponse setUserEnabled = clientApi.users.setUserEnabled(contextID, userID, "true");
log.info("User is Enabled and the status is : " + setUserEnabled);
log.info("Setting Forced User");
ApiResponse setForcedUser = clientApi.forcedUser.setForcedUser(contextID, userID);
log.info("User is set as Forced User and the status is : " + setForcedUser);
log.info("Enabling Forced User Mode");
ApiResponse setForcedUserModeEnabled = clientApi.forcedUser.setForcedUserModeEnabled(true);
log.info("Enabled Forced User Mode and the status is : " + setForcedUserModeEnabled);
break;
}
}
}
return userID;
}
public boolean isUserPresentInContext(List<ApiResponse> usersListInContext, String mcUser, String contextID) {
boolean isPresent = false;
String str = "User is not available in Context List let's add the user";
log.info("Checking if provided User name " + mcUser + " is already present in list of context");
for (ApiResponse userListResponse : usersListInContext) {
String userList = userListResponse.toString(0);
boolean userPresentInContextList = userList.contains("name = " + mcUser);
boolean contextIDPresent = userList.contains("contextId = " + contextID);
if (userPresentInContextList == true && contextIDPresent == true) {
isPresent = true;
str = "User is already added to the context, no need to add the user again";
}
}
log.info(str);
return isPresent;
}
public List<String> readJsonFileConvertUrlsToList(String jsonContextsFileName, String nodeName) {
String filePath = readJsonFile.getJsonFilePath(jsonContextsFileName);
log.info("File Path is : " + filePath);
FileInputStream fis;
List<String> urlList = new ArrayList<>();
try {
fis = new FileInputStream(filePath);
JSONTokener tokener = new JSONTokener(fis);
JSONObject jsonObject = new JSONObject(tokener);
JSONArray contextJsonArray = jsonObject.getJSONArray("contexts");
for (int i = 0; i < contextJsonArray.length(); i++) {
JSONObject testJsonObject = contextJsonArray.getJSONObject(i);
JSONArray urlJsonArray = testJsonObject.getJSONArray(nodeName);
log.info("Running ZAP Scan on " + urlJsonArray.length() + " URL's");
for (int j = 0; j < urlJsonArray.length(); j++) {
String urlEndPoints = urlJsonArray.get(j).toString();
urlList.add(mcHostUrl + urlEndPoints);
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
}
return urlList;
}
public void includeAllURLSToContext(List<String> listOfURL, String contextID, String contextName) {
try {
log.info("Going to include URLs to context : " + contextName);
List<ApiResponse> includeContextRegex = ((ApiResponseList) clientApi.context.includeRegexs(contextName)).getItems();
for (int i = 0 ; i< listOfURL.size();i++) {
String zapTargetURL = listOfURL.get(i);
if (includeContextRegex.isEmpty() || isContextRegexPresent(includeContextRegex, zapTargetURL) == false)
clientApi.context.includeInContext(contextName, zapTargetURL);
log.info("Included Context Regex to Context : " + contextName);
}
} catch (ClientApiException e) {
e.printStackTrace();
}
}
public void spiderCrawlScanAsUser(String contextID, String userID,List<String> urlList)
throws InterruptedException, ClientApiException {
for (int i = 0 ; i< urlList.size();i++) {
String zapTargetURL = urlList.get(i);
log.info("PREPARING FOR SPIDER CRAWL ON TARGET HOST :" + zapTargetURL);
log.info("Starting Spider Scan");
ApiResponse apiResponse = clientApi.spider.scanAsUser(contextID, userID, zapTargetURL, "500",
"true", "true");
int progress;
String scanId = ((ApiResponseElement) apiResponse).getValue();
do {
Thread.sleep(5000);
progress = Integer.parseInt(((ApiResponseElement) clientApi.spider.status(scanId)).getValue());
log.info("Scan progress: {}{}", progress, "%");
} while (progress < 100);
log.info("Spider scan completed");
List<ApiResponse> spiderResults = ((ApiResponseList) clientApi.spider.results(scanId)).getItems();
log.info("spider results {}", spiderResults);
}
}
Am I missing anything in the above code? I am not able to authenticate the user at all.
To be honest I wouldnt recommend configuring ZAP that way. I recommend testing everything in the desktop, making sure its all working and then exporting the context - you can then import that via the API. I also notices that you are using Forced User Mode - thats really there for manual testing so I wouldnt use that for automation either. Instead specify the user when you run the spiders and active scanner.
We will bee adding more authentication docs soon and enhancing the Automation Framework to support authentication - those 2 things should make things easier.