IBM RTC Python Create Workitem

Question

I am currently looking for a python script which can help in creating a workitem using the XML payload. I tried RTCClient but it is of not much help for future and hence I am looking for script via Requests library from Python

I tried cURL commands and I was able to create workitem in RTC but when I try to repeat the same via Python Requests, I don't get any luck in achieving it. Below is the snippet I am using to achieve the same. During my last GET, I get HTML error as "Javascript is either disabled or not available in your Browser". I believe my authentication is not working proper via Python whereas the same works fine with cURL

Can anyone help in correcting below syntax

RTCCookieURL = 'https://clmtest:9443/jazz/authenticated/identity'
RTCGetCookie = requests.get(RTCCookieURL, verify=False)
RTCCookies=RTCGetCookie.cookies
print(RTCCookies)
RTCAuthURL = 'https://clmtest:9443/jazz/authenticated/j_security_check'
RTCHeaders = {
'Accept': 'text/xml',
'Content-Type': 'application/x-oslc-cm-change-request+xml'
}
RTCAuth = requests.get(RTCAuthURL, auth=HTTPBasicAuth('uname','pwd'), verify=False,      allow_redirects=True)
print(RTCAuth.cookies)
RTCGetCatalog = requests.get('https://clmtest:9443/jazz/oslc/workitems/catalog', verify=False,     cookies=RTCAuth.cookies)
print(RTCGetCatalog.content)

Answer 1

I guess you're trying to replicate an example I've seen somewhere using Curl to login in two steps - a GET to collect cookies then a POST (because -d data is contained in the Curl command) to do form authentication, with explicit saving cookies on the GET and applying these cookies to the subsequent command(s).

You should use a requests session because it does all the cookie handling for you.

Reference material is here, see below heading FORM challenge https://jazz.net/wiki/bin/view/Main/NativeClientAuthentication. Properly handled, when making a request which requires login the response indicates this and tells you where to go to authenticate, which is a much better plan than simply hardcoding URLs as your code does and as does the simple example below.

Notes:

1. One point of care you should take - authentication has a timeout configured on the app server, and doing an explicit login will only work for subsequent requests as long as that timout hasn't expired, and then you'll get a challenge which if you ignore it you will start getting 403 responses. Basically, it's better in general to not use an explicit login but to simply always try to make the actual request you want to make and check the response headers (even when the request gets a 200 as for example the whoami before logging in, see code) to look for the need to authenticate, and then do the login and finally replay the original request which wouldn't have had any effect as authentication was needed. Take the approach of handling every request like this then authentication expiry is automatically handled by reauthenticating.
2. This code below uses hardcoded URLs so won't work if e.g. the jts context root changes from /jts. In a more robust implementation there are few hardcoded urls - obviously your code needs to know the application's URL e.g. https://myserver.com:9443/ccm1 and then I think only minimal harcoding is needed - such as j_security_check and rootservices - every other url should be found from the rootservices (the project catalog) or from content/headers in responses.
3. ccm and qm do authentication themselves, but rm delegates to jts for auth - the indicators for authrequired tell you where to go so you don't need to (shouldn't) hardcode the differences.
4. As noted in the reference material direct login as you attempt doesn't work on Tomcat, only Liberty. Also refer to the other note about replaying the original request for Tomcat.

This code below replicates what I can remember of the Curl login sequence, works for Form login to Liberty using the Liberty user registry and that's what I've tested it against. YMMV with other auth mechanisms and this definitely won't work for Jazz Authorization Server which does different redirect for the login.

import requests
import urllib3
    
# Disable the InsecureRequestWarning - pointless having warnings when
# accessing temporary servers which only ever have self-signed certs
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

SERVER = "https://clmtest:9443"

JTSCookieURL = SERVER+'/ccm/authenticated/identity'
JTSAuthURL = SERVER+'/ccm/j_security_check'
JTSWHOAMI = SERVER+"/ccm/whoami"
RootServices = SERVER+'/ccm/rootservices'

USERNAME='fred'
PWD = 'fred'

# get a session - use this for *every* request to the server
s = requests.session()

# when not authenticated, whoami returns 200 with HTML :-( - but it's a good test of authentication success
GetWhoAmI = s.get(JTSWHOAMI, verify=False)

print( f"{GetWhoAmI=}" )

# whoami when authenticated returns the user id - this is always on the jts    
# or when not authenticated returns html
if GetWhoAmI.content != SERVER+"/jts/users/"+USERNAME:
    print( "NOT AUTHENTICATED YET!!!")
else:
    raise Exception( "We are authenticated which should never happen before we login" )

#
# This works for (default) form authentication using Liberty user registry
#

# do a get to load up session with cookies
GetCookie = s.get(JTSCookieURL, verify=False)

print( f"{GetCookie=}" )

# do a post to authenticate
Auth = s.post(JTSAuthURL, data = { 'j_username': USERNAME, 'j_password': PWD}, verify=False,allow_redirects=True)

print(f"{Auth=}")

# now you can get protected resource - whoami 
GetWhoAmI = s.get(JTSWHOAMI, verify=False)

print( f"{GetWhoAmI=}" )

if GetWhoAmI.text != SERVER+"/jts/users/"+USERNAME:
#    print( "NOT AUTHENTICATED!!!")
    raise Exception( "NOT AUTHENTICATED" )
else:
    print( "HURRAY WE ARE AUTHENTICATED" )    

# now you can get protected resources, starting with rootservices

GetRootServices = s.get(RootServices, verify=False)

print( f"{GetRootServices=}" )

#print(GetRootServices.content)

Answer 2

The curl equivalent (for windows, for other OS just remove the rem/echo lines) for /rm is something like (for https://SERVER:9443 and admin user/password myid/mypassword):

rem GET to receive the JazzFormAuth cookie
rem important to use the -L option so the authentication redirects are followed
curl -k -L -c cookies.txt "https://SERVER:9443/jts/authenticated/identity" -v
echo ******************************************************************************************

rem POST to authenticate
rem important to use the -L option so the authentication redirects are followed
curl -k -L -b cookies.txt -c cookies.txt -d "" "https://SERVER:9443/jts/j_security_check?j_username=myid&j_password=mypassword" -v

rem GET a protected resource - if fails, returns 302 (because -L isn't specified on this request, if it fails then redirect to auth isn't followed)
curl -k -b cookies.txt -c cookies.txt "https://SERVER:9443/rm/oslc_rm/catalog" -v -H "OSLC-Core-Version: 2.0" -H "Accept: application/rdf+xml"

and then for further requests always use the -b cookies.txt -c cookies.txt options to provide the session authentication. NOTE the cookies will only be valid until the authentication timeout expires - might be six hours.

This example also works for ccm (Engineering Workflow Manager, EWM, formerly RTC) as well as DOORS Next and should work for Engineering Test Manager (ETM formerly known as RQM)