How to force falcon 40B to print in JSON format?

Question

I have been trying to extract start time and end time from the input text using Falcon 40B. This was my prompt,

Identify the following items from the given text which states random shipping details: 
 start_time 
 end_time
The input text is delimited with triple backticks.
Convert your response to 24 hour clock format and Format your response as a JSON object
with "start_time" and "end_time" as the keys. That is, {"start_time" :"","end_time":""}.
If the information isn't present or incomplete information is present, 
set start time as 08:00 and end time as 16:00 as the values.

input text: ```start time is 04 am and end time is 5 pm```

The output I got is: "The start time is 04:00 and the end time is 17:00."

I tried in https://huggingface.co/spaces/tiiuae/falcon-chat. Even though I mentioned the JSON format, it gives output in natural language. And also, the output changes every time we refresh the page. So how to make Falcon 40B to print output in JSON Format?

Answer 1

You can use the Guidance library.

From the "Guaranteeing valid syntax JSON example" section of their readme:

Large language models are great at generating useful outputs, but they are not great at guaranteeing that those outputs follow a specific format. This can cause problems when we want to use the outputs of a language model as input to another system. For example, if we want to use a language model to generate a JSON object, we need to make sure that the output is valid JSON. With guidance we can both accelerate inference speed and ensure that generated JSON is always valid.

And, according to their readme, it provides "easy integration with Hugging Face models".