Output truncation in colab

72 Views Asked by At

Here is my code:

model_name_or_path = "juanjgit/orca_mini_3B-GGUF"
model_basename = "orca-mini-3b.q4_0.gguf"
model_path = hf_hub_download(repo_id=model_name_or_path, filename=model_basename)

llm = LlamaCpp(
    model_path=model_path,
    max_tokens=2048,
    n_ctx = 2048,
    n_gpu_layers = 40,
    n_batch = 512
)

text_splitter = TokenTextSplitter(separator=" ", chunk_size=1024 , chunk_overlap=20)

metadata_extractor = MetadataExtractor(
    extractors=[
        TitleExtractor(nodes=1, llm=llm),
        KeywordExtractor(keywords=10, llm=llm)
    ],
)

node_parser = SimpleNodeParser(
    text_splitter=text_splitter,
    metadata_extractor=metadata_extractor,
)

docs = SimpleDirectoryReader(input_files=["/content/Book.pdf"]).load_data()
print(f"loaded book with {len(docs)} pages")

nodes = node_parser.get_nodes_from_documents(docs)

when i'm printing:

print(dog_nodes[4])

the result is:

Node ID: 12a31a04-b6f8-416a-bd01-56a4efb37f51 Text: Prologue 25 January 2022, 4 p.m. ‘Joining back on 1st Feb’. That was the subject line of the email I had shot off to the board of directors that cold January evening. Earlier that month, I had been coerced into going on a voluntary leave of absence from BharatPe, a company worth US$3 billion (over Rs 20,000 crore) that I had built painstakingly ...

if i convert the data type from list to dataframe,

df.loc[df.index[4]]

the reult is:

Prologue\n25 January 2022, 4 p.m.\n‘Joining back on 1st Feb’. That was the subject line of the email I had shot\noff to the board of directors that cold January evening. Earlier that month, I\nhad been coerced into going on a voluntary leave of absence from BharatPe,\na company worth US$3 billion (over Rs 20,000 crore) that I had built\npainstakingly at an unprecedented pace over the last three and a half years\nas its founder and managing director .\nThe whole of January had been a blur—I was hit relentlessly by one\ncontroversy after another . What started with a ransom call became a leaked\naudio, and then became leaked legal notices and arbitrary statements by\nKotak bank. While the nation was enjoying Shark T ank India and\ncelebrating the new wave of entrepreneurship that was taking the country\nand millions of TV screens by storm every weeknight from 9–10 p.m., I\nwas personally fighting a bloody board battle aimed at wresting control of\nBharatPe from me. During the...

In list data type, output is truncated real soon, while in df it's little later but still got truncated. I want full result. What should i do?

0

There are 0 best solutions below