DolphinDB: Fail to add columns to a DFS table in the TSDB engine due to out-of-memory issues

Question

I want to add a time column to a DFS table in the TSDB engine by using the command addColumn. The DFS table is stored on disk and is too big to be loaded into memory all at once. However, when I execute update loadTable("dfs://TSDB_Order", "order") set Time = time(OrigTime), an out-of-memory (OOM) error occurs.

Answer 1

If keepDuplicates ≠ LAST, the update operations in the TSDB engine will load all data into memory at once without partitioning pruning, leading to OOM issues. To address this, you can add filtering conditions to the update statement to update one partition at a time.