Below is the script for creating databases and tables:

db1 = database('',VALUE,2016.11.15..2016.11.18)
db2 = database('',HASH,[SYMBOL,10])
db = database('dfs://iot',COMPO,[db1,db2])

schema=table(1:0,`time`device_id`battery_level`battery_status`battery_temperature`bssid`cpu_avg_1min`cpu_avg_5min`cpu_avg_15min`mem_free`mem_used`rssi`ssid,
 [DATETIME,SYMBOL,INT,SYMBOL,DOUBLE,SYMBOL,DOUBLE,DOUBLE,DOUBLE,LONG,LONG,SHORT,SYMBOL])
 db.createPartitionedTable(schema,`readings,`time`device_id)

I want to write data into a partitioned table using MultithreadedTableWriter provided by the DolphinDB C++ API. How can I create a MultithreadedTableWriter object, and set its parameters (especially partitionCol)? Thanks in advance!

vector<COMPRESS_METHOD> compress;
compress.push_back(COMPRESS_DELTA);
compress.push_back(COMPRESS_LZ4);
compress.push_back(COMPRESS_LZ4);

MultithreadedTableWriter writer("127.0.0.1", 8848, "admin", "123456", "dfs://iot", "readings",
    false, false, NULL, 10000, 1, 5, "`time`device_id", &compress);
1

There are 1 best solutions below

0
carbonhydrate On BEST ANSWER

DolphinDB server does not support concurrent writes from multiple threads to the same partition; otherwise, it will result in write failures. Therefore, you need to set the threadCount and partitionCol parameters of the MultithreadedTableWriter object to ensure that data is written to a partition by only one thread.

For a table partitioned by a combination of time (i.e. time) and product identifiers (i.e. device_id), set the following two parameters:

threadCount: the number of partitions based on product identifiers. For the “dfs://iot” database, set the parameter to 10.

partitionCol: the product identifier column name (i.e. device_id).

vector<COMPRESS_METHOD> compress;
compress.push_back(COMPRESS_DELTA);
compress.push_back(COMPRESS_LZ4);
compress.push_back(COMPRESS_DELTA);
compress.push_back(COMPRESS_LZ4);
compress.push_back(COMPRESS_LZ4);
compress.push_back(COMPRESS_LZ4);
compress.push_back(COMPRESS_LZ4);
compress.push_back(COMPRESS_LZ4);
compress.push_back(COMPRESS_LZ4);
compress.push_back(COMPRESS_DELTA);
compress.push_back(COMPRESS_DELTA);
compress.push_back(COMPRESS_LZ4);
compress.push_back(COMPRESS_LZ4);
MultithreadedTableWriter writer("127.0.0.1", 8848, "admin", "123456", "dfs://iot", "readings",false, false, NULL, 10000, 1, 10, "device_id", &compress);