Context: I have a lib that uses Cereal to serialize/deserialize a json file, it is written in C++ and with a Python wrapper to help it being called from the pipeline.
Here are some scenarios showing how I use this lib: (I always use the pipeline to SERIALIZE(successful) the data and stored in the cloud)
- (Successful) Downloaded it to Mac (local) and use the same lib(C++) to deserialize.
- (Successful) In the pipeline which runs some modules (Python wrapper) that downloads the file to a server and eventually triggers the lib to deserialize the file.
- (Successful) I can also trigger the same module (Python wrapper) in another server for a test run, which also triggers the lib to deserialize the file.
- (Failed) Recently without any code changes in the lib that (2) starts to fail on deserializing, while serialize still works and it also succeeds when using that serialized data in (1)(3).
Example of the lib
struct MyStruct {
int version = 3;
std::string name = "";
time_t creationTime = 0;
}
template <class Archive>
void MyStruct::serialize(Archive& archive) {
// This is to avoid using cereal_class_version.
archive(::cereal::make_nvp("version", version));
// Do some version checking
archive(
::cereal::make_nvp("name", name),
::cereal::make_nvp("creation_time", creationTime));
}
void MyStruct::readFromFile(const std::string& filepath) {
try {
// Read header.
std::ifstream file(filepath);
LOG(INFO) << fmt::format("Loading header...: {}", filepath);
// This is to prevent Cereal from creating extra {} layer.
cereal::JSONInputArchive iarchive(file);
this->serialize(iarchive);
} catch (const cereal::RapidJSONException& ex) {
LOG(WARNING) << fmt::format(
"Failed to parse header: {}, RapidJSONException exception: {}",
filepath,
ex.what());
} catch (const std::exception& ex) {
LOG(WARNING) << fmt::format(
"Failed to parse header: {}, std exception: {}",
filepath,
ex.what());
}
}
void MyStruct::writeToFile(
const std::string& filepath) const {
try {
// Write header.
LOG(INFO) << fmt::format("Exporting header file: {}", filepath);
std::ofstream file(filepath);
cereal::JSONOutputArchive oarchive(file);
MyStruct metadata = *this;
metadata.serialize(oarchive);
} catch (const cereal::RapidJSONException& ex) {
LOG(WARNING) << fmt::format(
"Failed to write header: {}, RapidJSONException exception: {}",
filepath,
ex.what());
} catch (const std::exception& ex) {
LOG(WARNING) << fmt::format(
"Failed to write header: {}, std exception: {}",
filepath,
ex.what());
}
}
Exported json file
{
"version": 1,
"name": "MyJsonTestData",
"creation_time": 1695990193,
}
Error message:
XXXX 15:34:12.581558 1185293 MyStruct.cpp:72] Loading header...: /tmp/tmpm6f_itod/temp_in/ABC/myHeader.myExtensionName
*** Aborted at 1696026852 (Unix time, try 'date -d @1696026852') ***
*** Signal 11 (SIGSEGV) (0x20) received by PID 1185293 (pthread TID 0x7f1215753500) (linux TID 1185293) (code: address not mapped to object), stack trace: ***
@ 0000000028fb34cd (unknown)
@ 000000000004459f (unknown)
@ 000000004df9c287 void MyStruct::serialize<cereal::JSONInputArchive>(cereal::JSONInputArchive&)
@ 000000004df9bd5f MyStruct::readFromFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
Two questions:
a. (4) failed while (1)(3) still works, it seems like the server environment in the pipeline might be the root cause (correct me if I am wrong). In such case, what specifically should I look for, Cereal version? Any other suggestions?
b. From the error message I can tell that for some reason the try-catch isn't working properly, is it due to logging issue, or something is wrong with my exception handling for this Cereal case?
Thanks in advance.