We have a web application that collects data from milling machines. There are several machines connected and per day about 2000 to 3000 records of machine data are collected. From time to time it happens that the server or the database simply stops and no more new data can be written into the database. If a query is then made for the data, the Wakanda server returns an error message:
{"__ERROR":[{"message":"Memory allocation failed","componentSignature":"xbox","errCode":100},{"message":"Cannot load page for index UnixDay.ID of database models","componentSignature":"dbmg","errCode":1076},{"message":"Cannot complete scan on index UnixDay.ID of database models","componentSignature":"dbmg","errCode":1081},{"message":"Query cannot be completed","componentSignature":"dbmg","errCode":1200},{"message":"Query cannot be completed","componentSignature":"dbmg","errCode":1200},{"message":"Query cannot be completed","componentSignature":"dbmg","errCode":1200}]}
We have two locations and operate an independent Wakanda server at each location. The database model and the code basis for the data classes are identical in both locations. However, the problem described above only occurs at one location, and we have never had such a problem at the second location. There the server or the database runs for weeks without an error.
If the database stops, a data error often occurs. Perhaps the data error occurs first and then the server/database stops. The error message is always of the same type, only the affected data class varies.
BACKEND {"ErrorText":"List of deleted records is invalid","ErrorNumber":18,"ProblemType":5,"ErrorLevel":2,"ProblemTypeText":"Problem on the {TableName} data table","TableNum":35,"TableName":"ProductionOrder"}
BACKEND {"ErrorText":"The chained list of deleted records is invalid","ErrorNumber":54,"ProblemType":5,"ErrorLevel":2,"ProblemTypeText":"Problem on the {TableName} data table","TableNum":2,"TableName":"BaseDate"}
In most cases, the problem occurs after 2 to 3 days. If the database has been damaged (error as shown above in the second code box), a database repair must be performed and then everything is ok again. It will also create a new .waIndex file and run again for a few days.
Are there certain constellations of the host server (Windows Server 2012 R2 standard) that could trigger such behavior? What is behind "componentSignature": "xbox"? Does this indicate that Wakanda uses Microsoft Xbox libraries? When I look for the error message, the answers always refer to problems with Xbox libraries.
I appreciate any advice, it's a pretty persistent problem.
I've located the problem and been able to fix it.
The cause is the configuration of the memory for the entire solution (settings.waSettings). If the value for the available memory is too small, the memory will run out over time. If a lot of data is processed and stored, then sooner or later the memory will be full and the Wakanda server will not be able to open the data storage file or the index file anymore or will not be able to finish the write process properly. In this case, the system will stop with the error message described above. If the write operation could not be completed completely, inconsistencies will occur in the data storage file and the database will be damaged. This has to be repaired first, otherwise the Wakanda server will not start at all. I have now set the setting to fixed memory size and give the Wakanda server 8GB memory. This leaves enough space to process the data and the performance is better with more memory. Saving data in the database has been accelerated by a factor of 5 - 6 in some cases. This allows the server to also better handle the release of occupied memory. If the memory is too small, then it has also there problems and does not manage it always enough memory to free again and then the entire server suffocates over the time.
My question has been answered with that.