I have a requirement where I need to do a system design, for providing the masked/test data to the testers. For ex: 10K testers.
Each tester need at-least 50-100 records which needs to stay for 24 hours (i.e., TTL = 24 hours).
Requirements:
Test data needs to be available, fast. Thinking of cache here.
Test data needs to be kept in separate DB. Thinking of data cleaning activities after testing.
If any other (new) application wants to utilize the same functionality, what's the wrapper I need to attach/code to get it work?
- Tried having cache attached with DB to have high throughput.
- DB cleaning activities are hectic. I know this is required but, any better option?
- Every time when the testers want to test (i.e., before moving to PROD environment), data needs to be loaded. How to reduce this activity?
- Can we have a live CRUD operations of data, with less dependent on test/masked data?
I'm thinking to have a separate sandbox which has GraphQL for Query/Mutation, Aerospike as Cache, RDBMS as Database.