I'm working on a project where I need to process temporary JSON data based on user-generated queries, and I'm looking for advice on an efficient and secure method to implement this.
Context: Goal: Allow users to submit queries that temporarily process JSON data. I retrieve the data from a separate endpoint and want to allow the user to manipulate it. Ideally, I'd accept a json query like mongo and execute it in memory against the json response from my other endpoint. I do not have any restrictions on language or tools to use.
Security Concern: I want to avoid the risks associated with SQL injection and ensure that users can only access and manipulate their session-specific data. Hence I think its risky to dump data directly into mongo and query against it if I have concurrent requests.
Considered Approaches:
- Databases: I considered using MongoDB with user-specific collections and PostgreSQL with temporary tables, creating a new user with restricted permissions for each session. However, I'm concerned about the scalability and performance of these approaches.
- In memory: I considered using a tool like lodash but it doesn't seem to accept json queries so that might be difficult to accept a users query.
- I considered using a more graphql like approach where I would accept a users request in a graphql format and query against the json. However, this is limited in that it could restrict fields but I'd also like access to more advanced query options like aggregating results.
Ultimately I'm curious if anyone has and ideas on how this could best be handled. Any tools or alternative approaches to handle this problem that I haven't considered.