Despite the availability of newer, more cost-effective AWS instances such as c6.size and c7.size, the use of these instances with Databricks results in higher prices compared to our current instances, c4.size and c5.size.
Cost of aws instances:
Cost of databricks DBU:
attached screenshots for reference
Even if I opt for cost-effective instances, the Databricks Unit (DBU) pricing remains higher for those instances. I would like to understand why it is like that.
Apologies if you feel this as a dumb question.
Expectation is that what is the cost criteria despite using the cost-effective one's.