I'm a new Matillion user so suspect I'm overlooking an obvious answer... I am currently reading a number of identically formatted tables (yearly sales data) and vertically stacking them using a Unite component. As I am exploring the data rather building a pipeline with a specific function I would like to keep as much flexibility as possible. Hence, I would like to select columns after the unite component has run. I appreciate I can easily do such column selections reading the data in at the point of running Table Input components. Am I missing the obvious solution?
Select columns mid Matillion transformation job
202 Views Asked by Dave-C At
1
A humble Rename Component can help with that. Add one immediately after the Unite, and in the column mapping choose Add All. Then you can choose which columns to drop, mid-transformation, using the minus button.
The Rename Component's primary purpose is to add alias names, but it does double up as a good way to slim down the column selection. As a bonus side effect, doing that can improve performance.