Timeout issue for http connector and web activity on adf

Question

Timeout issue for http connector and web activity

Web activity and http connector on adf

We have tried loading data through Copy Activity using REST API with Json data some columns are getting skipped which is having no data at its first row. We have also tried REST API with cv data but it's throwing error. We have tried using Web Activity but its payload size is 4MB, so it is getting failed with timeout issue. We have tried using HTTP endpoint but its payload size is 0.5 MB, so it is also getting failed with timeout issue

Answer 1

• In Mapping settings, Toggle on the advanced editor and give the respective value in collection reference to cross apply the value for nested Json data. Below is the approach.

• Rest connector is used in source dataset. Source Json API is taken as in below image.

• Then Sink dataset is created for Azure SQL database. Once the pipeline is run, few columns are not copied to database.

• Therefore, In Mapping settings of copy actvity, 1. Schema is imported 2. Advanced editor is turned on 3. Collection reference given.

• When pipeline is run after the above changes, all columns are copied in SQL database.