What is the difference between Owner role vs Custom role?
Our service uses API to interact with Google Cloud. The algorithm is:
The app needs to create a dataset first by calling https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets/insert. This requires bigquery.datasets.create permission.
After the dataset is created, the app creates tables with https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/insert request. Required permission - bigquery.tables.create.
Then the app can insert data into tables: https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll. Required permission: bigquery.tables.updateData.
On each refresh of the same data source, the app has to remove all tables and re-create them. Thus, another required permission is bigquery.tables.delete.
One predefined IAM role covers all these permissions: bigquery.dataOwner. And documentation states the following:
If a user has bigquery.datasets.create permissions, when that user creates a dataset, they are granted bigquery.dataOwner access to it. bigquery.dataOwner access gives the user the ability to copy tables and partitions in the dataset, but access to the destination dataset is required unless the user also created the destination dataset.
This means custom roles with only bigquery.datasets.create permission should work.