Bulk-migrating Dataflow Gen1 to Gen2 (CI/CD) with Azure CLI

Microsoft put Dataflow Gen1 into a legacy state in April 2026. Gen1 dataflows still run and refresh, but Microsoft will direct new investment to Dataflow Gen2 (CI/CD).

For three Gen1 dataflows, use the UI: right-click and select Save as Dataflow Gen2. For forty dataflows across six workspaces, use the REST API and track each conversion.

The preview API works through az rest, so you do not need to write an application. I have used it on client tenants for the past few weeks. This post covers the workflow, four API problems, and the deployment-pipeline work that the conversion leaves behind.

The API creates a separate Gen2 item

saveAsNativeArtifact creates a Dataflow Gen2 (CI/CD) item in a target workspace from a Gen1 dataflow. The portal uses the same operation for Save as Dataflow Gen2.

The operation leaves the Gen1 dataflow in place and unchanged. After conversion, you have two artifacts. You can validate the Gen2 item before you decide when to retire the Gen1 source.

One workflow uses two API audiences

The workflow spans two APIs, each with its own token audience.

If you omit --resource, az rest may acquire a token for the wrong audience. The resulting 401 does not identify the audience mismatch.

Set the constants once:

$sourceWorkspaceId = '<source-workspace-guid>'
$targetWorkspaceId = '<target-workspace-guid>'
$pbiResource       = 'https://analysis.windows.net/powerbi/api'
$fabricResource    = 'https://api.fabric.microsoft.com'

Inventory the sources

The Power BI dataflows endpoint returns a generation property for each item. Filter for generation 1 because this workflow supports Gen1 sources.

$source = az rest --method get `
  --resource $pbiResource `
  --url "https://api.powerbi.com/v1.0/myorg/groups/$sourceWorkspaceId/dataflows" `
  -o json | ConvertFrom-Json

$source.value |
  Where-Object { $_.generation -eq 1 } |
  Select-Object name, objectId, generation, configuredBy, modelUrl

Check two fields beyond the name and ID:

• A modelUrl that points to custom storage indicates that the workspace uses bring-your-own storage. Review that setup before you create target items.
• configuredBy identifies the owner. The account running the migration needs Contributor or Admin access to the source workspace. On client tenants, the original owner may have left the organization.

Then retrieve the data sources and upstream dependencies for each candidate:

$sourceDataflowId = $sourceDataflow.objectId

az rest --method get `
  --resource $pbiResource `
  --url "https://api.powerbi.com/v1.0/myorg/groups/$sourceWorkspaceId/dataflows/$sourceDataflowId/datasources" `
  -o json

az rest --method get `
  --resource $pbiResource `
  --url "https://api.powerbi.com/v1.0/myorg/groups/$sourceWorkspaceId/dataflows/$sourceDataflowId/upstreamDataflows" `
  -o json

Use upstream dependencies to set the conversion order. If dataflow B consumes linked entities from dataflow A, migrate A first.

Data-source inventory also identifies remediation work. In my migrations, Snowflake and SharePoint List sources often required connection changes after conversion.

Four problems I hit

1. Repeating the call creates duplicates

If you run saveAsNativeArtifact twice with the same payload, the service creates two dataflows with the same display name. It does not return a conflict.

A timeout or ambiguous response may tempt you to retry. Check both target inventories first:

$legacyTarget = az rest --method get `
  --resource $pbiResource `
  --url "https://api.powerbi.com/v1.0/myorg/groups/$targetWorkspaceId/dataflows" `
  -o json | ConvertFrom-Json

$fabricTarget = az rest --method get `
  --resource $fabricResource `
  --url "https://api.fabric.microsoft.com/v1/workspaces/$targetWorkspaceId/items?type=Dataflow" `
  -o json | ConvertFrom-Json

$nameExists = @($legacyTarget.value | Where-Object { $_.name -eq $targetDataflowName }).Count -gt 0 -or
              @($fabricTarget.value | Where-Object { $_.displayName -eq $targetDataflowName }).Count -gt 0

if ($nameExists) {
  throw "The target already contains '$targetDataflowName'. Refusing a non-idempotent save-as operation."
}

Check both endpoints because they can return different sets of items. A match in either inventory should stop the operation.

2. Ampersands can trigger an unhelpful error

In my tests, a displayName containing & produced this response:

InvalidRequest: Unexpected dataflow error

The message does not mention the name. I checked permissions, capacity state, and workspace settings before I connected the failures to ampersands in names such as “Sales & Marketing.”

Sanitize each name before sending the request:

$targetDataflowName = $sourceDataflow.name -replace '\s*&\s*', ' and '
$targetDataflowName = $targetDataflowName -replace '\s+', ' '
$targetDataflowName = $targetDataflowName.Trim()

Show the proposed names to a person before creating anything. The migration plan should record each original and replacement name.

3. ConnectionsUpdateFailed can accompany a successful copy

The response uses its errors array for some non-fatal warnings. ConnectionsUpdateFailed means the service created the artifact but could not finish converting its connections.

The target dataflow and its queries exist. Rebind the connections before the first refresh. Do not retry the save-as operation, because a retry can create a duplicate.

Handle FailedToCopySchedule and SetDataflowOriginFailed the same way: record the warning, verify the target item, and schedule the required remediation.

4. The request has no folderId

You cannot create the copy inside a target-workspace folder. The documented request body has no folder parameter, and the public Fabric item APIs do not support moving an existing dataflow into a folder.

Record the intended folder in the migration plan, then place the items through the Fabric UI.

Run the conversion

Write the request body to a file. Inline JSON quoting through az rest is error-prone on Windows. Also omit -o json from the POST command.

$requestBody = @{
  displayName        = $targetDataflowName
  description        = 'Migrated from Gen1 dataflow via saveAsNativeArtifact'
  includeSchedule    = $false
  targetWorkspaceId  = $targetWorkspaceId
} | ConvertTo-Json -Compress

$requestFile = Join-Path $env:TEMP 'save-as-dataflow.json'
Set-Content -LiteralPath $requestFile -Value $requestBody -Encoding ascii -NoNewline

try {
  $response = az rest --method post `
    --resource $pbiResource `
    --url "https://api.powerbi.com/v1.0/myorg/groups/$sourceWorkspaceId/dataflows/$sourceDataflowId/saveAsNativeArtifact" `
    --headers 'Content-Type=application/json' `
    --body "@$requestFile"

  $migration = $response | ConvertFrom-Json
  $migration | Select-Object `
    @{ Name = 'newId';          Expression = { $_.artifactMetadata.objectId } }, `
    @{ Name = 'name';           Expression = { $_.artifactMetadata.displayName } }, `
    @{ Name = 'provisionState'; Expression = { $_.artifactMetadata.provisionState } }, `
    errors
}
finally {
  Remove-Item -LiteralPath $requestFile -Force -ErrorAction SilentlyContinue
}

I set includeSchedule to false. Configure a schedule after you validate the target connections and complete a manual refresh. This avoids a failed scheduled refresh before the migration team has finished its checks.

Verify the item through the Fabric API

Treat provisionState: Active as one check, then confirm that the target workspace contains the new item:

$newItemId = $migration.artifactMetadata.objectId

az rest --method get `
  --resource $fabricResource `
  --url "https://api.fabric.microsoft.com/v1/workspaces/$targetWorkspaceId/items/$newItemId" `
  -o json | ConvertFrom-Json |
  Select-Object id, displayName, type, workspaceId, description

Rework deployment pipeline rules

Deployment pipeline rules do not apply to Dataflow Gen2 (CI/CD). If your Dev, Test, and Prod pipeline uses data-source or parameter rules to replace a server, database, or destination between stages, the conversion leaves those rules behind.

Dataflow Gen2 stores destinations inside the mashup definition. Deployment rules cannot change them. If you deploy the converted artifact without reviewing its destinations, the Prod copy can keep writing to the Dev lakehouse.

Fabric variable libraries provide the supported replacement for values that the mashup can resolve. Create a variable library in each workspace, reference its variables from the mashup, and let each stage supply its own values:

Variable.ValueOrDefault("$(/**/ConfigLibrary/LakehouseId)", "<dev-fallback-guid>")

This model keeps the mashup script identical across stages. Git stores one artifact, while each workspace supplies its environment-specific values.

The conversion does not translate Gen1 deployment rules into variable references. You must open each migrated dataflow, identify the values that the old rules replaced, and express each supported value as a variable.

Plan around three constraints:

• Each stage workspace needs its own variable library because the library must share a workspace with the Dataflow Gen2 item.
• Variable references work inside mashup.pq; other files cannot resolve them.
• Variable libraries support boolean, datetime, guid, integer, number, and string values.

Scope this work before you start converting dataflows that use deployment pipelines. A dataflow with several environment-specific values may require more time to rework than to convert.

Plan for other settings that do not transfer

The conversion does not copy:

• Scheduled refresh settings
• Incremental refresh settings

Incremental refresh can require substantial manual work. Add policy reconstruction and validation to the migration plan for every source that uses it.

A safe bulk workflow

1. Inventory each Gen1 dataflow in the source workspaces.
2. Inventory the deployment pipeline rules and map each supported value to a variable-library replacement.
3. Build a plan with the source ID, source name, proposed target name, source types, conflict result, upstream dependencies, and status.
4. Get approval for the plan before creating target items.
5. Convert one item at a time in dependency order. Recheck the target name before each POST.
6. Record newId, provisionState, and the errors array in a CSV or JSON report.
7. Retire Gen1 sources through a separate, approved process.

Interpret the result

I packaged the workflow as an agent skill

I wrote an agent skill after repeating these checks across migrations. It keeps the commands beside the operational rules: verify generation, check for conflicts before every write, sanitize names, preserve the source, and wait for approval before triggering a refresh.

View the full skill in the repository.