Reservation Savings not visible in v14 Fabric Deployment

[anshu2813]

I have deployed v14 of Finops toolkit solution using both ADX and Fabric as Finops Hub.
Since I have started using v14 with ADX in the solution now with the reingestion of the cost export I am able to see the savings from Reservations that we are making.

We also wanted to try the fabric deployment of this solution instead of using ADX Cluster from cost point of view.

Couple of things which I noticed different is when I choose the deployment with Fabric then the ingestion_ExecuteETL pipeline in ForEach Activity Batch count setting values goes to 256 (Is it because I enter 256 as Fabric SKU where we have our Event house deployed) not sure why but when i checked in my ADX deployment of solution I saw that the batch count is set as 1 in the same pipeline. After updating the pipeline I could see the data getting ingested in Fabric deployment. Also we are using fabric capacity from central team instead of deploying fabric sku just for us.

I also see that in the Fabric deployment sometimes the pipeline ingestion_ETL_dataexplorer gets failed due to the activity called Unit Capacity is available as below. Not sure why this is happening is it due to the fabric capacity issue?

Also one most important thing for us which we cannot see when we setup Rate Optimization powerbi report using Fabric Eventhouse URI is the savings from Reservations which is not visible there. When I look at the table then I can see that the Effective Cost in fabric deployment is same as List Cost (Higher than Contracted Cost) whereas in ADX I could see that Effective Cost is less than contracted Cost for the rows where the commitment disount type is Reservation which helps in calculating savings from Reservations.

[RolandKrummenacher]

Thanks for the detailed write-up, @anshu2813 — and for the ADX-vs-Fabric comparison, that's really useful. You've actually flagged three separate things here, so let me take them in turn.

1. Batch count = 256 in ingestion_ExecuteETL (Fabric only). Confirmed — on a Fabric deployment that ForEach batch count comes from the fabricCapacityUnits parameter (the number in your Fabric SKU), so entering 256 gives a batch count of 256. On ADX it's derived from the cluster SKU and stays small (your 1). On a shared central capacity, 256 parallel ingestions can overrun the Eventhouse, which is likely why ingestion failed until you lowered it. Lowering it was the right move.

2. ingestion_ETL_dataexplorer failing at "Until Capacity Is Available". That activity waits for free ingestion capacity on the Eventhouse and times out if none frees up. On shared capacity, contention/throttling makes this surface intermittently — consistent with item 1.

3. Reservation savings not visible in Fabric (the important one). Here I don't want to guess. What you describe — EffectiveCost == ListCost and higher than ContractedCost in Fabric, while ADX shows EffectiveCost < ContractedCost — is specific: ContractedCost looks correct (so the price data appears present), and the anomaly is that EffectiveCost is too high in Fabric. The same transform logic runs on both engines, so this points at the data differing between the two ingestions rather than the report itself. To pin it down, could you check — for one affected Reservation row, side by side in ADX and Fabric:

• ListUnitPrice, ContractedUnitPrice, x_EffectiveUnitPrice and ListCost, ContractedCost, EffectiveCost.
• Prices_final_v1_2 | count and whether it has recent data in the Fabric Eventhouse.
• After lowering the batch count, did the full reingestion complete for all months, or could some Reservation/amortized rows still be missing from the failed runs?

Since these are really three different problems (and item 3 is the one blocking you), it would help us a lot if you opened item 3 as its own issue with the row-level numbers above — that keeps the savings investigation from getting tangled with the Fabric batch-count/capacity tuning.

#needs-info

[RolandKrummenacher]

🔍 Root-cause analysis (engineering)

Verdict: needs-info. Three separate reports bundled. Only item 1 is code-confirmed; the headline savings issue (item 3) is not root-caused — an earlier "missing Prices → cost collapse" hypothesis is contradicted by the reported numbers and must not be asserted.

Item 1 — batch count (code-confirmed, arguably by-design): ingestion_ExecuteETL ForEach batchCount: dataExplorerIngestionCapacity (app.bicep:1775) → useFabric ? fabricCapacityUnits : … (:216-218); fabricCapacityUnits is the raw F-SKU number, documented as the parallelization control (:114-117). So batch count == the SKU number by design; on shared capacity this over-parallelizes. → candidate improvement (clamp / decouple from raw CU), not clearly a bug.

Item 2 — capacity-wait failures: the .show capacity Until-loops (:842, :1280) time out under contention on shared capacity. Plausible; not deeply investigated.

Item 3 — reservation savings (UNCONFIRMED): Reporter sees Fabric EffectiveCost == ListCost > ContractedCost; ADX EffectiveCost < ContractedCost. Because ContractedCost is correctly lower, the price data is likely present — so "Prices_final_v1_2 not ingested → Effective/Contracted/List collapse to equal" does not fit (that would corrupt ContractedCost too). The anomaly is specifically an inflated EffectiveCost in Fabric. Same transform on both engines → most likely an incomplete/partial Fabric ingestion (downstream of items 1–2) or an Eventhouse-specific data difference. Needs row-level ADX-vs-Fabric data to confirm; do not assert a cause.

Action: needs-info; request splitting item 3 into its own issue with row-level numbers.

[anshu2813]

@RolandKrummenacher thank you for the response on the issue.

For Item -1 lowering the batch count to 1 works perfectly and there is no issue with the ingestion_ExecuteETL pipeline. I wanted to highlight this here so that maybe we can keep the batch count to 1 all the time even if the fabric deployment sku is more 2 capacity unit. Fabric 256 sku is quite expensive so deloying Fabric just for managing finops toolkit can be quite expensive. This is just a suggestion but not something which is breaking changes.

For item 2 ingestion_ETL_dataexplorer failing at "Until Capacity Is Available". Maybe you are right, does this mean we should not have share fabric capacity for this and have a separate fabric deployment?

Can we use this for only item 3 as that is a priority for us?

For Item 3
ListUnitPrice, ContractedUnitPrice, x_EffectiveUnitPrice and ListCost, ContractedCost, EffectiveCost.

I am identifying a row using a composite key of reservation (CommitmentDiscountId), resource (ResourceId), and exact usage time slice (ChargePeriodStart + ChargePeriodEnd).

In ADX for one RI

"ListUnitPrice": 0.08203725858827549, "ContractedUnitPrice": 0.07055204238591692, "x_EffectiveUnitPrice": 0.04101481991955477, "ListCost": 1.8868569475303363, "ContractedCost": 1.622696974876089, "EffectiveCost": 1.103897472206849
In Fabric

"ListUnitPrice": 0.04101481991955477, "ContractedUnitPrice": 0.04101481991955477, "x_EffectiveUnitPrice": 0.04101481991955477, "ListCost": 1.103897472206849, "ContractedCost": 0.9433408581497597, "EffectiveCost": 1.103897472206849

Prices_final_v1_2 | count and whether it has recent data in the Fabric Eventhouse.
Yes it has the price sheet which was ingested for the month of May 2026

After lowering the batch count, did the full reingestion complete for all months, or could some Reservation/amortized rows still be missing from the failed runs?

Yes once I lowered the batch count to 1 the ingestion was successfull after triggering the export again and I could see the cost data getting ingested in fabric.

[RolandKrummenacher]

Thanks @anshu2813 — these side-by-side numbers are exactly what we needed, and they're very telling.

Quick answers on the first two:

• Item 1 (batch count): noted, and it's a reasonable suggestion — sizing ingestion concurrency off the raw Fabric SKU isn't ideal on shared capacity. I'd suggest tracking that as a separate improvement so it doesn't block the savings work; happy to keep this issue focused on item 3 as you asked.
• Item 2 (capacity waits): on shared central capacity, ingestion gets throttled by other workloads, which is what that activity waits on — a dedicated/temporary capacity or off-peak runs reduce it, but it isn't strictly required.

Item 3 (the savings) — here's what your numbers show. In Fabric, ListUnitPrice == ContractedUnitPrice == x_EffectiveUnitPrice (all 0.04101). That's the exact signature of the price-sheet reconstruction fallback firing: when the transform can't find a matching row in Prices_final_v1_2 for a reservation usage row, it overwrites the real List/Contracted prices with the effective price — which erases the visible savings. In ADX the same rows kept their real prices (0.082 / 0.0705), so there the lookup matched.

The transform logic is identical between ADX and Fabric — there's no engine-specific code here. So the difference is in the data: the price lookup matched in ADX but missed in Fabric. The lookup key for reservation rows is:

lower( x_BillingProfileId + <YYYY-MM of ChargePeriodStart> + x_SkuMeterId + x_SkuOfferId )

matched against the price sheet's x_SkuPriceType == 'Consumption' rows.

To find which part of the key isn't matching in Fabric, for that same affected row:

1. EA or MCA? And the raw x_SkuOfferId on the cost row, plus whether the ingested price sheet has rows with that same OfferID. (Most common cause — if offer IDs differ or are blank on one side, every reservation row misses.)
2. The raw x_BillingProfileId on the cost row vs the price sheet's BillingProfileId / EnrollmentNumber.
3. The usage ChargePeriodStart month vs the price sheet's effective month — you mentioned a May 2026 price sheet; are the affected usage rows also May 2026?

One more thing that doesn't add up and would help a lot: your Fabric ContractedCost (0.9433) is lower than EffectiveCost (1.1039), but once the fallback fires the transform should recompute ContractedCost to ≈ ListCost ≈ 1.1039 — so 0.9433 shouldn't appear on current code. Could you confirm:
4. The deployed script version on your Fabric Eventhouse (to rule out a pre-v14 version), and the raw ContractedCost, PricingQuantity, x_PricingBlockSize on that row.

With those we can pinpoint whether it's an offer-id / billing-profile key mismatch, a price-sheet month gap, or a version skew.

#needs-info