The green check that tested the wrong role
The UAT pack looked thorough because it measured the wrong thing. The tester opened the report, saw rows, checked totals, and captured screenshots. The problem is that the signed-in account held workspace Admin rights. According to Microsoft's row-level security guidance, RLS only restricts data access for users with Viewer permissions; it does not apply to workspace Admin, Member, or Contributor roles.
That means the test never exercised the control. Admin, Member, and Contributor roles carry edit permission on the semantic model, and RLS is bypassed for those roles. A finance systems lead reviewing that evidence would not be looking at a control test. They would be looking at a report rendered under an elevated identity.
The production failure that starts with a clean status column
A practical example makes the failure visible. A Power BI report called Segment P&L — Actuals vs Forecast is published to a workspace. The RLS role is named Canada Retail and uses DAX to keep rows where [SalesRegionCode] = 'CA-RETAIL'. The UAT tester signs in as the workspace Admin because that account is already used for publishing, data refreshes, and last-minute fixes.
The evidence pack shows the report loads, total rows reconcile to the GL, and a screenshot is marked green. The sign-off email says row-level security was verified. In production, the audience group is provisioned as workspace Members so the report can be shared quickly. That role grants Write permission on the semantic model. Under that permission, RLS is not enforced and the user can see all data in the semantic model. The first refresh opens the Canadian retail segment rows to every member of that workspace group—with no error, no alert, and a control file that looks clean enough to satisfy an auditor.
Test as role is not the same as signing in as the role
Power BI offers a Test as role feature, and it feels like a shortcut. It is not a substitute for a viewer-identity test. The feature uses the tester's own identity when evaluating dynamic RLS expressions. It simulates role membership but does not fully replicate the authentication context of another user.
A tester who holds Write permission on the semantic model is still not the same as a production Read-only viewer. The test may pass while production identity resolution, group membership, or UPN mapping fails in a way that never shows up in the simulated role screen.
Why the Viewer role is the only valid test
RLS enforcement depends on semantic model permissions. Users with Read permission, or Read and Build permissions, have RLS enforced. Users with Write permission see all data and RLS is not enforced. Write permission can be granted through Contributor, Member, or Admin workspace roles, or through per-item Write permission on the semantic model.
When RLS is defined, users with only Read or Build permission cannot read data unless they belong to one of the RLS roles. Users with Write permission can read all data regardless of whether they belong to any RLS role. So a UAT sign-off under a Writer identity proves the report renders, but it does not prove that the row filter protects restricted financial data.
A validation sequence that catches the bypass
RLS validation should use the minimum-authority production identity. The sequence is:
1. Create a test account or security group that has only the production Viewer permission.
2. Remove workspace Admin, Member, Contributor, and semantic model Write permission from the test identity before opening the report.
3. Publish the report through the same workspace or app path that a production viewer would use.
4. Sign in as the test Viewer—not as an admin using Test as role—and open the report.
5. Confirm the expected rows are visible and, just as important, that rows outside the segment are absent.
6. Repeat for each RLS role with separate non-elevated test identities.
7. Save evidence with the signed-in role visible in the screenshot, not only the report page.
Where the validation still has a limitation
A limitation of this approach is that even a Viewer-role test can miss production identity drift. If the production audience is later moved into a workspace Admin, Member, or Contributor group for convenience, RLS will again be bypassed no matter how clean the original UAT evidence was. Test as role also does not fully replicate authentication context, so dynamic RLS expressions used by a production account may still resolve differently.
A realistic failure occurs when a monthly UAT evidence pack is reused for several version releases without re-checking workspace roles. The report version changes, but the permission drift does not trigger an alert. Security validation is not a one-time sign-off. It must be repeated when workspace roles, app audience groups, or semantic model permissions change.
The failures a successful macro won't catch
The same logic applies outside Power BI. A macro can validate file totals and flag exceptions, but if it runs under an admin network path or a service account that bypasses row filters, it will report a clean result while the control was never exercised. The reusable lesson is to validate under the identity that will hit the system in production, not the identity that is easiest to use.
Sources
Practical checklist
- Run every RLS test under a non-elevated account with only Viewer permission on the workspace or app.
- Remove Admin, Member, Contributor, and semantic model Write permissions from the test identity before sign-off.
- Avoid using Test as role as the only evidence; confirm the actual production authentication context.
- Include the signed-in role in UAT screenshots, not just the filtered report page.
- Verify absence of out-of-segment rows, not just presence of expected rows.
- Re-test after any workspace role, audience group, or semantic model permission change.
- Archive UAT evidence with the permission matrix and accessed date.