Moving a Catalog That Cannot Be Moved

The problem

Every non-production Unity Catalog catalog lived on a single legacy Azure storage account — test and UAT data sharing one blast radius. They needed to move onto environment-segregated accounts so that isolation was a property of the infrastructure rather than a convention people remembered to follow.

That sounds like a configuration change. It isn't.

A Unity Catalog catalog's storage root is immutable. There is no ALTER CATALOG ... SET MANAGED LOCATION. And because the tables were MANAGED, the data physically lives under the catalog root — so the root is not a pointer you can repoint. Moving a catalog means rebuilding it.

Rebuilding it is where the real difficulty starts, because a catalog is not just its tables. Every one of these had to survive the move exactly:

• Per-table and per-schema owners
• Workspace bindings and isolation mode
• Grants at every level
• Column-level masking functions
• Volumes

Get any of them wrong and you have a catalog that looks correct and quietly isn't — the worst possible outcome in a governed environment, because nothing alerts you.

The playbook

The work was designing something repeatable rather than performing one careful migration. Six steps, five of them reversible.

1. Capture every source attribute first. Owner, isolation mode, bindings, grants, per-schema root location, table owners, masks. Before touching anything, because after the drop there is nothing left to read them from.

2. Build a parallel <catalog>_new on the target account, creating each schema with an explicit MANAGED LOCATION.

This is the detail that decides whether the migration works. A plain CREATE SCHEMA silently drops data under __unitystorage/ and permanently orphans the storage. No error, no warning — the schema is created, the data lands in the wrong place, and the storage is unrecoverable. It is the kind of failure you only learn about by having caused it.

3. DEEP CLONE every managed table, which preserves Delta history and the MANAGED type — then restore each table's original owner, because cloning makes the operator the owner. A migration that silently reassigns ownership of every table to whoever ran it is a governance incident, not a migration.

4. Recreate what clone doesn't cover: volumes, empty schemas, masking functions, views. DEEP CLONE is not a catalog-level operation, and the gaps are exactly the objects most likely to be forgotten.

5. Validate before the point of no return. Row counts, physical paths, schema roots, grants — then stop for explicit human sign-off.

6. DROP CATALOG ... CASCADE, rename _new to the real name via REST PATCH (SQL rename is unsupported), replay the captured attributes, and re-point mask references broken by the rename.

The gate

Step 6 is irreversible. Everything before it can be abandoned with nothing lost but time.

So the playbook stops there and requires a person to say yes — not a confirmation prompt, an actual review of the validation output against the captured source attributes.

This is the same principle as designing a system to fail visibly rather than silently: automation should carry you right up to the edge of the irreversible action and then hand back control. The value is not that a human does the drop. It is that a human confirms the five reversible steps produced what they were supposed to, while abandoning is still free.

The runbook is the deliverable

The tooling is shell scripts, but the artefact that matters is the runbook — a living document that captures the sequence, the traps, and the exact commands, maintained alongside the scripts rather than written up afterwards.

That distinction matters. A migration performed once by someone who knows what they are doing produces a migrated catalog. A migration performed against a written playbook produces a migrated catalog and the ability to do it again — in the next environment, by a different person, after the person who designed it has moved on. A separate handover document takes it from working tooling to something an operations team owns.

What transfers

1. Check whether the thing you want to change is immutable before designing around the assumption that it isn't. The absence of an ALTER statement is the design telling you something.
2. Capture state before you destroy it. Owners, grants, and bindings are unreadable after the drop, and they are exactly what nobody remembers to record.
3. Watch for silent defaults. CREATE SCHEMA without an explicit location does not fail — it does something wrong quietly and permanently. Silent success is worse than a loud error.
4. Know who ends up owning what. Clone, copy, and restore operations frequently reassign ownership to the operator. In a governed catalog that is a real finding.
5. Put the human gate immediately before the irreversible step, not at the start. Approval given before the evidence exists is a formality.
6. Write the runbook while doing it, not afterwards. The traps you document at the moment you hit them are the ones you would otherwise forget.