Robust SQL for copying large amount of data between columns

49 views Asked by At

I'm working with a database that stores files as ordinary varbinary(max) type columns. I want to change the table to use FILESTREAM instead. I need to write a script to create the new FILESTREAM column and copy over the old data into it. This will run on databases that can potentially have thousands of files comprising hundreds of GBs of data.

I need the script to be able to handle errors and interruptions such as disconnects, memory issues, other crashes, etc. and be able to run it again and continue without loss of data.

-- Rename old column and create a new column with the old name.
-- Check if the temporary renamed column exists in case this is
-- resuming from a failed prior attempt.
IF NOT EXISTS (SELECT 1 
               FROM sys.columns 
               WHERE [name] = 'old_DataColumn' AND [object_id] = OBJECT_ID('dbo.TableWithData'))
BEGIN
    EXEC sp_rename 'dbo.TableWithData.DataColumn', 'old_DataColumn', 'COLUMN';

    ALTER TABLE [TableWithData]
    ADD [DataColumn] VARBINARY(MAX) FILESTREAM NULL;
END

DECLARE @Id UNIQUEIDENTIFIER;

DECLARE [DataTransferCursor] CURSOR LOCAL FOR 
    SELECT [Id] 
    FROM [TableWithData] 
    WHERE [DataColumn] IS NULL AND [old_DataColumn] IS NOT NULL;
OPEN [DataTransferCursor];

FETCH NEXT FROM [DataTransferCursor] INTO @Id;
WHILE @@FETCH_STATUS = 0
BEGIN
    UPDATE [TableWithData]
    SET [DataColumn] = [old_DataColumn]
    WHERE [Id] = @Id;

    FETCH NEXT FROM [DataTransferCursor] INTO @Id;
END

CLOSE [DataTransferCursor];
DEALLOCATE [DataTransferCursor];

-- Do not drop old column until all data has been copied.
IF NOT EXISTS(SELECT 1 
              FROM [TableWithData] 
              WHERE [DataColumn] IS NULL AND [old_DataColumn] IS NOT NULL)
BEGIN
    ALTER TABLE [TableWithData]
    DROP COLUMN [old_DataColumn];
END

Is the cursor query and the check before dropping the column enough to prevent dropping the column prematurely? Is there possibly a more efficient way of achieving this?

Also, are the CLOSE and DEALLOCATE actions on the cursor necessary for local cursors?

0

There are 0 answers