Monday, March 12, 2012

fastest way to deduplicate a list

Im trying to dedupe a table with only one field on it. The table has
40 million records in it. What is the fastest way?

1) create a table with a unque constraint on it insert into that
table?

2) create a table without a unique constraint on it and use insert
into table select distinct un from table2?

3) another way?

MichaelMichael Evanchik (mre224@.yahoo.com) writes:

Quote:

Originally Posted by

Im trying to dedupe a table with only one field on it. The table has
40 million records in it. What is the fastest way?
>
1) create a table with a unque constraint on it insert into that
table?


I assume that you would use the IGNORE_DUP_KEY option? Else the scheme
wouldn't work. That could very well be the fastest method.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx

No comments:

Post a Comment