t***@gmail.com
2009-06-10 20:18:10 UTC
i'm loading data at maybe 250,000 lines per file using IMPORT and it's taking FOREVER.. i realize LOAD is supposed to be faster. but it doesn't seem to preserve the order of my source data.
i did a test where i LOADed 2 files (test1, test2), each containing 10 lines into a table.. they both worked perfectly (preserved data order, primary key generated correctly).. then i added another 200 records to test2 and LOADed that into the same table. then sampled my data.. surprise! after the first twenty entries (test1 and original test2) i expected to see test2's first entry in row 21 of my table but it was actually the sixth from last entry in test2! it filled in the last six entries as the first entries in the LOAD then continued to load the rest.. what gives?
i did a test where i LOADed 2 files (test1, test2), each containing 10 lines into a table.. they both worked perfectly (preserved data order, primary key generated correctly).. then i added another 200 records to test2 and LOADed that into the same table. then sampled my data.. surprise! after the first twenty entries (test1 and original test2) i expected to see test2's first entry in row 21 of my table but it was actually the sixth from last entry in test2! it filled in the last six entries as the first entries in the LOAD then continued to load the rest.. what gives?